2022-03-04
md
Free Pascal/Lazarus TThread Gotchas
<-Free Pascal/Lazarus Gotcha

In the second version of my Tasmota backup utility tasmotasbacker (source on GitHub), I added the ability to scan for IoT devices with Tasmota firmware by poking every IP addresses. It made sense, because discovering IoT devices through an MQTT broker would only get a hold of those devices that were connected to the broker. Unfortunately scanning the range of all addresses that could be assigned to IoT devices on my network (192.168.1.100 to 192.168.1.199) takes a long time. Since I wrote the program, I knew that one had to be patient. To help others, I added a spinning cursor and the occasional call to a delay function that would process application messages. That way, the GUI was not completely unresponsive during the scan. But it was (and for the time being it remains) an awful hack. Lately I got a TBA brilliant idea™ and started reworking the code. It was the opportune time to fix the kludge. The obvious solution was to perform the IP address scan in its own thread, just as the mosquitto library wrapper TMQTTConnection by Károly Balogh (chainq) in mosquitto-p runs in its own thread. And that's when things went pear shaped. The GUI was not responsive at all, there were memory leaks, event handlers were not called; it was just a mess. Investigating the problem within the tasmotasbacker was not all that easy, so I created a simpler application and set about trying to figure out what was going on. About halfway through, I got a nagging feeling that the problems were entirely my fault. I decided to suspend my investigation, cleaned up my "speelunking" application, and started this note. On going back to tasmotasbacker, it became painfully obvious that I was making a really, really stupid mistake. Nevertheless I did learn a few things about the TThread class.

Table of Contents

  1. About the TThread Class
  2. The Vanishing UseCThreads Directive
  3. Use the cmem Unit in Linux
  4. Setting a Thread Instance to nil Once it is Freed
  5. Do not Make a Stupid Mistake
  6. Coming Up

About the TThread Class toc

Threads are sequences of code that can run independently on a system. Threads could run concurrently on multiple cores if the processing unit has that capability or they appear to run concurrently when each thread is allocated a slice of time in quick succession on a single core. As one can easily imagine, there's a lot of complexity behind this concept. An important one of these is the problem of synchronization and exchange of data between threads. This is particularly important in event driven graphic user interfaces which typically run in a separate main thread.

The TThread class is an object wrapper of the Pascal thread functions BeginThread, WaitForThreadTerminate, EndThread etc. These function are actually calls to the thread manager which is different for each supported Free Pascal target system. If I recall, the functions are not the simplest things to use, so the abstraction affored by the TThread class is welcomed. If Tthreads is something new or if you need a refresher, read Threads with Lazarus then go on to Multithreaded Application Tutorial in the Free Pascal Wiki. There is another page on the same site Code Snippet demonstrating Thread programming.

The Vanishing UseCThreads Directive toc

The Multithreaded Application Tutorial, says ... all new applications created by the IDE have already the following code in their .lpr file:

uses {$IFDEF UNIX}{$IFDEF UseCThreads} cthreads, cmem, // the c memory manager is on some systems much faster for multi-threading {$ENDIF}{$ENDIF}

I am not sure that was the case before. Looking at some of my Lazarus projects dating from June of 2021, it is clear that cmem was not present in the use clause generated by the IDE.

uses {$IFDEF UNIX}{$IFDEF UseCThreads} cthreads, {$ENDIF}{$ENDIF}

With the current version of Lazarus (2.2.0 (rev lazarus_2_2_0-102-g66098fdc9a) FPC 3.2.3 x86_64-linux-gtk2), the automatically generated use clause is different.

uses {$IFDEF UNIX} cthreads, {$ENDIF} {$IFDEF HASAMIGA} athreads, {$ENDIF}

Custom options in Lazarus IDE

Adding the UseCThreads define in the custom options of a program or a package as shown above (menu: Project / Project Options / Compiler Options / Custom Options) is no longer necessary but it will not cause any problem. However, do not count on the alternate C memory manager unit, cmem, being loaded in *nix systems. That can be a source of problems; see the next section.

Use the cmem Unit in Linux toc

I have found that there can be memory leaks if the following conditions obtain.

  1. The application target is x86_64-linux-gtk2.
  2. The FreeOnTerminate property of a TThread instance is set to true.
  3. The default memory manager is used.

There are solutions.

Of course, if all TThread instances are created at the start of a program and are not terminated until the program finishes, then one could just ignore the leaks.

Setting a Thread Instance to nil Once it is Freed toc

Setting a thread instance to nil provides a good way to avoid hanging a program by when invoking the Terminate and WaitFor on a terminated thread. That is why I suggested using freeandnil in the previous section when terminating by force the thread with the Terminate method.

But what happens if a thread terminates on its own because the TThread.Execute method has run out of things to do? Use the OnTerminate event. It is safe to set the intance to nil if FreeOnTerminate is true.

constructor TMyThread.Create(CreateSuspended: boolean); begin inherited Create(CreateSuspended); FreeOnTerminate := false; // or true OnTerminate := @Form1.Terminate; end; procedure TForm1.Terminate(Sender: TObject); begin MyThread := nil; end;

The story is a bit more complex FreeOnTerminate is false. It appears that cmem keeps better track of allocated memory and there will be no leak even if the instance is set to nil in the OnTerminate event handler. There will be a leak if the default memory manager is used in Linux. This warrants further investigation.

Do not Make a Stupid Mistake toc

So what was my big mistake? I started the HTTP request thread in suspended mode in order to have time to set up some local variables in the thread. Then to execute the thread I called the overridden protected Execute method instead of the Start method. So in essence it was as before, no system thread was created, the code in Execute was run much like any other procedure in the main thread also use for running the GUI. Don't do that. Once the thread was started with the correct Start method, calls to my delay function were removed, and the GUI was updated with the PostMessages things began work rather well.

Coming Up toc

There's more to say about TThreads. I will be also be putting up a repository on my GitHub account containing the "speelunking" code I used to explore the TThread class.

<-Free Pascal/Lazarus Gotcha