Calling Application.ProcessMessages in a Delphi program

 Delphi  Comments Off on Calling Application.ProcessMessages in a Delphi program
Sep 292018
 

Using Application.ProcessMessages is being frowned upon by many because often it means that your program design is flawed. These people usually suggest you should use multi-threading instead, which then opens another can of worms.

First of all, let me make clear, that I am talking about Windows programs written in Delphi using the Visual Component Library (VCL). This is not about Android or IOS development and also not about the Firemonkey (FMX) framework and most certainly not about any dotNET language.

With that out of the way, let’s discuss what Application.ProcessMessages does and why it is being used:

Application is the global object declared by the VCL in the Forms unit which provides methods that are independent of forms. One of them is ProcessMessages. What it does is looking into the Windows message queue and processing any messages it may find there. Since Windows GUIs are message driven, this is important for a program to react to user input as well as other events, e.g. redrawing controls that have changed.

Normally this processing of messages is done automatically in the program’s main message loop that is executed all the time when it is idle. But if your program does some long running work after e.g. the user pressed a button, it might take a while until it gets back to executing the message loop and to the user it will seem to have crashed or hung.

To alleviate this, for many years Delphi programmers have added the occasional call to Application.ProcessMessages in their code. This usually looks like this:

proccedure TMyForm.b_Execute(Sender: TObject);
var
  i: integer;
begin
  for i := 0 to GetCount - 1 do begin
    DoSomeWork(i);
    Application.ProcessMessages;
  end;
end;

Where DoSomeWork does part of the total work and returns. And then the program calls Application.ProcessMessages to update the user interface.

Why is this bad?

  • Processing Windows messages may have some undesired side effects. E.g.:
    • Your form might have multiple buttons which do different things. ButtonA might start a long running calculation A while ButtonB might start a different calculation B. And it’s most likely that you don’t want to start calculation B while calculation A is still running.
    • Even if your form only has one button that starts an action you still don’t want that action to be started again while the first run hasn’t finished.[*]
    • Your calculation might access the user interface to retrieve some configuration. If you do that frequently and in between call Application.ProcessMessages, these settings may have changed, so you might start with one set of configurations and continue with a different one. The result will most likely not be what you expect.
    • The user might close the form while the calculation is still running. That could result in resources that the calculation needs being freed, which usually causes errors like Access Violations.
  • Each call to Application.ProcessMessages takes time which might slow down your calculations significantly.

So, should we not call Application.ProcessMessages? And how do we keep the GUI active?

As mentioned above, the answer you usually get is to use multi-threading. That is: Move the work into a secondary (or even many secondary) worker thread and let the main thread take care of the user interface. If done properly this works very well. But can you do it properly? Well, I for one have written my share of multi-threaded programs and I can tell you: It’s difficult to do it properly. And most of the time it isn’t necessary.

One of the main stumble blocks is debugging. The Delphi debugger is not particularly suited for debugging multi-threaded programs. It shows you a list of threads and the code which is being executed by the currently active thread (on a multi core system, there is more than one currently active thread, I have no idea how the debugger selects the one to display). You can switch to a different thread and look at the code it is executing. Even worse: The debugger will switch between the threads automatically for no apparent reason, so you press F8 to step over one command and find yourself in the code of a different thread all of a sudden. Other than the debugger, there are of course other debugging methods, e.g. writing to a log or showing message boxes. The latter can only be done safely in the main thread because the VCL is not thread safe. Writing a log will pose the challenge of accessing that log in a thread safe manner.

All that does not mean, you should not use multi-threading if you need it. It just means that you should consider the implications this will have on the complexity of your code and debugging it.

This blog post is not about multi-threading in particular but about Application.ProcessMessages, so I will not go down that route any further.

So, what can we do with a single threaded program to alleviate the above mentioned pitfalls of Application.ProcessMessages?

  • First of all: Once you start processing some data, stop the user from interfering.
    • Disable the controls in the user interface, including the button that just started the processing. This also gives the user feedback that the program is busy right now. He cannot start anything else and also not change the settings being used by the calculations. Don’t forget to enabled the controls afterwards (e.g. use Try…Finally to ensure them to be re-enabled.)
    • Prevent the current form from being closed. This can be done with an OnCloseQuery event.
  • Call Application.ProcessMessages as often as necessary, but not too often.

In addition you should provide a visual feedback of the progress and have an Abort button that allows the user to – well – abort the calculations.

Basically you do something like this:

procedure TMyForm.SetIsRunning(_Value: boolean);
begin
  // set a flag that can be checked elsewhere
  FIsRunning := _Value;
  // en-/disable the button that started this
  b_Execute.Enabled := not _Value;
  // en-/disable any configuration controls
  fe_InputFile.Enabled := not _Value;
  fe_OutputFile.Enabled := not _Value;
  chk_SomeSetting.Enabled := not _Value;
  // en-/disable the Abort button, note the missing "not" here!
  b_Abort.Enabled := _Value;
  // ensure that the Abort button has the correct caption
  b_Abort.Caption := _('Abort ...');
  // reset a flag that tells you to abort
  FAborting := False;
end;

procedure TMyForm.FormCloseQuery(Sender: TObject; var CanClose: Boolean);
begin
  Result := not FIsRunning;
  // you might want to add a message box here to ask the user if he
  // wants to abort
end;

procedure TMyForm.b_AbortClick(Sender: TObject);
begin
  // you might want to add a message box here to ask the user if he
  // really wants to abort

  // set the flag that the user wants to abort processing
  FAborting := True;
  // give him feedback that we are doing what he requested
  b_Abort.Caption := _('Aborting ...');
end;

procedure TMyForm.b_ExecuteClick(Sender: TObject);
var
  LastAPMCall: Int64;
  ThisAPMCall: Int64;
  i: integer;
begin
  SetIsRunning(True);
  try
    LastAPMCall := GetTickCount;
    for i := 0 to GetCount - 1 do begin
      DoSomeWork(i);
      // now, instead of calling Application.ProcessMessages every time
      // we check that the last call was more than 100 ms ago
      ThisAPMCall := GetTickCount;
      if (ThisAPMCall - LastAPMCall > 100) or (ThisAPMCall < LastAPMCall) then begin
        // GetTickCount is a 32 bit value that wraps around about
        // every two weeks. That's what the second check is for.
        Application.ProcessMessages;
        LastAPMCall := ThisAPMCall;
        if FAborting then begin
          // The user has pressed the Abort button, so we do what he wants
          SysUtils.Abort;
        end;
      end;
    end;
  finally
    SetIsRunning(False);
  end;
end;

That’s the basic principle which many of our internal tools use. It avoids the complexity of multi-threading while still keeping the UI responsive and allow the user to abort processing. I’m not claiming that it is perfect but it works, is simple to understand and also simple to debug.

Drawbacks are the following:

  • If the processing is more complex and should be moved to a procedure in a different unit, it’s easy to forget the Application.ProcessMessages calls. Also if you have these calls in your code, you should not call it in a secondary thread. Bad things will happen, if you do.
  • If you call Application.ProcessMessaging in multiple places, it might not be as easy as in the example to keep track of how long ago the last call was. One possible solution is to have a global TTimedProcessMessages object that does the time keeping for you.
  • Checking the Abort button is also a problem once your code is no longer in the form’s object. Again, a TTimedProcessMessages object can solve that.
  • You cannot use that code in Firemonkey programs and, most unfortunately, you cannot use it for cross platform programs (which would require Firemonkey anyway.)
  • Nowadys most computers have multiple cores. This is a single threaded program, so it will only use the processing power of one core. Depending on the amount of processing that needs to be done and whether it can be split into independent work packages, a multi-threading solution would be much more efficient.

Oh, you might ask, where this TTimedProcessMessages object I am talking about can be found. Here you go:

type
  PBoolean = ^Boolean;
  TTimedProcessMessages = class
  private
    FDummyBool: Boolean;
    FAbortedFlag: PBoolean;
    FLastCall: Int64;
    FMinTicks: Int64;
  public
    constructor Create(_MinTicks: integer; _AbortedFlag: PBoolean = nil);
    function Execute(var _WasAborted: boolean): boolean; overload;
    function Execute: boolean; overload;
  end;

constructor TTimedProcessMessages.Create(_MinTicks: integer; _AbortedFlag: PBoolean = nil);
begin
  inherited Create;
  FMinTicks := _MinTicks;
  if Assigned(_AbortedFlag) then
    FAbortedFlag := _AbortedFlag
  else
    FAbortedFlag := @FDummyBool;
end;

function TTimedProcessMessages.Execute(var _WasAborted: boolean): boolean;
var
  ThisCall: Int64;
begin
  ThisCall := GetTickCount;
  Result := (ThisCall - FLastCall > FMinTicks) or (ThisCall < FLastCall);
  if Result then begin
    Application.ProcessMessages;
    _WasAborted := FAbortedFlag;
  end else
    _WasAborted := False;
end;

function TTimedProcessMessages.Execute: boolean;
var
  DummyBool: Boolean;
begin
  Result := Execute(DummyBool);
end;

You either create a global variable (no, of course that’s bad) a singleton of this class

  gblTimedProcessMessages := TTimedProcessMessages.Create(100, @FAborting);

or create an instance and pass it around to all methods that need it. These methods then use it like this:

  if gblTimedProcessMessages.Execute(WasAborted) and WasAborted then
    SysUtils.Abort;

(Note: All the code above has just been typed in as I went along, so it might contain bugs. I haven’t even tried to compile it.)

There you go. Good thing I have disabled comments on this blog anyway, so there will be no way for people to call me unprofessional because I recommend using Application.ProcessMessages. 😉

But if you really want to call me names, here is the corresponding Google+ post.

Footnotes

[*]
As Lars Fosdal pointed out in his comment on my Google+ post:
You also need to worry about reentrancy. I.e. processing a new message before the processing of the previous one have completed, something that is particularly troublesome during UI updates.

 Posted by on 2018-09-29 at 15:57

Delphi LongWord is not always a 32 bit unsigned integer

 Delphi  Comments Off on Delphi LongWord is not always a 32 bit unsigned integer
Sep 162018
 

Did you know that LongWord in Delphi is not (any longer) always a 32 bit unsigned integer?

I wasn’t aware of this until I asked about what others use for that data type in the Delphi Developers Google+ community.

I turned out that the majority of the participants use Cardinal, followed by Uint32.

Only less than 10% use LongWord.

Stefan Glienke was the first to point out that LongWord is not always 32 bits, even though the 64-bit Windows Data Types Compared to 32-bit Windows Data Types entry in the DokWiki seems to suggest it. But it only covers Data types for Windows.

There is a better overview table on the Simple Types reference page. Even though I find it rather odd that they say that Integer and Int64 are aliases for NativeInt (I wouldn’t have had that “alias” column in the Platform Dependent Integer Types table at all.)

So, where does that leave me? I guess it’s best to switch to Int / UInt + bitness for those types where it is important how many bits an integer has (such as in records that define data written to a file). For the other cases it might be better to use NativeInt or NativeUInt, which translates to Int32 / UInt32 on 32 bit platforms and Int64 and UInt64 on 64 bit platforms. Unless of course they don’t.

 Posted by on 2018-09-16 at 15:47

NativeInt / NativeUInt type in various Delphi versions

 Delphi  Comments Off on NativeInt / NativeUInt type in various Delphi versions
Sep 082018
 

Just in case you are maintaining Delphi code for several older versions of Delphi: Be aware that the declarations of NativeInt and NativeUInt are wrong in some of them.

Delphi version SizeOf(Native(U)Int) Win32 SizeOf(Native(U)Int) Win64
1 to 6 not available not available
7 to 2007
8 ← this is wrong
not available
2009 to XE 4 not available
XE2 to XE8 4 8
10.x 4 8

I just stumbled over this in GExperts, which had a wrong IFDEF for overwriting that declaration.

 Posted by on 2018-09-08 at 14:25

If your floating point calculations differ after a trivial change

 Delphi  Comments Off on If your floating point calculations differ after a trivial change
Sep 062018
 

Today I had a curious bug: After changing the way GPS coordinates were read from a file (not calculated!) all of a sudden lots of unrelated floating point calculations had different results.

I reverted the changes just to be sure and, yes, the results were back to the original values.

I added the new code again, same problem.

I changed the code to first use the new method for reading the file, followed by the old one, overwriting the data. The problem still occurred.

I reduced the new code to just constructing the object that does the reading. The problem still occurred.

I even removed the object construction. The problem still occurred.

Finally I had a closer look at the code in the unit that contains the object declaration. Nothing obvious, but after about an hour staring at not very complex code it turned out that it used another unit which automatically loads the proj4.dll which we use for converting geographic coordinates. It doesn’t actually call that dll, just loads it. And then it dawned to me: There is this thing called 8087 Control Word which controls how various operations are done, e.g. rounding. So if the dll changes that control word in its initialization, it will change the calculations done in other parts of the program.

To prove this, I added code that reads the 8087 Control Word before and after the dll was loaded. The values were different. So that probably was the culprit.

The SysUtils unit exports a SafeLoadLibrary function. In addition to calling the LoadLibrary Windows API function, it saves the error mode and the 8087 Control Word before loading the dll and restores these values afterwards. I could not use that function because the unit loads the dll from a resource rather than from a file, using my TdzResourceDllLoader (from dzlib). So I added code to safe and restore the control word to that class.

Guess what? The problem went away.

This is the code (copied from SysUtils.SafeLoadLibrary) I used:

var
  FPUControlWord: Word;
begin
  // [...]
  // save the FPU Control Word
  asm
    FNSTCW  FPUControlWord
  end;
  try
    // code to
    // load the dll into memory
    // relocate it
    // call dllmain
  finally
    // restore the FPU Control Word
    asm
      FNCLEX
      FLDCW FPUControlWord
    end;
  end;
 Posted by on 2018-09-06 at 18:35

Enable debug logging in gnugettext.pas

 Delphi  Comments Off on Enable debug logging in gnugettext.pas
Sep 032018
 

gnugettext.pas has got a conditional define called DXGETTEXTDEBUG. If it is defined, various debug messages are written to a MemoryStream. That won’t help much, if you can’t read that stream, so you need a way to write that stream to a file.

Guess what, that’s easily possible. Just call DefaultInstance.DebugLogToFile passing it a file name to which you want the log to be written. An additional parameter determines whether the file should be overwritten or been appended to.

In that call, the whole memory stream gets written to that file, so no log messages get lost. Also, any subsequent log messages will be written directly to that file instead of the memory stream.

 Posted by on 2018-09-03 at 17:34

GExperts Enhancements for docking in the Delphi IDE

 Delphi, GExperts  Comments Off on GExperts Enhancements for docking in the Delphi IDE
Sep 022018
 

The Delphi IDE has supported docking of various forms for a long time (I don’t remember if it ever did not). Unfortunately if not docked, the floating forms always seem to be in the way, and if you dock them, they take up screen space that you might rather use for the editor window. Given that most of us have got more than one monitor, wouldn’t it be nice to move some of these docking forms to the second monitor?

You can already do that by just undocking one form and docking other forms to it. The resulting floating window can be moved freely, even to another monitor. What you can’t do with it is minimize it or use WIN+Arrow Keys to snap them to the left or right monitor edge.

There are now two enhancements in GExperts that address this:

The first one is the new IDE enhancement “Enhance Dock Forms to allow minimize and Win+arrow positioning” which can be enabled on the IDE tab of the configuration dialog.

Once set, those floating forms get a minimize button (which of course works) and can be moved to the left or right half of the monitor with Win-Left or Win-Right arrow keys. The usual mouse gestures you can do with any window are also available.

Unfortunately this only works if more than one form have been docked together, so a single floating form will not show these enhancements.

The second enhancement is the new “Add dock window” expert. It creates a new top level window to which any docking form can be docked. It too allows all the usual Windows shortcuts for placing and moving forms. In addition it has got a button on the taskbar so you can minimize it and restore it easily. It can also be put behind the IDE main window or on a different monitor. The taskbar button makes it easy to bring it back.

You can even have multiple of these GExperts dock windows.

Unfortunately they have some drawbacks too:

  • You cannot close them as long as anything is docked to them. I tried to allow that but it crashed the IDE. To really get rid of the window, undock all docked forms from it and then close it.
  • Also, if you dock only a single form to it, you won’t get a caption for it to move it elsewhere. Tip: Disable docking using its context menu (which will make it float) and enable it again. Alternatively select one of your saved desktop layouts or restart the IDE.
  • There is no option to save the currently docked forms and their layout yet.

I’ll try to address these problems in a later version.

Both enhancements work for all supported Delphi IDEs, btw.

There is no release with these features yet. To get them, for now you have to compile your own GExperts dll which isn’t exactly rocket science anyway.

This idea is based on a question by Codehunter on the German Delphi board Delphi Praxis.

 Posted by on 2018-09-02 at 12:21