GExperts 1.3.16 experimental twm 2020-05-27 released

 Delphi, GExperts  Comments Off on GExperts 1.3.16 experimental twm 2020-05-27 released
May 272020
 

There they are, the promised GExperts 1.3.16 installers for older Delphi versions.

This time I didn’t blog much about the work I did on GExperts. I spent a lot of time actually working on it instead. There are lots of bug fixes and a few new features in the new version.

I hope this time the installers won’t be wrongly detected as malware by virus scanners. Sorry about that.

The new version is available for download on the GExperts download page.

 Posted by on 2020-05-27 at 21:14

GExperts 1.3.16 for Delphi 10.4

 Delphi, GExperts  Comments Off on GExperts 1.3.16 for Delphi 10.4
May 272020
 

Today I downloaded and installed the latest and greatest Delphi 10.4. The first thing to do, was of course compile GExperts for it.

Thanks to Achim Kalwa I already had a working project and he also made all other changes necessary to make it compile. So basically I loaded the project and pressed Compile.

A few tests have shown no problems, but I am sure there will be some. On the other hand I don’t want to be the one to find them all. That’s what users are for, aren’t they? ūüėČ

OK, so if you are brave, you can download an installer for GExperts 1.3.16 for Delphi 10.4. Please file any bug reports and feature requests as tickets on SourceForge.

For those still using older Delphi versions, there will also be a GExperts 1.3.16 release shortly. Watch this space.

 Posted by on 2020-05-27 at 20:35

When computers try to be clever

 Delphi, dzLib  Comments Off on When computers try to be clever
May 072020
 

Don’t you just hate it when computers try to be clever but get it wrong? OK, it’s not really the computer itself but the programmer who tried to be clever. The problem usually is that he overlooked a corner case that you then hit where his sophisticated strategy fails and leaves you with something – lets say less desirable.

Today I wasted several hours trying to find out what was wrong with my hack to create a TSpeedButton which can take the input focus (which standard TSpeedButtons can’t), by using a TBitBtn instead. I blogged about this before.

There was a certain button width which caused the button to display only a square part of the caption. At closer inspection it turned out to not be a fixed width but kind of a width to height ratio that caused the problem. Here are several buttons with different widths and heights that show this behaviour:

Buttons with a width to height ratio of (w-1) : (h-1) = 4 : 1 looked like the screenshot above, e.g. 97×25, 101×26, 105×27, 93×24 or 89×23. Other sizes were fine.

The code I use in this hack is very simple: I generate two bitmaps, one for the Up state of the button, the other for the Down state and assign them to the TBitBtn’s Glyph property.

procedure TdzSpeedBitBtn.UpdateGlyph;
begin
  if FBtn.Tag <> 0 then
    FBtn.Glyph := FDownBmp
  else
    FBtn.Glyph := FUpBmp;
end;

That bitmap is always one pixel smaller than the button itself, so the problem occurred with bitmaps of 96×24, 100×25, 104×26, 92×23 and 88×22 pixels respectively.

So I tried to reproduce that behaviour with a simple TBitBtn with default settings by assigning a bitmap with 96×24 pixels. It looked like this …

… even at design time. Closer inspection showed that the NumGlyps property also had a different value than expected: 4. Setting it to 1 made the button display the whole bitmap again:

So there seemed to be some code behind assigning a bitmap to the glyph that tried to guess how many images there actually are stored in the bitmap. If the width to height ratio is 4, it assumes that there are 4 images in the bitmap.

With that knowledge I looked into the documentation of TBitBtn.Glyph and guess what: It’s documented behaviour, kind of:

You can provide up to four images within a single bitmap. All images must be the same size and next to each other in a row. Bit buttons display one of these images depending on their state.
[…]
If you have multiple images in a bitmap, you must specify the number of images that are in the bitmap with the NumGlyphs property

I actually knew that but since I rarely use TBitBtn controls I hadn’t thought of it. It doesn’t say that the code tries to be clever and guess how many images there are though.

Tests showed that the code calculates the width to height ratio and for ratios of 2, 3 or 4 assumes that to be the number of images and sets NumGlyphs accordingly. And of course it’s easy to find once you know what you’re looking for:

procedure TButtonGlyph.SetGlyph(Value: TBitmap);
var
  Glyphs: Integer;
begin
  Invalidate;
  FOriginal.Assign(Value);
  if (Value <> nil) and (Value.Height > 0) then
  begin
    FTransparentColor := Value.TransparentColor;
    if Value.Width mod Value.Height = 0 then
    begin
      Glyphs := Value.Width div Value.Height;
      if Glyphs > 4 then Glyphs := 1;
      SetNumGlyphs(Glyphs);
    end;
  end;
end;

(from unit Vcl.Buttons)

Why did it take me several hours to find some simple problem like this, you ask? (I asked myself that question too.) Of course this didn’t happen in a simple test program like the one in the screen shot. It happened in a dialog in GExperts. GExperts is a plugin for the Delphi IDE and as such must be built with runtime packages. And if you do that, you can’t compile with debug dcus and simply step into the RTL/VCL code in the debugger, all you can see is some assembler code. So I first tried to find the problem in my code. Did I maybe overlook a corner case? Then I tried to reproduce the problem with a test program but couldn’t, because the width to height ratio of my test buttons wasn’t the one which caused the problem. I puzzled quite a while over this, looked into the DFM files to find anything suspicious, but there was nothing. Only after I copied the buttons from the GExperts form to the form of the test program I could reproduce it.

Once I knew the cause, the fix was easy: After assigning the Glyph change NumGlyphs back to 1.

procedure TdzSpeedBitBtn.UpdateGlyph;
begin
  if FBtn.Tag <> 0 then
    FBtn.Glyph := FDownBmp
  else
    FBtn.Glyph := FUpBmp;
  FBtn.NumGlyphs := 1;
end;

This is the way it should have looked all along:

The fix is now in my dzlib repository on OSDN.

To be fair: The programmer meant this as a convenience function and it probably works most of the time. But if it doesn’t it becomes a pain in the lower back to debug.

If you want to discuss this article, you can do so in the corresponding post in the international Delphi Praxis forum.

 Posted by on 2020-05-07 at 17:28

More features for the IFDEF expert in GExperts

 Delphi, GExperts  Comments Off on More features for the IFDEF expert in GExperts
Apr 262020
 

The IFDEF editor expert was added to GExperts in 2016 and improved again in the same year, to support symbols defined in include files.

Unfortunately sometimes an include file itself sometimes includes other files (e.g. jclNN.inc in the JEDI Code Library includes jedi.inc) which will usually add additional symbols which are then available for conditional compilation. But the IFDEF expert only listed symbols from the original include file.

This has irked me for a long time, so today I changed this: Every include file that is included via an include file (and recursively via these include files) is now listed in the IFDEF expert:

In this example dzlib.inc itself includes dzlibjedi.inc which is simply a copy of jedi.inc. (I don’t want to include jedi.inc directly in order to not have a stale file laying around which might break compilation of other libraries.) The expert lists this file as if it had been included in the unit source code in addition to dzlib.inc.

And, since I seem to have forgotten to blog about it: The expert also searches for available include files in the search path and allows you to add them:

Just select the file from the menu and select the symbol you want to use. The required include directive will automatically be added to the interface section of the current unit.

If you want to discuss this blog post, go to the corresponding post in the international Delphi Praxis forum.

 Posted by on 2020-04-26 at 13:20

Skipping the UTF-8 BOM with TMemIniFile in Delphi 2007

 Delphi  Comments Off on Skipping the UTF-8 BOM with TMemIniFile in Delphi 2007
Mar 072020
 

Recently I came across a problem with INI files: Some editors (including recent versions of Windows Notepad) add a byte order mark (BOM) to the files they save. In particular the BOM for UTF-8 kept appearing in INI files which then were read incorrectly by the Delphi 2007 implementation of TMemIniFile (I guess the same applies to all pre Unicode versions of Delphi). In particular this was a problem with programs that used TJvAppIniStorage for streaming application settings to disk. (TJvAppIniStorage internally uses TMemIniFile.) So I tried to fix this, first by adding code that reads that file, removes the BOM and writes it back, before actually using it. This had some unpleasant side effects because some programs that usually start at the same time tried to access the file in parallel and failed. (No problem when only reading, but a big problem when writing.)

So I dug deeper and found that modifying TMemIniFile.LoadValues like this fixed the problem:

procedure TMemIniFile.LoadValues;
const
  BOM_LENGTH = 3;
var
  List: TStringList;
  st: TMemoryStream;
  Buffer: array[0..BOM_LENGTH-1] of Byte;
begin
  if (FileName <> '') and FileExists(FileName) then
  begin
    List := nil;
{$MESSAGE hint 'UTF-8 fix for TMemIniFile.LoadValues is active'}
    st := TMemoryStream.Create;
    try
      st.LoadFromFile(FileName);
      st.Position := 0;
      if BOM_LENGTH = st.Read(Buffer, BOM_LENGTH) then begin
        // the file contains at least BOM_LENGTH bytes
        if (Buffer[0] = $EF) and (Buffer[1] = $BB) and (Buffer[2] = $BF) then begin
          // we have a BOM -> Just leave the stream position as it is
        end else begin
          // no BOM -> reset stream position
          st.Position := 0;
        end;
      end;

      List := TStringList.Create;
      List.LoadFromStream(st);
      SetStrings(List);
    finally
      List.Free;
      st.Free;
    end;
  end
  else
    Clear;
end;

Note that this will only skip the BOM for UTF-8, but that is the only case I have ever encountered, because UTF-8 is an encoding that is mostly compatible with ANSI encoding. Other encodings will break TMemIniFile completely. But even with UTF-8 you will still encounter problems with characters that are encoded with more than one byte. So this is more of a simple workaround than a bugfix. For a bugfix, you will have to properly decode the whole file. (Or use a Unicode aware version of Delphi where this problem doesn’t exist.)

Of course TMemIniFile is declared in the RTL unit IniFiles so modifying it is not something to do on a whim. It turned out that at least in my case there was no problem as apparently there are no other RTL units that needed to be recompiled to include the changed IniFiles unit. So the easiest way was to copy IniFiles.pas to my program’s source directory, add it to the project (I prefer doing that so it’s easier to spot such a modified unit.) and recompile.

 Posted by on 2020-03-07 at 10:45

Delphi’s TStream.Read returns the number of bytes read

 Delphi  Comments Off on Delphi’s TStream.Read returns the number of bytes read
Mar 022020
 

Note to self: TStream.Read in the Delphi RTL returns the number of bytes read. It does not check whether the intended number of bytes could actually be read. So if you do not check it yourself, call TStream.ReadBuffer instead.

So, it’s either:

var
  st: TFileStream;
  Buffer: array[0..7] of Byte;
  BytesRead: Integer;
begin
  st := TFileStream.Create(fn, fmOpenRead);
  try
    BytesRead := st.Read(Buffer, SizeOf(Buffer));
    if BytesRead <> SizeOf(Buffer) then
      raise Exception.CreateFmt('BytesRead (%d) <> SizeOf(Buffer) (%d)',
        [BytesRead, SizeOf(Buffer)]);
    // do something with the content of buffer
  finally
    FreeAndNil(st);
  end;
end;

or

var
  st: TFileStream;
  Buffer: array[0..7] of Byte;
begin
  st := TFileStream.Create(fn, fmOpenRead);
  try
    st.ReadBuffer(Buffer, SizeOf(Buffer));
    // do something with the content of buffer
  finally
    FreeAndNil(st);
  end;
end;

The same logic applies to TStream.Write and TStream.WriteBuffer.

I have just grep-ed my sources and found way to many places where I used Read instead of ReadBuffer and Write instead of WriteBuffer.

Unfortunately that’s an easy mistake to make, so I guessed that I am not the only one who made it. And lo and behold, I found lots of places in the Delphi 2007 RTL and VCL (so they might have been fixed in the mean time, I didn’t check though) and several 3rd party libraries (including the current JCL and JVCL) where this mistake was made. So it’s probably a good idea if you do that check on your own code.

The regular expressions I used for this were:

^\s*\w*\.read\(
^\s*\w*\.write\(

If you use GExperts Grep, don’t forget to enable the “Regular Expression” option!

 Posted by on 2020-03-02 at 16:03

dzBdsLauncher 1.0.3 released

 Delphi, dzBdsLauncher  Comments Off on dzBdsLauncher 1.0.3 released
Mar 012020
 

The latest version of dzBdsLauncher can now also handle some .dproj files with invalid ProjectVersion entries (e.g. those generated by project JEDI which apparently uses ProjectVersion 17.3 for all Delphi versions >XE8). It does this by evaluating the DllSuffix entry, if one exists. This can also help solving conflicts if the ProjectVersion is not unique.

In addition the tool can now also open .grouproj files. It handles them by inspecting the first project listed in the file.

Another small improvement is colored diagnostic output to help troubleshooting if something goes wrong.

See the main dzBdsLauncher page for download links.

 Posted by on 2020-03-01 at 19:24

dzPackageInst 1.0.2 for Delphi 6 to 10.3 released

 Delphi  Comments Off on dzPackageInst 1.0.2 for Delphi 6 to 10.3 released
Feb 282020
 

Today I released version 1.0.2 of dzPackageInst. Its a command line tool that allows installing and uninstalling design time packages into the Delphi 6 to 1.3 IDEs.

Source code and download are available from the project page on SourceForge.

It’s also part of my buildtools which I use for nearly all my projects.

 Posted by on 2020-02-28 at 09:37

TStringList vs. THashedStringList vs. TDictionary

 Delphi  Comments Off on TStringList vs. THashedStringList vs. TDictionary
Feb 162020
 

Prompted by the topic Dictionaries, Hashing and Performance in the international Delphi Praxis forum I did some timing to compare the performance of data structures in the Delphi runtime library that can be used to store data indexed by strings:

  • a sorted, case sensitive TStringList (available since Delphi 6)
  • a sorted, case sensitive THashedStringList (available since Delphi 6)
  • a TDictionary<string,Integer> (available since a Delphi 2009)

Just in case you did not know about THashedStringList: It is a TStringList descendant declared in System.IniFiles. It’s used to speed up access to TMemIniFile. (EDIT: As Uwe Raabe pointet out, that’s no longer true. As of Delphi 10.3 (and possibly earlier, I haven’t checked) TMemIniFile also uses TDictionary<string,Integer>.)

The test adds 676 strings (‘AA’ .. ‘ZZ’) to each structure and does that 10000 times (which means that there are quite a few checks for duplicates to be done on adding). Then – again 10000 times – it does a lookup for each of these strings.

Of course that is just a simple test and it is neither a large number of entries nor long strings. I just wanted to get a feel for the performance of these structures.

Here is the main code for TStringList and THashedStringList:

procedure Tf_HashedStringListTest.DoTiming(sl: TStringList);
const
  CYCLES = 10000;
var
  k: integer;
  i: integer;
  j: integer;
  sw: TStopwatch;
  s: string;
  Idx: Integer;
begin
  sl.Sorted := True;
  sl.CaseSensitive := True;
  sl.Duplicates := dupError;
  sw := TStopwatch.StartNew;
  sl.BeginUpdate;
  for k := 1 to CYCLES do begin
    for i := Ord('A') to Ord('Z') do begin
      for j := Ord('A') to Ord('Z') do begin
        s := chr(i) + chr(j);
        if not sl.Find(s, Idx) then
          sl.AddObject(s, Pointer(i * 100 + j));
      end;
    end;
  end;
  sl.EndUpdate;
  sw.Stop;
  m_Output.Lines.Add(sl.Count.ToString + ': Add: ' + sw.Elapsed.ToString);

  sw.Reset;
  sw.Start;
  for k := 1 to CYCLES do begin
    for i := Ord('A') to Ord('Z') do begin
      for j := Ord('A') to Ord('Z') do begin
        s := chr(i) + chr(j);
        sl.IndexOf(s);
      end;
    end;
  end;
  m_Output.Lines.Add(sl.Count.ToString + ': IndexOf: ' + sw.Elapsed.ToString);
end;

And very similar for TDictionary:

procedure Tf_HashedStringListTest.DoTiming(sl: TDictionary<string, integer>);
const
  CYCLES = 10000;
var
  k: integer;
  i: integer;
  j: integer;
  sw: TStopwatch;
  s: string;
  v: integer;
begin
  sw := TStopwatch.StartNew;
  for k := 1 to CYCLES do begin
    for i := Ord('A') to Ord('Z') do begin
      for j := Ord('A') to Ord('Z') do begin
        s := chr(i) + chr(j);
        if not sl.TryGetValue(s, v) then
          sl.Add(s, i * 100 + j);
      end;
    end;
  end;
  sw.Stop;
  m_Output.Lines.Add(sl.Count.ToString + ': Add: ' + sw.Elapsed.ToString);

  sw.Reset;
  sw.Start;
  for k := 1 to CYCLES do begin
    for i := Ord('A') to Ord('Z') do begin
      for j := Ord('A') to Ord('Z') do begin
        s := chr(i) + chr(j);
        sl.Items[s];
      end;
    end;
  end;
  m_Output.Lines.Add(sl.Count.ToString + ': IndexOf: ' + sw.Elapsed.ToString);
end;

The result is not really surprising:

TDictionary is the winner by a large margin, followed by THashedStringList and then TStringList. The two string lists only differ in the IndexOf times, the adding times are very similar.

On my computer, with an AMD Phenom II XE 1090T processor, and compiled with Delphi 10.3 I get the following times:

Structure Time for Add [sec] Time for IndexOf [sec]
TStringList 7.43 7.48
THashedStringList 7.45 4.40
TDictionary 1.05 1.04

EDIT: I just found that changing the code for the THashedStringlist from using Find to using IndexOf reduced the time for adding entries to about the same time as for IndexOf. So both are about 4 seconds. This makes me wonder whether there is a bug in THashedStringList because it does not override AddObject. It simply inherits it from TStringList which for sorted lists calls Find to see if the string is already in the list. In contrast to IndexOf the Find method does not use the hash so it’s as slow as in TStringList. But maybe that is on purpose because the hashes get calculated rather frequently for all entries. I get the impression that THashedStringList is not really well implemented and nobody noticed because it was just good enough.

EDIT2: As it was only used in TMemIniFile to get fast access to the entries without needing them to be sorted, the implementation probably was good enough. My test above doesn’t check the performance of Add for an unsorted THashedStringList which is what TMemIniFile used.

If you like, you can download the full source code of my test program.

 Posted by on 2020-02-16 at 16:05