GExperts: access violation in the 64-bit IDE caused by a ToolsAPI declaration

AI;DR – Reproducing this bug and writing this blog post were done mostly by Claude Code. If you don’t want to read “AI slop”, stop reading now.

If you have used a GExperts DLL compiled from the sources in the 64-bit IDE of RAD Studio 13, it may have raised an access violation in coreide370.bpl when you opened the GExperts configuration dialog. markbd13 reported this as bug #536 and sent an analysis and a fix with it. Thanks a lot!

Which versions are affected

The problem was introduced in revision #5593. That revision added a check for keyboard shortcuts that are already used elsewhere, both to the configuration dialog and to the Keyboard Shortcuts window. It was fixed in revision #5853. So only DLLs compiled from revisions #5593 to #5852 are affected, and only in the 64-bit IDE. The official release 1.3.29 is older than revision #5593, so it does not contain this code.

The cause

To find out which plugin has bound a key, GExperts asks the IDE through the Open Tools API: IOTAKeyboardServices.LookupKeyBinding returns the first binding of a key, and GetNextBindingRec moves on to the next one. The link between the two is kept in the record TKeyBindingRec, which ToolsAPI.pas declares like this:

TKeyBindingRec = record
  KeyCode: TShortCut;
  KeyProc: TKeyBindingProc;
  Context: Pointer;
  Next: Integer;
  Reserved: Integer;
end;

Next holds a pointer, but it is declared as a 32-bit Integer. In the 64-bit IDE the upper half of that pointer is cut off. GetNextBindingRec then follows what is left, sign-extended to 64 bits, and reads from an address that does not exist. If the memory happens to lie below 2 GB, the value survives the cut and everything works. That is why the crash did not show on every machine or in every session. On my own computer, the 64-bit IDE of RAD Studio 13.2 never crashed. I could reproduce it in the 64-bit IDE of Delphi 12.3, though, with a small test package that registers two bindings for the same key.

Any plugin that enumerates key bindings in the 64-bit IDE has the same problem, so this is not something GExperts can fix properly. I have reported it to Embarcadero as RSS-6058, together with the test package.

The workaround

Since revision 5853, GExperts no longer calls these two functions in the 64-bit IDE (see GX_OtaUtils.pas). As a consequence, a shortcut that another plugin binds without an action is no longer reported as a conflict there. Conflicts with IDE actions, menu keys and the keymapping tables are still reported. The 32-bit IDE is not affected.

As usual, if you want to try this before the next official release, you can compile your own DLL.

Discussion about this in the corresponding post in the international Delphi Praxis forum.