Nov 252017

You might have noticed that I haven’t blogged as much as usual and that there have been no recent updates to GExperts either. The reason of course was that I was busy doing other things. Some of them were programming related and one of these was writing a caching class wrapper for TStream descendants (e.g. TFileStream, but it should work with any other streams as well).

It is simple to use and totally transparent to the calling code because the cache itself descends from TStream:

procedure TestCache;
  fs: TFileStream;
  cache: TdzStreamCache;
  i: integer;
  by: byte;
  cache := nil;
  fs := TFileStream.Create('C:\test.dat', fmOpenWrite);
    cache := TdzStreamCache.Create(fs);
    // now, do any operation you usually do on the stream, but use the
    // cache instead
    // e.g. write 50 megabytes of random bytes
    for i := 0 to 50*1024*1024 do begin
      by := Random(256);
      cache.WriteBuffer(by, SizeOf(by));
    // Writing a single byte to a TFileStream will result in very
    // bad performance, which is why we usually copy the bytes to a
    // larger buffer instead and then write this buffer to the stream.
    // Using the TdzStreamCache you won't see much of a difference between
    // these two approaches.

TdzStreamCache can also be used for read and read/write access to a stream. It supports linear and also random access. Performance is worst when reading a stream backwards a single byte at a time.

To get an idea how well it performs, I have benchmarked it against David Heffernan‘s excellent CachedFileStream and found that it compares very well. (I could not simply use David’s class because I needed caching for other types of streams, not just files.)

I have been using TdzStreamCache extensively in the last several weeks and I am pretty sure that I have weeded out most of the bugs. But of course, that does not mean that there aren’t any left, so there are no guarantees. If you use it, all problems are your own.

TdzStreamCache is released under the Mozilla Public License (MPL) 1.1 as part of my dzLib library, but the unit is pretty much stand alone. The only thing it requires is because Delphi 2007 declares NativeInt inconsistently to other Delphi versions. As with all my dzlib code, it is available from sourceforge as u_dzStreamCache.pas.

The code for the benchmarks I mentioned above is also available if you want to do the benchmarks yourself.

I would appreciate any feed back, especially if you find bugs or improve on it.