etckeeper – version your /etc directory

 Linux  Comments Off on etckeeper – version your /etc directory
Feb 222018
 

Keeping track of changes in the Linux configuration can be a chore but sometimes it is vital to know what was changed. As a software developer I am used to using version control systems for source code, so why not use one for configuration files?

Enter etckeeper a tool that does exactly that: It tracks changes to files in /etc (including access rights) in a git repository (alternatively it can be configured to use Mercurial, Bazar or Darcs, unfortunately svn is not supported).

It hooks into apt to automatically track changes made by updates. Manual updates can be committed explicitly, but there is also a daily cron job that does it automatically.

When Mozilla Thunderbird gives you 0 bytes attachments

 thunderbird  Comments Off on When Mozilla Thunderbird gives you 0 bytes attachments
Feb 212018
 

.. make lemonade. Yes, that would be nice, but unfortunately we are not talking about life and lemons here.

Today a coworker had a problem with Mozilla Thunderbird: She could no longer open attachments sent to her. Saving these attachments resulted in 0 byte files.

This nearly drove me nuts:

The email source looked fine to me. When she forwarded these emails to me, I could open and save these attachments fine, so it wasn’t the emails.

Also, attachments sent to a different account in her Thunderbird worked fine. So it wasn’t Thunderbird or the virus scanner or some kind of access rights problem.

As a test, I installed a new Thunderbird Portable for her account, guess what? It could also open these attachments.

I eventually solved the problem by removing that apparently defective account from Thunderbird and adding it again. Everything started to work normal.

There are days when you just have to accept that you don’t understand why it works now and why it didn’t work before. I definitely hate it when that happens.

Fixing the Delphi 10.2.2 February 2018 Patch

 Delphi, GExperts  Comments Off on Fixing the Delphi 10.2.2 February 2018 Patch
Feb 172018
 

UPDATE: The problem was that I was using Delphi 10.2.2 build 1978. Apparently I have missed an update in December 2017 to build 2004. I have downloaded and installed the new version, and now the patch worked fine.

I hereby apologize for being stupid, because it clearly says in the description:

This patch for RAD Studio 10.2.2 (build 2004 — it won’t work on build 1978) resolves some compatibility issues in the RTL and fixes a problem with Android animations. It is designed to be installed only on top of an existing RAD Studio 10.2.2 Tokyo installation.

(emphasis mine)

I was so sure to be running the latest version that I didn’t even check. Sorry about causing so much fuss.

I’ll leave the rest of the blog post unchanged just in case anybody is googling for some of the key words and to remind me to check first and complain later.

After I reported the problem with the Delphi 10.2.2 February 2018 Patch, Embarcadero first removed the download and has now released a new installer. Unfortunately this new patch might have solved some other problems, but it didn’t solve mine. I did a complete uninstall of Delphi 10.2.2 (I even removed all directories and files that the uninstall left behind) and a new installation from my DVD followed by applying the new patch. I could still not compile GExperts and the simple test program I described in the steps for the bug report did not compile either. (Guess what: Embarcadero apparently never actually tried to reproduce my problem, but the ticket was closed anyway.). I don’t know what exactly they think they fixed, but they didn’t fix the problem I reported which is:

RAD Studio 10.2.2 Tokyo February 2018 Patch breaks “Link with rutime packages”

  1. Create a new VCL project
  2. Open project options
  3. Select Packages -> Runtime Packages
  4. Set “Link with runtime packages” to true
  5. Close dialog with OK
  6. Compile the project
  7. -> Error

Today I tried to find out what the problem was and how to fix it. It turned out to be actually quite simple:

They updated the packages rtl.bpl and fmx.bpl together with their corresponding .dcp files. Since the .dcp files contain some versioning information the packages that use these packages (e.g. vcl.bpl which uses rtl.bpl) were no longer compatible with the new packages and the compiler complained about this.

So, the fix for this would be to recompile all the packages depending on rtl.bpl and fmx.bpl, but since I don’t have all the sources required for that, I couldn’t do that (but Embarcadero should).

What I did instead was replacing the .dcp files in libs\win32\debug and libs\win32\release with the .bak files that the Patch created. Afterwards, GExperts compiled again. And since probably the interface of the bpl files didn’t change, I don’t expect any problems. But of course, I can’t guarantee this. So, if you follow my example, you might still run into problems and, No, I won’t fix your computer. 😉

Deleting old backups

 Linux  Comments Off on Deleting old backups
Feb 162018
 

Let’s say you have a directory of backups looking like this:

.\
 \-> user1\
          \-> [date1]_[time1]
          \-> [date2]_[time2]
          \-> some more sub directories with date and time in the name
 \-> user2\
          \-> [date3]_[time3]
          \-> [date4]_[time4]
          \-> some more sub directories with date and time in the name
 \-> some more user sub directories

Where [dateN] is the date of the backup starting with a 4 digit year, followed by a two digit month and day, e.g. 20160531.

Now, you run out of disk space and you want to delete the oldest backups, let’s say those from 2015 and 2016. How do you do that?

You could, of course write a program, or, if you are more of a scripting person, a script, that

  1. recurses through the first level of sub directories
  2. looks for sub directories starting with 2015 or 2016
  3. deletes these recursively

Or, you could combine the shell commands find and rm:

find . -maxdepth 2 -mindepth 2 -type d -name "2015*" -exec rm -r {} \;
find . -maxdepth 2 -mindepth 2 -type d -name "2016*" -exec rm -r {} \;

find searches for files and directories that match the given query and does something for each file found, which in this case is call the command rm. But lets have a look at the specific commands above. It restricts the results by the following conditions:

  • “.” (a dot) means: Start in the current directory
  • “-maxdepth 2” means: Recurse sub directories down to two levels maximum
  • “-mindepth 2” means: Recurse sub directories down two levels minimum
  • “-type d” means: Only process directories (not files or devices or links)
  • “-name “2015*”” means: Process only entries whose name matches the shell wildcard “2015*”, so starts with “2015”
  • “-exec rm -r {} \;” means: For each entry execute the command “rm -r {}”, where {} is a place holder for the current entry name.

If you want to test the find command without risking to lose data, leave out the -exec part at the end. The find command will then simply output the entries it finds.

find . -maxdepth 2 -mindepth 2 -type d -name "2015*"

My improved version of IMAPCopy

 Delphi  Comments Off on My improved version of IMAPCopy
Feb 142018
 

IMAPcopy is a tool written by Armin Diehl. Its purpose is to copy emails from one IMAP server to another (or from one IMAP account to another) including the folder hierarchy and even the read status. It’s a command line program written in Delphi and fpc.

Since I am currently migrating a mail server I had use for such a tool and this one was the only one that did not require Java or dotNET, so I tried it. Of course, being a software developer, I found it lacking and improved it.

So, what did I improve?

Mostly one part: I did not like the fact that I had to specify the passwords of the IMAP accounts in the configuration file, so I added an option to specify “?” as the password and the tool would then let me input it at run time. I also added colouring to the the output, so it is easier to spot warnings and errors. And last but not least, I fixed a few typos in the messages.

It’s GPL code, so I’m required to give the source code to everybody to whom I give the executable. I could, of course, simply use the improved version and not tell anybody about it, so I could keep the source code “secret”, but on the other hand, why should I?

So, here is the source code. It compiles with Delphi 2007, I have not tested any other versions and I am sure it won’t compile with fpc (yes, that’s no improvement, but I don’t care).

It also needs a a Delphi port of the Borland CRT unit. I used the one released 1998 by Sergey Perevoznik, which is NOT licensed under the GPL. I have no idea where I got it from and I am not sure about the copyright since it looks like it started out as a copy of Borland’s source code. So I am not going to provide it here, sorry. But you might have a look at Embarcadero’s CodeCentral for alternatives.

Extended = Double in Lazarus Win64

 Delphi, Lazarus  Comments Off on Extended = Double in Lazarus Win64
Feb 112018
 

Note to self: In Lazarus Win64 there is no floating point type Extended, it’s mapped to Double. So, overloading something like:

function TryStr2Float(const _s: string; out _flt: Extended; _DecSeparator: Char = '.'): Boolean; overload;
function TryStr2Float(const _s: string; out _flt: Double; _DecSeparator: Char = '.'): Boolean; overload;
function TryStr2Float(const _s: string; out _flt: Single; _DecSeparator: Char = '.'): Boolean; overload;

will not compile.

Workaround: Add an {$IFNDEF Win64} around it:

{$IFNDEF Win64}
function TryStr2Float(const _s: string; out _flt: Extended; _DecSeparator: Char = '.'): Boolean; overload;
{$ENDIF}
function TryStr2Float(const _s: string; out _flt: Double; _DecSeparator: Char = '.'): Boolean; overload;
function TryStr2Float(const _s: string; out _flt: Single; _DecSeparator: Char = '.'): Boolean; overload;

This probably also applies to Delphi when compiling for others than the Win32 target, e.g. Win64, Android, IOS etc.

Warning: RAD Studio 10.2.2 Tokyo February 2018 Patch has issues

 Delphi  Comments Off on Warning: RAD Studio 10.2.2 Tokyo February 2018 Patch has issues
Feb 102018
 

Embarcadero just released a patch for RAD Studio 10.2.2. I downloaded and installed it, and regretted doing that: It breaks linking projects with runtime packages, like GExperts, any other IDE expert and probably also most component packages.

See here for a discussion.

I submitted a bug report as RSP-19914.

Web Proxy Autoconfiguration

 Linux  Comments Off on Web Proxy Autoconfiguration
Feb 072018
 

Since the last time I looked up how to configure the Web Proxy, apparently somebody came up with WPAD – the Web Proxy Auto-Discovery Protocol (Or maybe I simply missed it).

The idea is quite neat: In the dhcp server, add an entry where the browser can request an url which in turn returns the information for Proxy configuration. Alternatively, instead of the dhcp server, the name server can have entries for wpad.company.com or wpad.department.company.com which are then used to request the configuration.

I decided I’d go with the dhcp entry (bust just in case I also added a wpad entry to bind). So, what exactly needs to be done?

1. Add an option local-pac-server with number 252 and type text to the dchp configuration and set its value to the download url of a wpad.dat file. This can be done for the isc-dchp-server with an entry like this to the /etc/dhcp/dhcpd.conf file:

option local-pac-server code 252 = text;
option local-pac-server "http://someserver.excample.com/wpad.dat";

2. Set up a web server on that computer that serves the file. I didn’t want to install a full blown Apache for this, so I went with mini_httpd. I simply installed the Ubuntu package for it and made two changes to the configuration file /etc/mini-httpd.conf:

# was:
# host=localhost
host=[the IP adress]
# was
# data_dir=/var/www/http
data_dir=/var/www

3. Create a wpad.dat file in the root of the data_dir like this:

function FindProxyForURL(url, host) {
return "PROXY proxy.example.com:8080; DIRECT";
}

4. Wonder WhyTF this was so complicated:

  1. Why not simply configure the string “PROXY proxy.example.com:8080; DIRECT” in the dhcp server?
  2. Why return a text file with a JavaScript function rather than a text file with just the string that JavaScript function returns?

This worked fine even for the Avira Antivirus updater.

how resolvconf generates /etc/resolv.conf

 Linux  Comments Off on how resolvconf generates /etc/resolv.conf
Feb 072018
 

In olden times, we would add entries for name resolution to /etc/resolv.conf and be done with it. Nowadays, with these newfangled scripts that change the configuration all the time, this file simply gets overwritten by a tool/library called resolvconf, so if we want to add something permanently to it, we must do it somewhere else.

Fortunately it’s quite easy, once you know where:
resolfconf uses the directory /etc/resolvconf/resolv.conf.d as the base for its entries. It usually contains three files:

  • head
  • base
  • tail

(And sometimes a file called original which contains the original contents of /etc/resolv.conf before resolvconf was installed. This file is ignored.)

To add something permanently, just edit the file “head” and be done with it.

But wait, there is more:
Most likely you don’t want to add the information to “head” but rather add it to the iface entries in the /etc/network/interfaces file. It allows you to add one or more name servers like this:

iface eth0 inet static
  address 192.168.1.3
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameserver 192.168.1.254
  dns-nameserver 8.8.8.8
  dns-search foo.org bar.com

Switching a XenServer VM from HVM to PVM

 Linux, xenserver  Comments Off on Switching a XenServer VM from HVM to PVM
Feb 032018
 

(Disclaimer: I am by no means an expert with XenServer. So please don’t take anything you read here for granted. It’s my own experience and what I found in documentation and online.)

Switching a XenServer Linux VM from hardware assisted virtualization to paravirtualization nowadays is quite simple, since most Linux distributions already come with a Xen aware kernel (Ubuntu 16.04 defintely does, but you should check). So, most of what is described here is no longer necessary.

The very first thing to do, is this: Take a snapshot of your working HVM. So, if anything goes wrong, you can easily revert to the snapshot. If the VM is too large (e.g. you just virtualized a large file server), you can get away with temporarily detaching the data disks from the VM, take the snapshot and re-attach the data disks again.

Now, if you haven’t already done that, install the XenTools as described in point 30 and following of the article linked above:

  1. Attach the “guesttools.iso” image to the virtual DVD drive or your VM
  2. Mount it from the console (or via ssh)

    mount /dev/disk/by-label/XenServer\\x20Tools /mnt/cdrom
    
  3. Goto the directory named “Linux” and run the install.sh script
  4. Reboot to make sure the VM still works

Once you have verified that, get the boot parameters from the first menu entry in /boot/grub/grub.cfg

You need:

  • The kernel (or make sure that /vmlinz points to the right kernel)
  • The ramdisk (or make sure that /initrd.img points to the right ramdisk)
  • The boot parameters for the kernel, in particular the UUID of the root partition

You should verify that the UUID is correct otherwise you will end up with an unbootable system!

Shut down your VM.

Once you have all that, use either the script listed on the linked article or download my slightly modified version.

Log in to the console of the physical host on which the VM resides (I suggest using ssh) and execute the script.

It will ask you for the name of the VM to paravirtualize. If you are not sure, enter L to get a list.

After entering the name, you will be asked several questions. For most of them it should be safe to simply press enter and go with the default, but for one it isn’t:

Specify Kernel arguments (root=UUID=... ro quiet):

You must specify the kernel parameters, in particular the UUID of the root partition here. If you don’t your VM will not boot.

Once the script exits, everything is done. XenCenter should now show the VM as being in Virtualization mode “Paravirtualization (PV)”.

Boot the VM and enjoy.

What? It doesn’t boot? You get a grub error? OK, you did make a snapshot as I told you above, didn’t you? If yes, simply revert to that snapshot and try again, just in case you made a mistake. You did not make a snapshot? You’re an idiot. Yes, I mean that, I was an idiot too and I regretted it, why do you think I wrote the previous article Switching a XenServer VM from PVM back to HVM? Try the steps I list there and you might get lucky.