Converting from Subversion to Mercurial

 Delphi, Mercurial  Comments Off on Converting from Subversion to Mercurial
Dec 262013
 

Here, I will be posting notes on converting Subversion repositories to Mercurial.

NOTE: This is work in progress. I will probably come back and change things as well as adding.

Some links on using Mercurial:
Hg Init: a Mercurial tutorial by Joel Spolsky
QuickStart in the Mercurial Wiki

I have got TortoiseSVN 1.8.3 including the command line clients and TortoiseHG 2.10.1 installed.

There are several short tutorials on converting from svn to hg. They all suggest that you first create a local copy of your subversion repository to work on.

A – Create a local copy of your subversion repository
Creating this local copy can easily be done as described by Massimo Iacolare, which I used as a guide.

1 – Create a local svn repository
You can use any local directory as the path to your local svn repository, I will be using the extension .svn to remind me that it is a svn repository. The converted Mercurial repository will not get an extension.

Unfortunately there is the first gotcha: Massimo seemed to be using an older version of svn that used db format 4. The current svn 1.8 uses db format 6 by default, and as Jeroen W. Pluimers found out, the current Mercurial does not support it yet. So, what can we do until the Mercurial developers fix this problem? Why, use db format 4, of course:

svnadmin create --compatible-version 1.7 [path\to\your\new\repository.svn]

I am not sure whether the steps 2 to 4 are necessary, it seemed to work fine without them. I’ll keep them here just in case I run into trouble later because I left them out.

2 – Enable writing on your new repository
Open the file [path\to\your\new\repository.svn]\conf\svnserve.conf and uncomment the line

[general]
password-db = passwd

This tells SVN which file stores the user credentials.

3 – Add a new user
Open the file [path\to\your\new\repository.svn]\conf\passwd and add a new user:

[users]
svnsync_user = svnsync

4 – Give your new user read/write permissions
Open the file [path\to\your\new\repository.svn]\conf\authz and add:

[/]
svnsync_user = rw

5 – Enable rev propchange (revision property change)
create a text file pre-revprop-change.bat in [path\to\your\new\repository.svn]\hooks containing:

exit 0

This can be done easily with

echo exit 0 > [path\to\your\new\repository.svn]\hooks\pre-revprop-change.bat

6 – Initialize the repository

svnsync init file:///[path/to/your/new/repository.svn] [REMOTE PATH]

Getting the local repository path right can be tricky. Pay close attention to the number of slashes and use forward slashes. e.g.

svnsync init file:///c:/some/path/here.svn https://svn.somesvnserver.com/svn/repository

7 – Synchronize
Once the repository is initialized, you can synchronize it as often as you like:

svnsync sync file:///[path/to/your/new/repository.svn]

Be aware, if your repository is big, the first synchronization may take very long.
After the synchronization has been done once, you can keep it in sync by just repeating step 7.

B – convert your local Subversion repository to a Mercurial repository

This is a one step action:

hg convert [path/to/your/new/repository.svn] [path/to/your/hg/repository]

If you omit the second parameter, hg convert will assume [path/to/your/new/repository.svn-hg] which is probably not what you want, but you can easily rename the directory later on.

The above converts the whole svn repository, including any branches to a single Mercurial repository, as this command shows:

hg branches -a

(You must execute it inside the repository.)
This will also list the pseudo branch “default” which corresponds to the “trunk” in svn.

You can also specify any path within the svn repository to convert, e.g. only the trunk:

hg convert file:///[path/to/your/new/repository.svn]/trunk [path/to/your/hg/repository]

Alternatively, you can convert the whole repository and clone it for every branch to a new Mercurial repository, like this:

hg clone -r [branchname] [path/to/your/hg/repository] [path/to/your/hg/branch]

Due to different concepts of users in svn and Mercurial (one uses names, the other uses email adresses), you can provide a mapping file for users to emails. It must contain one line per user like this:

arist = Aristotle <aristotle@phil.example.gr>
soc = Socrates <socrates@phil.example.gr>

(This example is taken from Mercurial: The Definitive Guideby Bryan O’Sullivan)
And you pass it to the conversion with the –authors option.

 Posted by on 2013-12-26 at 21:31

Some Batch File Wizardry for Automated Builds

 Delphi, dzLib  Comments Off on Some Batch File Wizardry for Automated Builds
Dec 222013
 

At work and also on my private computer I am maintaining multiple Delphi projects that are (currently) managed in subversion repositories. They have a general structure like this:

projectname
  \- buildtools (-> svn:external)
  \- src (project sources)
  \- libs
    \- dzlib (-> svn:external)
    \- some other libraries, all svn:external

Being a lazy bastard (deutsch: faule Socke) I don’t want to start Delphi just to compile the projects or check whether they still compile but have a build script to do that. Also, I don’t want to right click and svn update all the time (and build after that) so I have got an autobuildcheck script. And last but not least, since on my computer all Delphi versions since version 6 are installed, I need a method to automatically launch the correct IDE for each of the projects.

First, auto_build_check.cmd:

That one is simple. It needs to

  • svn update
  • call the build script
  • revert the build number increment which the build script automatically made
  • if no error occurred, just exit, otherwise pause to let me examine the error message
rem Update sources from repository
rem Build them
rem in case of error(s): Stop and show the error
rem revert the automatic build number incrementation

svn update
if errorlevel 1 goto Error

set BatchBuild=1
call _BuildProject.cmd

if not exist src\*_version.ini goto :eof
svn revert src\*_version.ini
if errorlevel 1 goto error

goto :eof

:error
echo ***** ERROR *****
pause

Next, _buildproject.cmd:

It needs to

  • determine the project name from the current directory (I don’t want to write different build script for each project, lazy bastard again)
  • determine the Delphi version to use for building (this one only supports Delphi 2007 and later)
  • call msbuild
  • if an error occurs, show that error
  • pause, to let me examine the results
rem @echo off
REM - assumes that the parent directory is also the project name, extracts it
REM - and calls msbuild with %project%.dproj

setlocal

rem Has the project been passed as parameter?
set project=%1
if not "%project%"=="" goto projgiven
rem Is there a __SetProjectName.cmd which supposedly sets %project%?
if exist __SetProjectName.cmd call __SetProjectName.cmd
if not "%project%"=="" goto projgiven
rem Still no project, use the parent directory name to get it
call :GetLastDir %0
set project=%result%

:projgiven

call buildtools\delphiversions.cmd

set DelphiVersion=Delphi 2007
set DelphiDir=%Delphi2007Dir%
if exist __SetDelphiVersion.cmd call __SetDelphiVersion.cmd

set OldPath=%PATH%
echo building project %project%.dproj using %DelphiVersion%
call "%DelphiDir%\bin\rsvars.bat"

if NOT "%DelphiVersion%"=="Delphi 2007" goto no2007
rem Delphi 2007 sets the FrameworkDir incorrectly on 64 bit Windows, so we fix this here
SET FrameworkDir=%SystemRoot%\Microsoft.NET\Framework\
SET PATH=%FrameworkDir%%FrameworkVersion%;%FrameworkSDKDir%;%OldPath%
:no2007

pushd src
msbuild %project%.dproj | ..\buildtools\msbuildfilter ..\errors.txt
popd
if errorlevel 1 goto error

endlocal
if "%BatchBuild%"=="1" goto nopause
pause
:nopause
goto :eof

:error
echo ************************************
echo ***** Error building %project% *****
echo ************************************
if "%MAILSend%"=="" goto nomail
%MAILSend% -sub "Error building %project%" -M "Build Error" -attach errors.txt,text/plain,a
endlocal
goto :eof

:nomail
endlocal
pause
goto :eof

:GetLastDir
rem extract path
set directory=%~p1%
rem remove backslash (=last character)
set directory=%directory:~0,-1%
rem extract "filename" (= last directory of path)
call :LastItem %directory%
goto :eof

:LastItem
set result=%~n1%
goto :eof

Actually, this script does some more things:

  • It allows me to pass the project name as a parameter.
  • If no parameter is passed, it looks for a file called __SetProjectName in the current directory, which can set %project%.
  • If that doesn’t exist either, it calls the GetLastDir subroutine to get the last part of the current directory and uses that as the project.
  • next, it calls buildtools\delphiversions.cmd which is a script that knows where all my Delphi versions are installed and sets the base directories for them as Delphi2007Dir, Delphi2009Dir etc. environment variables.
  • Determine wichh Delphi version it should use. It assumes Delphi 2007 (which I still use for the majority of my projects), looks for __SetDelphiVersion.cmd in the current directory, which is supposed to set %DelphiVersion% and %DelphiDir% if a different version is to be used.
  • If Delphi 2007 is to be used, it implements a workaround to fix a problem this version has with setting %FrameworkDir% on 64 bit Windows.
  • It calls rsvars.bat (which is a script installed by Delphi)
  • It switches to the src subdir
  • It calls msbuild for %project%.dproj (msbuildfilter is an internal tool that filters the output and counts hints, warnings and errors)
  • If there is no error it either pauses, or if in batchbuild mode, it just exists.
  • If there is an error, it checks whether it should send an e-mail and does that (I’ll come back to that in a future blog post.).
  • If it sent an e-mail, it exists, otherwise it pauses to let me examine the error.

Last, _OpenInIde.cmd

It needs to:

  • Determine the project name (see above)
  • Determine the Delphi version to use (again: see above)
  • Call the IDE of that version and pass it the appropriate .dproj file
@echo off
REM - assumes that the parent directory is also the project name, extracts it
REM - and calls the IDE with %project%.dproj

setlocal

rem Has the project been passed as parameter?
set project=%1
if not "%project%"=="" goto projgiven
rem Is there a __SetProjectName.cmd which supposedly sets %project%?
if exist __SetProjectName.cmd call __SetProjectName.cmd
if not "%project%"=="" goto projgiven
rem Still no project, use the parent directory name to get it
call :GetLastDir %0
set project=%result%

:projgiven

call buildtools\delphiversions.cmd
set DelphiVersion=Delphi 2007
set DelphiDir=%Delphi2007Dir%
if exist __SetDelphiVersion.cmd call __SetDelphiVersion.cmd

start "%DelphiVerson%" "%DelphiDir%\bin\bds.exe" -pDelphi src\%project%.dproj
endlocal
goto :eof

:GetLastDir
rem extract path
set directory=%~p1%
rem remove backslash (=last character)
set directory=%directory:~0,-1%
rem extract "filename" (= last directory of path)
call :LastItem %directory%
goto :eof

:LastItem
set result=%~n1%
goto :eof

So, am I lazy? I don’t really know. Writing the scripts described above was quite a lot of work. On the other hand, that was interesting work and it keeps me from making stupid errors again and again, like opening a project with the wrong Delphi version and having to revert the changes done by that version, or forgetting to do svn update before checking if the project still compiles. Also, they allow me to run a continuous build server to automatically find compile errors caused by incompatible changes to one of the libraries used, before I or my coworkers forget the reason for that change.

But why have copies of these scripts in all my projects rather than having them in buildtools and only small stubs in the project, that call these scripts? So that’s the next step.

If you want to use these scripts in your own projects or roll your own based on them, feel free to do so. You can find them in my buildtools repository.

 Posted by on 2013-12-22 at 20:48

Multiple Batch Files in Pre-/Postbuild Events in Delphi / msbuild

 Delphi  Comments Off on Multiple Batch Files in Pre-/Postbuild Events in Delphi / msbuild
Dec 212013
 

Since version 2007 Delphi supports pre- and postbuild events for projects. If you only want to start programs or one batch file it’s a simple matter of adding the call to the respective section of the project options dialog. I use it to manage version numbers in the program resources and manifest files in prebuild and append translations and jcl debug info in postbuild and it works fine.

But a few days ago, I wanted, in addition to the above, to copy some dlls that the program needs to the output directory. So I went and added another batch file like this:

..\..\buildtools\postbuild.cmd $(OUTPUTDIR)$(OUTPUTNAME)
..\copydlls.cmd $(OUTPUTDIR)

To my astonishment I found that only the first of the batch files was executed. After looking for the obvious problems (like typos, wrong paths etc.) and adding some debugging code to the batch files which confirmed the assumption, a question on StackOverflow seemed to be in order (which usually works much better than posting to the Embarcadero forums, because of the great guys frequenting it.) While I was waiting for an answer I continued to experiment and found one workaround:

%comspec% /c ..\..\buildtools\postbuild.cmd $(OUTPUTDIR)$(OUTPUTNAME)
%comspec% /c ..\copydlls.cmd $(OUTPUTDIR)

This was like a flashback to the bad old days of DOS, but it worked. I added this result to my question and while I was at it, checked for answers. Still none (but then it had been less than 5 minutes since I posted it. So I started looking for similar questions on StackOverflow that were not tagged Delphi but MSBuild. I found this answer and tried it:

call ..\..\buildtools\postbuild.cmd $(OUTPUTDIR)$(OUTPUTNAME)
call ..\copydlls.cmd $(OUTPUTDIR)

It also worked so I added it to my question as well. It looked like a more elegant solution than the %comspec% hack to me, so after more digging didn’t turn up anything else, I posted it as an answer to my own question and went on with my work.

Around half an hour later I looked at the question again and found that it had been answered by David Hefferman. I posted the question at 15:19 and got his answer at 15:36 which is not unusual for StackOverflow (At least this time he didn’t beat me to the solution, but hey, I was actively working on that problem while he was probably doing something completely different.).

 Posted by on 2013-12-21 at 18:40