Trond Norbye's Weblog

Bazaar shared repositories and Hudson build slaves

I created a Bazaar plugin for Hudson a while back, and we have been using that with great success on our Hudson build farm to build Drizzle, libmemcached and Gearman.

If you look at our build farm you will see that we compile the same source project multiple times on the same slaves, only with a variation in the configuration (compiler, 32/64 bit etc). With the normal configuration you will check out the complete repository for each of the projects, using a lot of bandwidth and disk space. Disk is cheap these days so I don't care that much about that, but my bandwidth is limited so I would like to reduce that if possible.

If we look at the disk layout on the build slaves it perfectly suited for using Bazaar shared repositories:

$SLAVE_HOME/workspace/

So if we create a Bazaar shared repository in $SLAVE_HOME/workspace, all of our Bazaar projects will store the upstream information in $SLAVE_HOME/workspace/.bzr. Not only will this dramatically reduce the disk footprint, it will also reduce the amount of data being downloaded. So how do you set up a shared repository? It's no magic, simply log into the build slave and run the following command:

trond@storm > bzr init-repository $SLAVE_HOME/workspace

Unfortunately there is a "bug" in Hudson causing Hudson to nuke the directory every day when it is doing housekeeping of the workspace directory. Luckily for us there is a simple workaround for the issue, simply create a project named ".bzr".

Update: Hudson will also nuke the directory if it hasn't been modified on a time period, and it seems that bazaar doesn't update the modified date on the .bzr directory. A simple workaround for this is to install the following cron job:

1 1 * * 0 /usr/bin/touch /home/hudson/hudson/workspace/.bzr

Posted at 09:53AM Oct 17, 2009 by trond in OpenSolaris  |  Comments[0]