Kelly O'Hair's Weblog (blogs.sun.com)

Ant and Importing

Just how many copies of junit.jar have been added to source repositories on the planet? Quite a few I imagine, seems like a waste of repository data space and well, just wrong. Not junit, which is a fantastic product, just the fact that we have so many copies. Granted you have gained a pretty stable tool by freezing the version you have, and you have guaranteed having a copy at all times, but is it a good idea to add all these binary fines to your repository data? As the list of tools like this grows and grows, does the "just add it to the repository" solution continue to scale? And each time you need a new version, you end up adding even more binary data to your repository.

Some projects have taken to doing a kind of "tools bootstrap" by downloading all the open source tools the first time you setup a repository, making the files immune from normal 'ant clean' actions. Ant has a a task called <get> which can allow you to download tool bundles and it works quite well, but there are some catches to doing it this way. Expecting all the download sites to be up and available 24/7 is not realistic. And predictability is really important so you want to make sure you always download the same version of the tools, keeping a record of what versions of the tools you use.

So what we did in the openjfx-compiler setup repository, was to create an import/ directory to hold the downloaded tools, automate the population of that area with the <get> task, and also allowed for quick population of import/ with a large import zip bundle. The initial version of the repository had a very similar mechanism, so this idea should be credited to the original authors on the OpenJFX Compiler Project.

This logic is contained in the file build-importer.xml of the setup repository and for each tool NAME downloaded, a set of properties is defined (import.NAME.*), and 2 ant tasks import-get-NAME and import-NAME. Probably best to look at the bottom of this file first. As before quite a few macrodefs were used to make this all work.

The ant build script then just uses ${import.junit.jar} to get a junit.jar file.

You can actually try this out yourself pretty easily if you have Mercurial (hg) and ant by doing this:

hg clone https://kenai.com/hg/openjfx-compiler~marina-setup setup
cd setup
ant import

Of course I'll predict that it fails the first time for 50% or more people, this kind of downloading is just not that reliable when depending on all these sites. So you may have to run ant import a few times.

-kto

Posted at 04:36PM Feb 28, 2009 by kto in Java  |  Comments[5]