Warning Hunt: Hudson and Charts on FindBugs, PMD, Warnings, and Test Results

"Drink, ye harpooners! drink and swear, ye men that man the deathful whaleboat's bow -- Death to Moby Dick!" Once again, my sincere apologies to the Whales and Whale lovers out there, just remember Moby Dick is just a fictional story and I am in no way condoning the slaughter of those innocent and fantastic animals.

Hudson and Trends

Using Hudson to watch or monitor trends in your projects is a fantastic tool, although my graphs in this example are not that interesting, you can see how this could help in keeping track of recent test, warnings, findbugs, and pmd trends:

This blog should demonstrate how you can reproduce what you see above.

To get Hudson and start it up, do this:

1. Download Hudson (hudson.war)
2. In a separate shell window, run:

   java -jar hudson.war

Congratulations, you now have Hudson running. Now go to http://localhost:8080/ to view your Hudson system. Initially you need to add some Hudson plugins, so you need to:

1. Select Manage Hudson
2. Select Manage Plugins
3. Select Available
4. Select the check boxes for the following plugins: Findbugs Plugin, Mercurial Plugin, and PMD Plugin
5. Select Install at the bottom of the page.
6. Select Restart Now

We need a small Java project to demonstrate the charting, and one that has been setup to create the necessary charting data. You will need Mercurial installed, so if hg version doesn't work, get one of the Mercurial binary packages installed first. I will use this small jmake project from kenai.com. So get yourself a mercurial clone repository with:

hg clone https://kenai.com/hg/jmake~mercurial ${HOME}/jmake

and verify you can build it from the command line first:

cd ${HOME}/jmake
ant import
ant

The ant import will download junit, findbugs, and pmd. Access to junit, findbugs, and pmd could be done in a variety of ways, this import mechanism is just happens to be the way this ant script is setup.

This jmake build.xml script is already setup to create the necessary xml files that Hudson will chart for junit, findbugs, and pmd. You may want to look at the build.xml file, specifically the lines at the end of the file for how this is done. Your build.xml needs to create the raw data (the xml files) for the charting to work. This build.xml file happens to be a NetBeans configured ant script so it can be used in NetBeans too.

Now let's create a Hudson job to monitor your jmake repository. Go to http://localhost:8080/ and:

1. Select New Job
2. Name the job, I'll use "jmake".
3. Select Build a free-style software project.
4. Select OK.

Next we need to configure the job. You should be sitting in the job configuration page now, this is what you need to do:

1. Under Source Code Management, select Mercurial and enter the path ${HOME}/jmake.
2. Under Build Triggers select Poll SCM and enter */5 * * * * so that the repository is checked every 5 minutes.
3. Under Build select Add Build Step and Invoke Ant, add import as the ant target.
4. Select Add Build Step again and Invoke Ant, leave the ant target blank.
5. Select Publish JUnit test report result and provide the path build/test/results/TESTS-*.xml
6. Select Scan for compiler warnings.
7. Select Publish PMD analysis report and provide the path build/pmd.xml.
8. Select Publish Findbugs analysis report and provide the path build/findbugs.xml
9. Finally hit the Save at the bottom of the job configuration page.

Lastly, you need to use the Build Now a few times to build up some build history before the charts show up.

This Hudson job will automatically run when changes are added to the repository.

Hope this helps. Using Hudson for this is a definite plus, everyone should invest some time in setting these kind of systems up.

-kto

Posted at 01:21PM Aug 14, 2009 by kto in Java  | 

Windows Visual Studio Stupid Pet Trick of the Day

Sept 1, 2009: Updated script, now works with MKS shell environments.

Re-posting this blog about a shell script interface to the Visual Studio environment setting files vsvars32.bat or vcvars32.bat. Mine is called vsvars.sh. This was my Windows Visual Studio Stupid Pet Trick of the Day, but it's turned into the stupid pet trick of the year. It's mostly for you CYGWIN Visual Studio Users.

The vsvars.sh script will exec a Windows command shell, have it spit out the environment variables it sets, and then it echos out shell environment variable setting commands. And here it is:

Running "vsvars -v7" will get you the settings for Visual Studio 2003:

Nasty looking stuff, but matches what the official Visual Studio vcvars32.bat (or vsvars32.bat) files do to the Windows command shell environment. Note that not all these must be defined in the environment before running Visual Studio, depending on what you are running the critical ones are: INCLUDE, LIB, PATH, LIBPATH, VCINSTALLDIR, VSINSTALLDIR, MSVCDIR, and DEVENVDIR. The first ones INCLUDE, LIB, PATH, and LIBPATH are needed by the compiler, LIBPATH for the newer compilers. The others may be needed by other Visual Studio tools.

The nice thing is that you can use it in a shell environment like so:

eval `vcvars.sh -v7`

By default it should figure out what kind of shell you are using and do the right thing. Use the -help option to see all the options:

It works pretty slick... although I can't possibly test it in all situations.

This will probably be hooked into the JDK builds at some point, or provided with those sources.

-kto

Posted at 11:24AM Aug 14, 2009 by kto in Java  |  Comments[4]

Windows PATH Pecking Order

Just a reminder to the crazy JDK Windows developers out there ;^) The order of paths in the PATH variable is critical:

First Visual Studio Then CYGWIN Lastly the Windows system32 directories

The link.exe needs to come from Visual Studio, not CYGWIN. And the standard Unix utility named tools (like grep) need to come from CYGWIN.

-kto

Posted at 09:56AM Aug 13, 2009 by kto in Java  |  Comments[2]

Anatomy of the JDK Build

The recent "javac -target 7" changes to the jdk7 repositories has prompted me to post this blog about OpenJDK builds. To those unfamiliar with Java classfile versions, each JDK release typically has upgraded the version of the classfiles created by the javac compiler, newer JDKs understand it's own classfile version, plus all the older versions. But older JDKs can NOT understand the newer classfile versions, and their javac instances will not understand how to create newer versions, which of course, makes perfect sense, not sure why I had to say that... oh well... The JDK itself usually contains the newest classfile version, however JDK6 delivered JDK5 classfiles. With JDK7 we want to deliver the latest classfile version in the JDK itself, so the way the JDK is built becomes more interesting to know.

See Alex's version blog, Joe's version blog, and Joe's build advice blog for more information.

When building the OpenJDK from scratch, there are 7 repositories involved, many different system components used in the build process, and an ordering or dependency chain.

For now I'll ignore the build variations or flavors of builds, such as product (optimized), fastdebug, and debug, which generally just repeat the same steps but with slight variations on how the bits are built. What I wanted to do here was describe the dependencies between the repositories and the bootstrap jdk used in the build.

Granted this could change, but currently building OpenJDK JDK7 sources from scratch happens in this order:

1. The langtools repository (repository that holds javac, javah, etc.) is built first using the bootstrap jdk6 javac. This creates a bootstrap javac.jar compiler that can be run with the bootstrap jdk. This bootstrap javac.jar is capable of compiling and creating the newer jdk7 source and classfiles, e.g. it understand -target 7 and -source 7.

   Note that the langtools sources need to be jdk6 compiled sources, and when this repository is built, it also uses the bootstrap javac.jar to compile itself into -target 7 classfiles to be put into the final jdk7 image. Keep in mind that we cannot run jdk7 classfiles yet, we can create them using the bootstrap jdk and the bootstrap javac.jar, but we have no jdk7 image to run yet.

2. The corba repository is built using the bootstrap jdk and the bootstrap javac.jar from the langtools build.
   
3. The jaxp repository is built using the bootstrap jdk and the bootstrap javac.jar from the langtools build.
   
4. The jaxws repository is built using the bootstrap jdk and the bootstrap javac.jar from the langtools build.

   Note that up to this point, these are jdk7 classes, but built with a jdk6 runtime (the jdk6 rt.jar classes). The jdk7 rt.jar has not been built yet. This has not been an issue in that the sources in corba, jaxp, and jaxws are restricted to jdk6 interfaces and have no need for jdk7 specific interfaces. This is also why a jdk5 cannot be used as the bootstrap jdk, jdk5 does not have all the interfaces needed by the above repositories.

5. The hotspot repository is built using the native C++ compiler.

   Note that hotspot includes java source, however that source is currently compiled with the bootstrap jdk javac, not the langtools bootstrap javac.jar. This may need to change in the future, but there may be some requirements for using -target 6 or even -target 5 with whatever javac is used here.

6. Lastly the jdk repository is built. This repository pulls all the above builds together, and also builds the rest of the jdk using the bootstrap jdk and the langtools bootstrap javac.jar. The actual javac included in the jdk7 image will have been the -target 7 built classes. Eventually it will have created enough of a jdk7 image that it could be run on it's own, at that point the demos are built and the image is finalized.

   Note that when building these jdk sources, they are built in a particular order defined by the Makefiles, and against themselves, so that in a sense, these classes are compiled against what will become the jdk7 runtime.

A really good test at this point is to use this newly built jdk7 as the bootstrap jdk, and build a second jdk7. It should work. The top level Makefile has an option to do this make SKIP_BOOT_CYCLE=false.

You can build each repository separately and most developers do, but now that we have this -target 7 requirement, you may need to specify the bootstrap jdk as jdk7, include the langtools repository as part of your build, or set ALT_LANGTOOLS_DIST to point to the langtools/dist directory (a built langtools repository).

Hope this helps to explain things a little. Please post any questions you have, I'll try and answer all I can. And hopefully Jonathan will keep me honest here. :^)

-kto

Posted at 12:21PM Aug 12, 2009 by kto in Java  |  Comments[6]