Naresh's Blog

Continuous Integration Tools

     A continuous integration tool is an automated build system that checks out the most current code from the source code repository, builds it, and makes the resulting artifacts available for download. Such tools come handy when your application has multiple modules and there are multiple engineers working on them. These tools could be used to integrate these various modules, build the system and also maybe run some tests to ensure that everything is fine.

Hudson: What is it?

     Hudson is an open-source continous integration tool, which has become very popular for some time now. It provides various features like options for checking out the source code from various version controlling systems like CVS, SVN, etc., setting the version of Java to be used, the machine(s) on which to run the integrations/builds, notifying an user or a group of users about a build failure, scheduling the job execution, etc. More information can be obtained from the Hudson project site at http://hudson-ci.org/.

Master-Slave Configuration

          One interesting feature of Hudson is its provision to run a job in a master-slave configuration, i.e., there would be a Hudson master machine which would take up all the requests like defining a job, configuring it, triggering it, etc., while there will be a set of slave machines on which the executions would actually happen.

        Lets say you want to define a job which has to be run on more than one platform. You define it in the Hudson master, the interface for all your configurations, and then tie it to all the machines that you would want to run the job on. The underlying mechanism of Hudson takes care of all the communication between the Hudson master and each of the slaves which are registered with it.

        Now, we will see how to setup Hudson - the master and slave(s).

Setting up the Hudson Master

    The master could be setup on any OS, for convenience lets assume we are setting it up on Linux.

• If you are using Ubuntu, be sure to upgrade to 8.04 to avoid problems with RSA keys (keys generated by keygenerator in 7.10 are blacklisted!)
• Login as someone with root role (e.g. uadmin)
• sudo useradd -d /space/hudson -m hudson - this creates user hudson with home in /space/hudson (Folder hudson does not need to exist)
• sudo passwd hudson changes the password for user hudson
• Login as hudson user
• Create folder $HOME/jdks and install there jdk1.6.0
• ssh-keygen -t rsa generate public / private RSA key, public key is used for ssh login to slaves without passwords
• =touch /space/hudsonserver/master = - creates foo file master as a workaround for hudson issue #936- parent project occuppies executor on slave
• Download hudson.war bits the Hudson site at http://hudson-ci.org/
  
• Install webserver (e.g. tomcat) and deploy Hudson
• If you do not want to run Hudson in Tomcat, use built-in server Winstone:
  • run java -jar hudson.war to start Hudson master on port 8080
  • NOTE: best idea is to use something like this script (we use in this script non-default port 18080 because this is second instance of Hudson on the same machine):

               #!/bin/bash               
               # kill running hudson
               kill `ps aux|awk '$13 == "./hudson.war" {print $2}'` 2> /dev/null
               # nohup new hudson
               nohup /space/jdks/jdk1.6.0_05/bin/java -jar ./hudson.war --httpPort=18080 --ajp13Port=18009 &

• Once you have the war file deployed, you could launch the application.
  
• Various settings like different JDKs, MAVEN_HOME could be made at the "Manage Hudson" page by clicking on the "Manage Hudson" link in the side pane.

Setting up the Hudson Slave

The following steps could be used to create a slave on a linux/solaris machine:

• Create an user "hudson" with home in /space/hudson.
  • sudo useradd -d /space/hudson -m hudson
• Set a password for this user
  • sudo passwd hudson
• Login as user "hudson".
• Create folders "ant" and "jdks" in /space/hudson.
• Download and unzip ant into folder "ant".
• Download and unzip various versions of jdk into jdks.
• Copy the id_rsa.pub of the Hudson master to file /space/hudson/.ssh/authorized_keys. This enables the Hudson master to establish a remote connection to the slave without having to enter the login credentials. To do this:
  • Setup a FTP connection to the Hudson master.
  • Get the id_rsa.pub file from the hidden folder ".ssh".
  • Close the FTP connection.
  • Copy this id_rsa.pub file to /space/hudson/.ssh/authorized_keys. Create the directory .ssh if it doesn't exist already.
  • To verify that the Hudson master's key is successfully added to this slave's "ssh" keys, try setting up a ssh connection from the Hudson master to this slave node, as "ssh hudson@your_hudson_slave". This should setup a connection without prompting for a password.
• Copy the slave agent "slave.jar" to /space/hudson. The jar file can be obtained from the archive "hudson.war" which has been downloaded to setup the Hudson master. The hudson.war file may be downloaded from the site http://hudson.dev.java.net.
• Write a small shell script which sets the various paths and starts the slave agent. Name it runSlave.sh. A typical script file would be like:
  • cat /space/hudson/runSlave.sh
  • echo "Starting the slave agent on the node XYZ..."
  • export JAVA_HOME=/space/hudson/jdks/jdk1.6.0_06
  • export ANT_HOME=/space/hudson/ant/apache-ant-1.7.1
  • export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH
  • java -jar slave.jar
  • echo "Slave agent started..."
    • This shell script would be called by the Hudson master. Running the script directly would not start the slave agent, this gets started only when run from the Hudson master.
      

• Register this slave machine with the Hudson master in the "Manage Hudson" page.

You have your Hudson Master/Slave configuration ready. Now, you could just go ahead and define, configure and schedule your jobs to run according to your choice.

 In case you have any queries you could consider sending a mail to the Hudson user's mailing list - users@hudson.dev.java.net  which is a pretty active mailing list.

       This entry describes how easy it is to get your Jersey web application running on Google App Engine.

For the illustration purpose lets see how we can deploy the simple Helloworld-Webapp Jersey sample on the Google App Engine.

Simple Steps:

1. Download Google App Engine.
2. Follow the installation instructions for installing the app engine on your machine. It isn't really too much work, just unzip the bundle, and add its bin directory to your path.
3. Download the Jersey helloworld-webapp sample, if you do not already have it.
4. Create an XML file named appengine-web.xml, under WEB-INF parallel to the deployment descriptor web.xml. Google App Engine requires this file in the webapp's WEB-INF directory, for it to be able to run the application.
5. Copy the following content to the created appengine-web.xml file:

   <?xml version="1.0" encoding="utf-8"?>
   <appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
       <application></application>
       <version>1</version>
   </appengine-web-app>
   

6. Package the application using: mvn clean package -Dmaven.test.skip=true
7. Deploy the application on Google App Engine using the command: dev_appserver.sh target/helloworld-webapp
8. You will see the message which says server is running at http://localhost:8080/.
9. In a web browser enter http://localhost:8080/helloworld. You see the application running.

That's it. You got the Jersey application running on Google App Engine :)

     Here's a cool utility for non-unix users, who have been looking for a curl  like solution for sending HTTP requests with the various HTTP methods - GET, POST, PUT, DELETE, etc. It is the Firefox add-on Poster.

         Poster is a  developer tool for interacting with web services and other web resources that lets you make HTTP requests, set the entity body, and content type. This allows you to interact with web services and inspect the results.

        The add-on can be installed from https://addons.mozilla.org/en-US/firefox/addon/2691.

        Once the add-on is installed, you will be able to see an icon P in the Firefox browser status bar.

Clicking this, would popup a Poster window, where  you can enter  the URL to which  you want to send the HTTP request,  set the HTTP method, set the  request headers, any query parameters, etc. 

Also, you will be able to view  the response that is sent back.

        This tool will be very useful at the time of development of RESTful Web Services.

   Supported versions of Firefox: 1.5 - 3.0.*

   In my opinion, this tool is even better that curl, thanks to the GUI. Moreover, it is pretty easy to learn and use, pretty much self-explanatory.