Experimenting with Hadoop can be a lot of fun, but if you don't have easy access to your own cluster, your options are somewhat limited. You can run map/reduce jobs locally on a single node, using the "Pseudo" distributed configuration described in the Hadoop Quick Start Guide. While this is a good way to get started initially, it is a somewhat limited environment to play around with, since you only have one DataNode, and one TaskTracker.
We can take advantage of OpenSolaris's Zone capability to create a fully distributed version of Hadoop, in which each Hadoop node runs in its own zone. Zones are virtual machines that look and act like independent machines. Zones are different than VMs like Xen or VMWare in that they have a much smaller memory requirement. Less memory overhead means more memory for your Hadoop jobs.
Creating a fully distributed Hadoop virtual cluster on a single node takes about half an hour or so, depending on the speed of your network. This blog post describes the first part of this process: building a Zone. In Part 2, we will clone that zone to build up our cluster, and in Part 3, we will install and configure Hadoop.
We begin by creating the first zone, which we will call node1, which will run the DataNode and TaskTracker daemons. Once this zone is set up, we can clone it to create additional zones. The global zone (which is always running) will run the JobTracker and NameNode. In terms of networking, we are going to setup our virtual Hadoop cluster so that each zone has the following network addresses:
- Global zone will be 192.168.3.1
- node1 will be 192.168.3.11
- node2 will be 192.168.3.12
- node3 will be 192.168.3.13
The steps to do this are listed here. Under some of the steps are comments, which are in grey italics. You don't have to type the question mark--it just represents the system prompt, which may look different on your machine.
Set up the global zone's networking first. You can type
$ ifconfig -a
to get a list of the network interfaces on the machine. In my case, my wireless card is 'ath0' (yours might be e1000g0, rte0, or another driver). We'll use this now, and it will come in handy later on. Start by creating the NameNode and JobTracker's IP address:
$ pfexec ifconfig ath0:1 plumb
This creates a virtual network interace for the NameNode and JobTracker. Let's assign an address to it:
$ pfexec ifconfig ath0:1 192.168.3.1 netmask 255.255.255.0 broadcast 192.168.3.255 up
Now we create the first DataNode and TaskTracker zone:
$ pfexec zonecfg -z node1
'pfexec' temporarily gives you root privleges on the system. It is similar to 'sudo' on unix. The '-z node1' flag lets you specify the name of the zone. In this case, we're configuring node1.
node1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:node1> create
zonecfg:node1> set zonepath=/export/zones/node1
The zonepath is the location on the filesystem that will hold the data for this zone. Make sure that /export/zones has enough space--at least 10 GB. You can also specify a different path here, if there is one that has more space.
zonecfg:node1> add inherit-pkg-dir
zonecfg:node1:inherit-pkg-dir> set dir=/opt
zonecfg:node1:inherit-pkg-dir> end
zonecfg:node1> add inherit-pkg-dir
zonecfg:node1:inherit-pkg-dir> set dir=/usr/java
zonecfg:node1:inherit-pkg-dir> end
These mean that the local zones are going to 'inherit' /opt and /usr/java, meaning that these two paths will be shared by all the zones. The local zones will only be able to read from these paths--not write.
zonecfg:node1> add net
zonecfg:node1:net> set physical=ath0
Make sure that you put the name of your network interface here. You can find that by typing 'ifconfig -a' on the system prompt. In my case, I'm using the Atheros driver, or ath0. Other likely options are rge0 or e1000g0.
zonecfg:node1:net> set address=192.168.3.11
zonecfg:node1:net> set defrouter=192.168.3.1
zonecfg:node1:net> end
zonecfg:node1> verify
zonecfg:node1> commit
zonecfg:node1> exit
$ pfexec zoneadm -z node1 install
The installation of the first zone will take awhile, depending on the speed of your network. It took me about 15 minutes.
When it is done, you will likely see something like:
Authority: Using http://pkg.opensolaris.org:80/.
Image: Preparing at /export/zones/node1/root ... done.
Installing: (output follows)
DOWNLOAD PKGS FILES XFER (MB)
Completed 52/52 7778/7778 75.02/75.02
PHASE ACTIONS
Install Phase 12867/12867
PHASE ITEMS
Reading Existing Index 8/8
Indexing Packages 52/52
Note: Man pages can be obtained by installing SUNWman
Postinstall: Copying SMF seed repository ... done.
Postinstall: Working around http://defect.opensolaris.org/bz/show_bug.cgi?id=681
Postinstall: Working around http://defect.opensolaris.org/bz/show_bug.cgi?id=741
Done: Installation completed in 483.728 seconds.
Next Steps: Boot the zone, then log into the zone console
(zlogin -C) to complete the configuration process
Great! Now we can boot up the zone and configure it:
$ pfexec zoneadm -z node1 boot
It may complain about 'no matching subnet found in netmasks', which is ok.
We can now proceed to 'system identification', which sets the machine up and builds some keys.
$ pfexec zlogin -C node1
You will have to hit enter a few times to see a menu. This menu will configure the system. Start by picking a terminal emulator. I used option #6 (Xterm). It will then generate some RSA keys and configure your network interface. Eventually it will ask for the host name. Type in 'node1'. To go to the next screen, instead of hitting enter, you have to type Escape, followed by '2' (don't ask!). Say no to Kerberos security, and when it asks for what type of name service to use, just select "none." Use the NFSv4 options derived from the system. Next, select your time zone. Next, pick a root password. After this step, the system should be "identified".
Log in as user 'root', with the password you just picked out. This will bring you to the system prompt within the node1 zone.
First, we create a user, which should have the same userid as you. To find your user id, open a new terminal window (not the console we just opened to node1) and type 'id'. This will print out something similar to:
gporter@pukapuka:~> id
uid=153216(gporter) gid=10(staff)
gporter@pukapuka:~>
-bash-3.2# mkdir -p /export/home
-bash-3.2# useradd -u <userid> -m -d /export/home/<login_name> <login_name>
In my case, I would type 'useradd -u 153216 -m -d /export/home/gporter gporter'
-bash-3.2# passwd <userid>
In my case, I would type 'passwd gporter'
It will ask you for a password. Type one in and confirm it by typing it in a second time.
Disconnect from the zone by typing ~. (The ~ key followed by a period). This should return you to the system prompt.
You (should) now have a running zone on your system that you can log in and treat like a separate machine. Try logging into it:
$ ssh 192.168.3.11
In the next part, we will clone out our newly crated zone.