Draco Dormiens Nunquam Titillandus : Weblog

Wednesday May 02, 2007

OpenSolaris OpenSesame!

If you're one of the many people out there who are intrigued by OpenSolaris, and want to try it out for yourselves, but are worried about how to go about doing it, you're in the right place. In this article, I will show you step by step how to start with a machine with no OS (or running windows) and download OpenSolaris binaries and install it. Then create a 2nd partition using OpenSolaris LiveUpgrade, download OpenSolaris source, compile it and install it on the 2nd partition. In the end, I will walk you through how to modify the source and install the modified kernel on the 2nd partition so that your original OpenSolaris partition and workspace don't get impacted while you are experimenting with modifying the kernel.

There are lot of articles/tutorials on each subject, but if you are a newbie to OpenSolaris but know 'C' programming language and have some *nix kernel experience, then this might serve as a quick and dirty startup for you. For each section, I have given links to real articles in case you need more detailed explanations.

It is important to note that I am a kernel engineer with a few years' experience and work for SUN but I wrote this based on my experiences from installing a machine at home while connected to open internet and my only access was to sources, tools, and articles available publically on OpenSolaris. My other motivation was to help people enhance NWAM project to do more interesting things. NWAM stands for Network AutoMagic and automatically configures the network for machine running Solaris.

Installing OpenSolaris
Creating Liveupgrade partition for My Kernel
Building OpenSolaris from Source
Modifying OpenSolaris kernel

Installing OpenSolaris

The first step was, of course, to get the Solaris Express Developer Edition installed on my machine. There were two options available to me. The first was to get the install bits by downloading them off of the OpenSolaris website. The second was by getting the DVD. I chose the latter, primarily because I had fast access to the DVD (Whichever way you do it is your prerogative, as long as you end up with a working CD/DVD which can be used for the install.)

Before install, it was important to lay out the configuration I wanted for my machine. I wanted to create a multiboot machine which could run OpenSolaris as well as another OS(Windows/Linux). In addition, for OpenSolaris, I wanted a regular root partition for the stable kernel bits, a scratch partition for kernel bits that I could play around with, and space for home directories etc. My machine had 72 GB of hard-disk, of which I decided to use 36 GB for OpenSolaris in its various partitions. I wanted to keep approximately 7 GB for each of the root and scratch partitions, around 20GB for home directory (/export/home), and half GB for swap.

(Please note that if you are starting with a machine that already has Windows/Linux or any other OS installed, and you want to shrink the existing partitition, then I can't provide much help with that. You have to be comfortable using some software like PartitionMagic.)

Once I had the DVD, I restarted the machine using the OpenSolaris DVD. In the GRUB menu that showed up, there were several choices:

Solaris Express, Developer Edition
Solaris Express
Solaris Express Serial Console ttya
Solaris Express Serial Console ttyb (for lx50,v60x and v65x)

I chose the first option - Solaris Express, Developer Edition, since that comes with Sun Studio 11 and NetBeans etc.
In the next menu that showed up, I chose Solaris Interactive.
Next was the option to configure Keyboard Layout - I chose US-English
Up next - Pick a Language - I chose #1 - English
Next - Pick a Continent and country
Next - Enter Time
Next - Enter Root password (Not telling you this one )
Next - Legal Agreement - Have to say yes to this one
Next - Software Language localizations
Next - System Locale - I only cared for the (English Posix C (C))
Next - Select disks for fdisk partition customization - I chose c0d0 you choose whatever is appropriate for your machine.

If you are wondering what the <c0d0 is, Solaris nomenclature uses c to denote the disk controller, d for disk number, and s for the slice or partition on that disk. Within the slices, s2 on any disk denotes the entire disk and should not be mucked with, so cXdYs2 will always denote the entire disk. The typical convention is that slice 0 is for root, slice for 1 swap, and slice 6 or 7 for /export/home. I decide to use slice 5 for my scratch partition, and slice 7 for the home directories (/export/home).

Next was the tricky one -

Customize fdisk partitions - Disk c0d0

This is what I did (note that I had two DOS partitions from before that I didn't touch):

Partition		Size(MB)
x86Boot		12
Solaris		36001
PRI DOS		18001
PRI DOS		22304

Next question was whether I wanted to Preserve Data. Since I was not planning to save the existing data on the hard disk, this decision was easy for me. I chose to preserve the DOS partitions, you'll have to decide for yourselves.

The next question was to determine the layout of the filesystems. I wanted my final layout to look like this:

Disk Modify

Slice 0 / 12

Slice 1 swap 514

Slice 5 --(Empty) 7309

Slice 7 /export/home 20858

Note that the Empty disk-slice will become the scratch partition where I will install the LiveUpgrade bits.

After this, I finally got the screen which said that OpenSolaris was Ready to install. This was the final configuration:

Solaris Operating System: Solaris Software
Solaris Software Group: Entire Group plus OEM
Root device: c0d0
Fdisk partitions:
	Disk		Type	Start Cyl	Size	Action
	c0d0		x86Boot		1	6	existing
	c0d0		Solaris		7	18286	existing
	c0d0		PRI DOS		18293	9143	preserved
	c0d0		PRI DOS		27436	11329	preserved
File Systems
	c0d0s0	/		7309 MB
	c0d0s1	swap		514 MB
	c0d0s5	unnamed		7309 MB
	c0d0s7	/export/home	20858 MB
Region and Locales:
	North America
		English (United States) (en_US)
		English (United States, UTF-8) (en_US, UTF-8)
	English (Posix C) (C)
System Locales: English (Posix C) (C)
Products:
	Solaris Developer Express	1503.0 MB
		Solaris Developer Tools	1503.0 MB

Then came the one that all this was about - "Install Now"

Creating Liveupgrade partition for My Kernel

Once the install was complete, I reboot'ed the machine (after removing the install CD/DVD). The Grub menu that came up showed "Solaris Nevada build_xx", which I selected. Then, I got the login screen where I logged in as root using the password I had entered during install. I was prompted to Select Desktop, I picked GNOME.

The next pop-up was "Users and Groups", I selected "Add User" to create my home directory. Just as a sidenote, you can do this via commnad-line also using add-user command.

Another thing to note is that Solaris Express, Developer Edition automatically enables DHCP on your system. So if you want a static address and hostname of your choice, you'd have to take some extra steps afer install to disble DHCP and configure your static IP. I didn't particularly care for static IP so I just carried on

Next Up was the LiveUpgrade which has to be done as root.

First off, I had to edit the configuration file. This was done by

# vi /etc/default/lu

In this file is an environment variable LU_ALLOW_ANY_SOLARIS_VERSION_UPGRADE This needs to be set to YES.

Next I created a copy of the primary boot environment(what I installed in "/" c0d0s0 slice) onto the unnamed c0t0d0s5 and called this slice development

# lucreate -n development -m /:c0t0d0s5:ufs

Next I had to activate the boot environment and reboot

# luactivate development
# init 6

It is critical to note that you do not reboot, YOU MUST DO INIT 6 FOR LIVE UPGRADE TO WORK RIGHT.

The machine will reboot after the init 6 , and once it boots back up, the GRUB menu will show development as one of the options to select in addition to Solaris Nevada build_xx

To make sure that everything had worked correctly, I selected development, and at the login screen gave the username and password that I had picked with Add User So far, so good.

Building OpenSolaris from Source

Now I had to create a workspace where I could build OpenSolaris, and use the compiled kernel bits to install on the development slice. I reboot'ed the machine again, and this time selected the c0d0s0 partition (which was my primary boot environment) from the GRUB menu.

I chose to go the HTTP download route in order to get the source code, so all I had to do was create a directory where I wanted the source tree, and untar the source tar-ball (SUNWonbld-b62.i386.tar.bz2), and the ON Binary-only components (on-closed-bins-b62.i386.tar.bz2) in this directory. (In retrospect, I should have gone the Mercurial route since that would have spared me the final nightly. The Mercurial way is described below in the section Modifying OpenSolaris kernel.) Anyhow, this is what I did:

> mkdir osol
> cd osol
> bzcat SUNWonbld-b62.i386.tar.bz2 | tar xvf -
> bzcat on-closed-bins-b62.i386.tar.bz2 | tar xf -

Here my workspace is named osol and I will refer to it in the following text as my $GATE

Once the source tree was populated, I downloaded the onbld tar-file from OpenSolaris downloads, in /var/tmp, untar'ed it, and pkg-add'ed it. (As always, make sure that the binaries you are downloading correspond to your system architecture)

> cd /var/tmp
> bzip2 -dc SUNWonbld-b62.i386.tar.bz2 | tar xf -
> cd onbld/
> pkgadd -d onbld SUNWonbld

Next I added /opt/onbld/bin and /opt/SUNWspro/bin to my $PATH (you can do that via your .login file or whatever you use to define your environment).

The next step is to run a nightly. To do this I had to modify the environment file present in $GATE/usr/src/tools/env . I copied over this file (opensolaris.sh) to the top-level directory in my $GATE.

> pwd
/export/home/garima/osol
> cp usr/src/tools/env/opensolaris.sh .
> chmod 755 opensolaris.sh

I then modified this file to change four variables from their default value.

GATE=osol;
CODEMGR_WS=$HOME/$GATE;
STAFFER=garima;
VERSION="$GATE:`date '+%Y-%m-%d'`";

Other than these changes, I did not modify the file at all.

Now I cranked the nightly from the same place

> nightly ./opensolaris.sh &
> tail -f log/nightly.log

The second command was to make sure that the nightly was progressing correctly. (In case you are curious, you can see the mail_msg that gets sent out at the completion of the nightly here).

Once the nightly was all done, I had to use the archives created by it, and install them on the development slice via the BFU utility. Before I did that, however, I had to set three environment variables - FASTFS, BFULD, and GZIPBIN as mentioned in the OpenSolaris Developer's Reference Guide. Basically I added these three lines to my .cshrc file to avoid having to this again and again in the future :

setenv FASTFS /opt/onbld/bin/`uname -p`/fastfs
setenv BFULD /opt/onbld/bin/`uname -p`/bfuld
setenv GZIPBIN /usr/bin/gzip

Now came the tricky part of doing the actual BFU. Make sure to completely read the section on Using BFU before you go through with the BFU.
The archives for doing the BFU are located in ~/osol/archives/i386/nightly, so this is what I did (as root) (also note the difference in the df -hkl output:

garima-AT-solaris-devx:~:~>df -hkl
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s0        7.0G   5.0G   2.0G    72%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.9G   788K   1.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/usr/lib/libc/libc_hwcap2.so.1
                       7.0G   5.0G   2.0G    72%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                   1.9G    44K   1.9G     1%    /tmp
swap                   1.9G    36K   1.9G     1%    /var/run
/dev/dsk/c0d0s7         20G    18G   1.6G    92%    /export/home

Now I changed id to root and did

root@solaris-devx>mount /dev/dsk/c0d0s5 /mnt
root@solaris-devx>df -hkl
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s0        7.0G   5.0G   2.0G    72%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.8G   788K   1.8G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/usr/lib/libc/libc_hwcap2.so.1
                       7.0G   5.0G   2.0G    72%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                   1.8G    44K   1.8G     1%    /tmp
swap                   1.8G    36K   1.8G     1%    /var/run
/dev/dsk/c0d0s7         20G    18G   1.6G    92%    /export/home
/dev/dsk/c0d0s5        7.0G   5.1G   1.9G    73%    /mnt
root@solaris-devx>which bfu
/opt/onbld/bin/bfu
root@solaris-devx>/opt/onbld/bin/bfu /mnt ~/osol/archives/i386/nightly

Once BFU completes, all conflicts have to be resolved, so I made sure to run acr. This command was

root@solaris-devx>/opt/onbld/bin/acr /mnt /export/home/garima/osol/archives/i386/nightly

On completion, acr creates an allresults file (the complete path will differ for each instance); I made sure to check that the return codes looked correct. (I'm appending my allresults file here for your reference). Then I did

root@solaris-devx>reboot

Modifying OpenSolaris kernel

Once I did the BFU and successfully booted the development partition which I'd BFU'd, I was ready to do the most fun part - change OpenSolaris source code. This time I decided to go the Mercurial route to create the Repository. But before I did this, I had to reboot the primary boot environment c0d0s0. In case you are wondering why I keep going backk and booting up that BE - the main idea of creating the two BEs is that I use the primary BE (c0d0s0) to make changes to and compile the source code. The secondary BEdevelopment is completely a scartch partition, so that even if I made a mistake and messed up the kernel bits that get installed on development, I can still go back and boot up c0d0s0, fix the problem in the workspace there, and BFU development with the new fixed bits. To create a repository, I did

hg clone ssh://anon@hg.opensolaris-DOT-org/hg/onnv/onnv-gate my_ws

This created a directory my_ws under which I could see the complete source tree. Before I went about changing anything, I copied over the opensolaris.sh file that I had earlier created in my osol directory, and changed the $GATE variable to my_ws. Then I ran bldenv as advised in the Developer's Reference Guide.

garima@solaris-devx:~/my_ws:~>bldenv ./opensolaris.sh
Build type   is  non-DEBUG
RELEASE      is  
VERSION      is  my_ws:2007-05-01
RELEASE_DATE is  

The top-level 'setup' target is available to build headers and tools.

Using /bin/tcsh as shell.
garima@solaris-devx:~/my_ws:~>

I should point out here that cloning the onnv-gate repository through Mercurial did not get me the on-closed-bins directory which I would ultimately need to do a full compilation of the source code. I had to go back to the HTTP downloads page and download that tar-ball separately in the top-level of my workspace, and untar it there.

garima@solaris-devx:~/my_ws:~>pwd
/export/home/garima/my_ws
garima@solaris-devx:~/my_ws:~>ls
deleted_files/                         opensolaris.sh*
garima@solaris-devx:~/my_ws:~>bzcat on-closed-bins-b62.i386.tar.bz2 | tar xf -
garima@solaris-devx:~/my_ws:~>ls
closed/                                on-closed-bins-20070423.i386.tar.bz2   usr/
deleted_files/                         opensolaris.sh*
garima@solaris-devx:~/my_ws:~>ls closed/
BINARYLICENSE.txt               THIRDPARTYLICENSE.ON-BINARIES
README.ON-BINARIES.i386         root_i386/

Since I have been involved with the NWAM project, I decided to see if I could change something there as a quick and dirty "hello, world" equivalent of changing OpenSolaris source code. The NWAM source code currently resides in the usr/src/cmd/cmd-inet/lib/nwamd directory. Since I just wanted to make a quick change, I decided to change the title one of the Zenity dialog boxes that lists all the available wireless networks. This is the series of steps that I took:

garima@solaris-devx:~/my_ws:~>cd usr/src/cmd/cmd-inet/lib/nwamd/
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>ls
Makefile          events.c          interface.c       main.c            structures.h      variables.h
defines.h         functions.h       llp.c             state_machine.c   util.c            wireless.c
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>hg status
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>xemacs wireless.c &
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>hg status
M usr/src/cmd/cmd-inet/lib/nwamd/wireless.c
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>hg diff
diff -r f47ca9f35c1d usr/src/cmd/cmd-inet/lib/nwamd/wireless.c
--- a/usr/src/cmd/cmd-inet/lib/nwamd/wireless.c Tue May 01 05:33:55 2007 -0700
+++ b/usr/src/cmd/cmd-inet/lib/nwamd/wireless.c Tue May 01 17:55:20 2007 -0700
@@ -1081,7 +1081,7 @@ connect_to_new_wlan(const struct wireles
            "%s --list --title=\"%s\""
            " --height=%d --width=500 --column=\"#\" --column=\"%s\""
            " --column=\"%s\" --column=\"%s\" --column=\"%s\""
-           " %s ", ZENITY, gettext("Choose WiFi network you wish to activate"),
+           " %s ", ZENITY, gettext("Choose WiFi network to activate"),
            zenity_height(j), "ESSID", "BSSID", gettext("Encryption"),
            gettext("Signal"), buf);
 
garima@solaris-devx:~/my_ws/usr/src/cmd/cmd-inet/lib/nwamd:~>hg commit

At this point, Mercurial asks for a comment, for which I entered "change Zenity title for Wireless networks listing". Another hg status after this showed no more files being edited. Now it was time to compile, so I did another nightly. (NWAM needs several other libraries pre-built before it can compile, so another nightly was required). After this, it was pretty straight-forward to use the archive from this nightly to BFU the development partition, and run my own changed bits of OpenSolaris!

If you had already done the complete build in the beginning, you can also just run dmake in the relevant subdirectory, and copy over the updated binary in its relevant place. E.g. if you made a change in some file of the directory $your_ws/usr/src/uts/common/inet/tcp, then the compiled 32 and 64bit binaries will be in $your_ws/usr/src/uts/intel/ip directory. You can simply copy over these new binaries to /kernel/drv/ip and /kernel/drv/amd64/ip of the development partition to use it. For you to be able to do this, the development partition needs to be mounted in the / slice. (You need to have selected development to boot from the GRUB menu and it should be the top most line in the df -hkl output).

garima@solaris-devx:~:~>df -hkl
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s5        7.0G   5.0G   2.0G    72%    /
:
:
:

Useful Links

Technorati Tag:OpenSolaris
Technorati Tag:Solaris
Technorati Tag:DeveloperExpress
Technorati Tag:NWAM

Posted at 01:08AM May 02, 2007 by Garima Tripathi in Solaris | Comments[1]