I always tell myself that I don't have much to tell or preach to the world. I rarely have much to blog. But blogging is cool and I have made a handful of half-hearted attempts at blogging including this one.

twitter.com/manoj_joseph
Photo Journal
Broomfield Bugle
Bangalore Bugle
Kernel Junkie
Random Thoughts

As you know, Sun is now a wholly-owned subsidiary of Oracle. And this blog will henceforth, only have entries that are work related - Lustre, Oracle... The above will have my personal stuff.

And in case you did not realise, the views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Consider this. Since I joined CFS and Lustre, my job stack has been unwinding.

Sun bought Cluster FileSystems (Lustre) and I returned back to Sun. Now Oracle is on the verge of buying Sun and I might be back with Oracle too! It is quite funny!

Update: Looks like I will boomerang back to Oracle!

2002 Jun ..... iNautix
2003 Mar ..... Oracle India
2004 Jul ..... Sun Microsystems India
2007 Feb ..... Cluster Filesystems
2007 Oct ..... Sun Microsystems India
2008 May ..... Sun Microsystems Inc. (USA)
2010 Jan ..... Sun Microsystems Inc., A Wholly Owned Subsidiary of Oracle Corporation.
2010 ??? ..... Oracle Inc. (USA)

Safe Harbor Statement: This blog contains one forward looking statement! :P.

Disclaimer: The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

Looks like Opensolaris is all set for its official release. The package repository seems to be all loaded and ready.

I had OpenSolaris DP2 on my laptop. And upgrading it to 2008.05 was as simple as running th upgrade script - a piece of cake!

Regardless of one's take on the Opensolaris naming controversy, this distro rocks!

I had earlier posted a demo of installation of lustre on a single node.

But most, if not all production installations of lustre are on multiple nodes - tens of Object Storage Servers (OSS) and thousands of clients.

Here is a demo of a multi node setup - though on a much lesser scale - one Meta Data Server (MDS), two OSS and a single client. And I show this on VMWare. I take three virtual machines on which lustre 1.6 packages are installed. (See earlier post on how to do this.
Lustre Installation

Here is what the demo does:

A few days ago, I was putting together a demo of lustre for some teams in Sun interested in Lustre. Then yesterday I got another request for a demo, this time at a college in Tamil Nadu. This got me interested in producing a flash demo that can be reused.

So, with the help of vnc2swf and tightvnc, here is what I put together. On a virtual machine running CentOS 4.4 on my laptop, I show the setting up of an MDS, a OSS and a client on a single node. The filesystms are on top of loop devices.

Lustre Installation

Lustre Configuration
[Note: Right click and select 'Play' to play the demo.]

Here is what the demo does:

$ wget http://downloads.lustre.org/public/lustre/v1.6/latest//kernel-lustre-smp-..rpm
$ wget http://downloads.lustre.org/public/lustre/v1.6/latest//lustre-..rpm
$ wget http://downloads.lustre.org/public/lustre/v1.6/latest//lustre-ldiskfs-..rpm
$ wget http://downloads.lustre.org/public/lustre/v1.6/latest//lustre-modules-..rpm

$ mount -t lustre ${MDS}@tcp0:/lustre /mnt/lustre

As you can see, it is quite simple - a big change from the lustre 1.4.x days!
Download lustre packages from here

In my earlier post, I described how I setup Indiana on my laptop.

I had two must haves for my work - virtual machines, and support for my ethernet card. My Indiana Preview 2 installation met both; well sort of...

I installed VirtualBox and then went on to install OpenSolaris build 84 and CentOS 4.4 on virtual machines. While VirtualBox is a bit different from VMWare, the installation was a piece of cake.

Setting up NAT took care of Internet access for the VMs. The NAT interfaces use DHCP. Piece of cake again.

Then my troubles started.

I need to SSH in to my VMs. On VMWare it is as simple as adding host-only network cards to the virtual machines. The host is automatically on this network. The VMs can also talk to each other over this network.

On VirtualBox, things are a bit different. The host can talk to the guests (AFAIK) only by setting up port forwarding. This is explained in the VirtualBox User Manual.

The following commands did it for the 'osol' virtual machine. I could ssh and sftp in!

$ VBoxManage setextradata "osol" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/Protocol" TCP
$ VBoxManage setextradata "osol" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/GuestPort" 22
$ VBoxManage setextradata "osol" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/HostPort" 2222

$ ssh -p 2222 root@localhost
$ sftp -o port=2222 root@localhost

See also Renaud's post.

If you have multiple VMs, use different host ports (2222, 2223 etc).

One problem solved. However, the VMs still can't talk to each other. To do that I needed InternalNetwork. This is also explained in the UserManaual.

Setting it up is easy. Using CLI, this is what I did.

$ VBoxManage modifyvm osol -nic2 intnet
$ VBoxManage modifyvm osol -intnet2 intnet
$ VBoxManage modifyvm rhel -nic2 intnet
$ VBoxManage modifyvm rhel -intnet2 intnet

That's it. In theory this should have just worked. However it did not.

I spent a couple of days fiddling around with this. I could ping the other node at times. But once I try to ssh in, ssh fails and ping fails thereafter!

To isolate the problem, I made a copy of the linux VM and InternalNetwork worked fine between the linux VMs. That suggested, I was not goofing up with the VirtualBox setup.

Next, I tried with two OpenSolaris VMs. The same problem showed up again - I could ping but ssh failed. After this failure, ping also failed.

This suggests a bug somewhere. But hey, VirtualBox is still beta on OpenSolairs. I am hopeful that this issue will get sorted out soon.

But till then, I am going back to Ubuntu. Bye bye OpenSolaris! :'(

OpenSolaris and VirtualBox

I used to use OpenSolaris on my workstation while I was with the Sun Cluster group. When I moved to Cluster Filesystems, I got a new Dell laptop and I moved to Ubuntu out of necessity - I needed to use virtual machines (VMWare Server) for my work.

I have been looking for a way of moving to OpenSolaris and have been eying Xen as a possible replacement of VMWare. But the traffic on xen-discuss@opensolaris.org suggested enough difficulties and I kept away.

VirtualBox on OpenSolaris gave me a new reason to try things out. So, I decided to throw away Windows Vista (given to me, thanks to Dell) and decided to take OpenSolaris for a spin.

OpenSolaris Developer Preview 2 vs Solaris Express

I have used OpenSolaris and Solaris on Virtual machines and on workstations. So I am not a newbie. I had two options - Solaris Express or Indiana. Given that it is easier to upgrade Indiana (thanks to IPS a.k.a pkg(5)) and having tested it on a virtual machine, I decided on Indiana.

Installation of Indiana

So, I burnt the iso on to a CD and booted from it. Partition editing is disabled in preview 2. And certainly did not want to use the entire hard disk. So, I booted back into Ubuntu and changed the partition type of the Vista partition to Solaris and booted back to Indiana LiveCD.

Now the Solaris partition was recognized. And the rest of the installation was a piece of cake.

Grub entries for Ubuntu (and ext3 read only support)

Installation did not make entries for Ubuntu. And the default page on Firefox told me to get the entries from the Linux partition. Well, time for a resuce CD to access the Linux partitions? No! Time to install read-only support for ext3/ntfs.

# wget http://www.genunix.org/distributions/belenix_site/binfiles/FSWfsmisc.tar.gz
# gzip -cd FSWfsmisc.tar.gz | tar -xvf -
# pkgadd -d . FSWfsmisc
# mount -F ext2fs /dev/dsk/c7d0p1 /mnt/lin1
# cat /mnt/lin1/boot/grub/menu.lst

Copied the grub entries that I wanted to /rpool/boot/grub/menu.lst and I was done.

Thanks Moinak and Martin Rosenau!

Network drivers

Indiana had no trouble supplying drivers for my wireless network. But the Ethernet driver was not supported. I do have a wireless router. And nwam configured it just fine. But I did want the LAN to be functional as well. And thanks to Masayuki Murayama and Moinak Ghosh , it was as simple as

# wget http://blogs.sun.com/moinakg/resource/nicdrv.tar.bz2
# bzip2 -cd nicdrv.tar.bz2 | tar -xvf -
# cd nicdrv
# for i in `ls -d SFE*`; do yes | pkgadd -d . $i; done
# ifconfig -a plumb

VirtualBox

Installing Virtual Box was a piece of cake

# wget http://virtualbox.org/download/testcase/VirtualBox-opensolaris-x86-1.5.51-r28414-beta1.tar.gz
# gzip -cd VirtualBox-opensolaris-x86-1.5.51-r28414-beta1.tar.gz | tar -xf -
# pkgadd -d VirtualBox-1.5.51-SunOS-x86-r28414
# VirtualBox

I had some trouble launching Virtual Box though.

No 64-bit VirtualBox

I have a 64 bit system and the VirtualBox vboxdrv driver is 32 bit only. So I had to boot into 32 bit. So, I added a new grub entry:

  title OpenSolaris Developer Preview 32 bit
  kernel$ /platform/i86pc/kernel/unix -B $ZFS-BOOTFS
  module$ /platform/i86pc/boot_archive

Now the driver vboxdrv loaded just fine. 'add_drv vboxdrv' confirmed this.

But still VirtualBox failed to start a virtual machine with the message:

   VBox status code: -1909 (VERR_VM_DRIVER_NOT_ACCESSIBLE).

Need to be root user! But "su -" refused to connect to X. Setting DISPLAY, "xhost +", ForwardX11 etc did not fix the issue. Google came to the rescue again. Doing the "su" instead of "su -" did the trick.

Sound driver

My laptop has a 82801G (ICH7 family) Intel audio controller. And OpenSolaris does not support it yet. No problem, OSS has drivers for it!

# wget http://www.4front-tech.com/release/oss-solaris-v4.0-1014-i386.pkg
# pkgadd -d oss-solaris-v4.0-1014-i386.pkg

However, though osstest produces sound, the audio devices do not show up in the sound preferences. This is a problem to be investigated for another day.

Blastwave

Blastwave has a huge repository of packages for Solaris. So, I pkg-get a piece of it.

# pkgadd -d http://www.blastwave.org/pkg_get.pkg
# /opt/csw/bin/pkg-get -i xchat
# /opt/csw/bin/pkg-get -i xemacs

This should become simpler once the pkg(5) version of blastwave is unleashed.

Now, I am installing CentOS 5 on a virtual box!

After about 3 years with Sun Microsystems, I moved to work for Cluster Filesystems Inc. to port Lustre to Solaris. Half-way through the project, Sun decided to buy CFS. And that brings me back to Sun.

At Sun, they call this a boomerang.

I am Manoj Paul Joseph. I am with the Sun Cluster development team. I work from Bangalore, India. I work with the Global Data and Devices (GDD) group. I work on DCS, PxFS and Sun Cluster SMF services... I work on all of that but my heart is with PxFS.

Before joining Sun, I had a short stint with Oracle and an even shorter one with iNautix Technologies.

That's me on the left.

That's all for now!

-Manoj