Mike Ramchand's blog, covering mainly JET (Jumpstart Enterprise Toolkit), but with diversions into other stuff, like sailing.. JET Stream

I was giving my standard pitch covering Sun Virtualisation Technologies which essentially entails me talking at length about the following slide. I kind of make the point that it is a means not an end, and that really what you're trying to do is run more applications on the same piece of hardware, and the trick is to try and do it most efficiently.

I've done this talk countless times with a pretty wide variety of customers, and I tend to start talking from the right, and move to the left, explaining how the "layer of additional non app resource utilisation" grows from right to left. (not true for Dynamic System Domains on the far left though.)

It finally occurred to me yesterday that the Unix and Windows world have an utterly different view of Virtualisation, and the need for Virtual Machines.

In the Unix world, application co-existence is second nature to most sys-admins. Virtualisation is all about application isolation. i.e. You START with all the apps on a single O/S instance, and you apply more and more constraints: resource management --> containers/zones --> virtual machines --> dynamic system domains.

In the Windows world, application co-existence is so rare that it isn't even considered. Virtualisation is all about application consolidation. (and the primary way to do it is to create lots of virtual machines, one per app).

The problem is simply viewed from completely different angles.

It's not just an application co-existence issue either. There's a scalability problem. A lot of "other" O/S's can't scale to fit the larger more powerful Intel/AMD multi-core servers available today, and the ONLY way to make use of them is to carve them up into smaller machines that Windows or Linux can digest. Solaris, on the other hand can scale up to hundreds of CPUs, and hundreds of Gigabytes of memory.

From the slide above, Sun/Solaris provides a wide choice of technologies to allow multiple applications to be consolidated onto a single server. Don't let the limitations of "other" O/S's blinker you into thinking that a Virtual Machine is the only answer. 

(I'm not saying that Virtual Machines is the WRONG answer btw, it's ONE of the answers, but it shouldn't be the only one that is considered.)

tags: virtualization containers ldom zones vmware xen

 So, I've been working on JET 4.7 for a while now. A couple of weeks ago I released it internally to Sun, and no-one's complained about any showstopping bugs yet, so I've released a fresh bundle to the sun.com downloads site. The best way to go straight to it is to follow the link at the top of the wiki site.

JET 4.7 Changes:

The MAJOR JET change is that I've kind of merged JetNAS functionality into the main SUNWjet module. This isn't an exact replication of the JetNAS code, and it does things a little differently, but in short you can choose to place the NFS functionality somewhere different from your JET server.

Other JET changes are covered by bugids: 6835016, 685027, 685023, 685019, 685030.

In summary:

6835016:  JET needs to support alternative servers for NFS media. The JET User Guide has already been updated to cover this functionality. Additionally, I've written a short article about it on the JET wiki site.

685027:  JET jetjump.xml script does not stop graphical login from starting. Tidied it up. graphical login no longer starts until J/S is complete.

685023: JET should not create /dhcp directory. We used to do this to shorten the dhcp macro. This is no longer necessary, so we've removed the complication of having it.

685019: JET ksh scripts do not work when using OpenSolaris. There were a couple of constructs in our ksh code that ksh93 choked on. We've updated them so that they are compatible with ksh and ksh93. This allows all you OpenSolaris people to install JET on your OpenSolaris based Jumpstart server. (Note, AI is required to "jumpstart" OpenSolaris.)

685030: JET error message when suitable server is not found could be more meaningful. Well, this is just an attempt to reduce the e-mail traffic. Probably the first error, people new to JET see. We've updated the error message to tell you what to do to fix it.

JET 4.6 - 4.7 Change Summary:

For all you "external" people, the last externally available version was 4.6. Here are the changes from 4.6 - 4.7:

4.6.1: Ability to compress the rpool, and specify /var for zfs boot builds.

4.6.2: 6798648: JET needs to workaround Bug: 6528699: This is to stop Jumpstart from labelling ALL your disks and blowing away all your data. Also added a tweak to the ssh scripts to generate the keys properly.

4.6.3: Added an additional bootadm update-archive to the end of the finish.common script. We were getting some situations where the first reboot had a stale boot-archive, possibly due to a race condition in SPARC S10 Update 6 and above builds.

JET bundled Modules:

We've updated some/most of the bundled products. ALL of them (if required) have been updated to make them ksh93 compatible. Some of them have had more substantial changes:

JetLDOM

This module has had the most substantial rewrite. We've gone against ALL JET precedent and completely changed the structure of the ldom.conf file, so it's not backward compatible (for once). We needed to do that so that we could provide a much easier mechanism to make use of the ldm 1.3 functionality. Sorry.... but you'll like the new one better.

JetFLASH

I've updated JetFLASH so that is actually checks the flar (if it can), to see that it is the right architecture, and the right boot filesystem for the build. (Yes, zfs based flash is coming, but it is a different format to the ufs based one). Bruce added some code to clean up service tags if required.

JetZFS

Quite a few changes in this module. You can now update your rpool properties, I've provided more verbose progress messages (so you can see exactly what it is doing), added code to deal with converting EFI disks back to SMI (it does it for you if required), and finally added a strange hack so that you can optionally use ::: instead of = in your variables to get around a xVM OC gui restriction.

JetZONES

Fixed a bug where make_template was in the wrong place, and added some code to boot the zone before trying to add stuff to the rootpath/var dir. (because it might not be there unless the zone is booted). (Thanks to Mike Hendon for pointing it out). (plus a bunch of new functionality like support for branded zones)

Summary

So just a reminder, the following modules have NEW template variables. Old templates (other than the ldom based ones) will continue to work, but if you want to make use of the new functionality (with the new comments that describe them), you'll need to refresh your templates. (Have a scan through sample.template in /opt/SUNWjet/Templates to see what's changed.)

tags: Solaris JET Jumpstart

I've been working on JET 4.7 (current released JET version is 4.6.3).

The main change is that I've now provided support to allow the Solaris Media and the /opt/SUNWjet directories to be hosted by a server (or servers) other than the JET server. This effectively removes the requirement for the JET server to provide NFS services.

Along the way I was able to tidy up and consolidate a lot of code, particularly the client_allocation_* methods. I've added a few functions, and used the variables more consistently throughout. For those of you doing DHCP/PXE based builds, I've removed the /dhcp hack that was used to overcome the 255 char limitiation of the DHCP macros.

Full details on the wiki site

I'm hoping to release it by the end of April.

tags: Solaris JET Jumpstart

A few months ago I wrote  Solaris Cluster on a laptop using VirtualBox, iSCSI and a quorum server, which detailed some of the hoops that I needed to jump through to get it to work. The good news is that the new features in VirtualBox 2.0 (and now 2.1) have gotten rid of 2 of those hoops:

1.  I no longer need to play with vnics and that crazy script as Host Interface networking simply works out of the box
2. I don't need to install the 32bit internal only SC package, as VirtualBox now supports 64bit guests.

I'm in the process of rebuilding my laptop cluster making use of the above new features.

 Hurrah!!!

tags: JET Jumpstart SunCluster Solaris VirtualBox xVM

As a follow-on to my previous post, JET 4.6 is now available externally. Check the JET wiki which has a link to the download.

This is mainly a bug fix release, and has been given a major number increase because it's the JET version that will be in the next release of xVM OpsCenter. (and we like to release the OpsCenter releases at major version numbers).

In terms of changes, I've fixed a couple of bugs with turning ntp on, and multiple disks in zfs pools for ZFS disks. This version also has tweaks in place to workaround some DHCP issues when installing S10 U6 and Solaris Nevada.

tags: Solaris JET Jumpstart

As a follow-on to my previous post, JET 4.4.7 is now available externally. Check the JET wiki which has a link to the download.

Also have a look at the JET User Guide which has now been posted on the wiki. I expect this to grow over time, but its a conversion from the original user guide with updates for correctness.

tags: Solaris JET Jumpstart

I've just released JET 4.4.7 internally. Bruce is in the process of putting together a new bundle including 4.4.7 for the external download, so it'll arrive in couple of weeks. If you need it NOW just ask nicely.

Changes

The latest  version currently available externally is 4.4, and we've added a bunch of features since then. Here's a brief overview of what's new/changed/fixed:

4.4.1

1. 6660631: Disabling UFS logging no longer works.
2. 6673213: Selecting dhcp should automatically select grub if required.
3. 6675467: JET needs to workaround Nevada installation bug 6633017.
4. 6315828: JET "shutup_sendmail" variable does not work on Solaris 10.
5. 6526561: Typo in make_client, missing r on JS_error line 130.

4.4.2

1. 6656549: Add ability to do fdisk commands in the profile.

4.4.3

1. 6694299: SUNWjet fails to install on system with super long group.
2. 6690411: base_config_profile_fdisk needs to be ingored for sparc.
3. 6698860: "rarp" service should not be enable by JET (on dhcp builds)

4.4.4

1. 6706581: JET needs to support ZFS root installs. This is a pretty big one, with the inclusion of 5 new base_config variables. Might require a separate blog article to explain, but as usual, the comments in the template should get you on the way.

4.4.5

1. 6735314: JET incorrectly sets NFSMAPID_DOMAIN to "dynamic".
2. 6735318: JET should support noatime mount options.
3. 6740625: JET client_environment should never output a warning message.

4.4.6

Never released.

4.4.7

1. 6675467: (again, as we need to apply workaround for Sol10 U6 as well)
2. 6761787: JET custom_files should allow the overwritten file to be backed up. (inspiration from Christopher Hubbell)

Summary of Changes

A lot of the changes are bug fixes, but a number of them begin to make use of some of the new Solaris features, or leverage new ways of doing things. The most important of these is the ability for JET to install ZFS boot Solaris. This applies to Nevada build 90 and above, as well as the imminently to be released Solaris 10 Update 6. Other things that are kinda cool is the ability for JET to fdisk your disk for you when doing an install, as well as set noatime mountpoint options (which more and more people seem tobe doing by default now.

tags: Solaris JET Jumpstart ZFS

I was in the process of playing with some stuff using VirtualBox, and it occurred to me that this was a pretty cool screenshot, so I did, and here it is.

tags: Solaris belenix ubuntu opensuse virtualbox

Over the past few weeks/months, I've been involved (sometimes peripherally) with setting up our demo kit in the UK Customer Briefing Centre. One of my design goals was to ensure seamless access to our demos from anywhere in the world or more realistically anywhere in the UK, and more specifically, at any customer site. 

Within the CBC network (which is not connected to our internal Sun network) we set up a Sunray Server also running SGD. We initially had some hiccups getting the firewall traversal to work, and had to resort to using VNC which was adequate, but painful as we were tunnelling through ssh, so there were a bunch of steps required to ssh in, start vncserver, then re-ssh in to tunnel the right port, then start vncviewer and after all of that the screen refresh rate wasn't stellar.

However, once we got SGD working it was absolutely awesome.

If you want to have a play with how it works, simply go to Sun's SDG demo site . Unfortunately MyDesktop doesn't work (for obvious reasons) for anonymous users, but it will let you play with the webtop functionality.

For our purposes, we used the MyDesktop functionality. Running a desktop session inside the CBC from anywhere in the world was as easy as pointing a browser at the right place, logging in and clicking on the Desktop icon. Within seconds, as if by magic, a full screen desktop takes over your current screen (you can switch back to local using Alt-TAB). I'd show you a picture of it, but it just looks like a normal desktop and feels remarkably snappy even if it is remote.

The reason we were doing all of this is that we  (The UK Systems Practice) have been running some "Discovery Days" around LDOMS and ZFS which have live demos of the technology. The cool part is that we get to showcase the SGD technology at the same time!

tags: Solaris SGD ZFS LDoms

So I got the boat down, and managed to go sailing in and around the Helford river in Cornwall. It was absolutely awesome. On the day the picture was taken, the wind was pretty light and so I decided to brave going out single handed. Even got the asymmetric out on the downwind leg.

Joerg Moellenkamp has written a pretty cool 16 part tutorial on Jumpstart using JET here. It's already linked at the JET wiki page.

tags: Solaris Jumpstart JET  

I've not blogged about sailing in some time. Its not that I've not been Sailing. Its more due to me not blogging enough. I'm going to try and do a weekly recreational blog. (Watch me fail).

Since my last post, I've been helping out at my local Sailing Club (Burghfield Sailing Club)
with their Oppie Club. My oldest daughter got quite into it, and I got her an old wooden Oppie called Major Tom. Here's me sailing it.

As you can see, it's for kids, not adults. (And yes, I did eventually pick up the kid swimming behind the boat).

I've gotten a lot better sailing my Laser 2000, and have been trying to take part in the Wednesday Evening Pursuit races that take place during the Summer at the club. (On Wednesday evenings, funnily enough). I actually came first in one of them this year, and pretty happy with my progress.

Last week we were doing pretty well in quite windy conditions (I think we were 4th or 5th with 15 mins to go), when my main halyard broke, resulting in me having to DNF. The bigger problem was that we're taking the boat down to Cornwall on holiday next week, so I had to get it fixed pretty quickly. This weekend, I spent Saturday morning down at the Sailing club taking the mast down, and feeding a piece of wire up the inside of the mast, and rerunning the old main halyard. (This is harder than it sounds) The Chandlery at the Sailing Club was closed (the owners had a day at the races at Ascot or something) so I couldn't get a new halyard. Felt I might as well run the old rope, and once it's in, its easy enough to replace without having to take the mast down. On Sunday, today, they were open, so I was able to replace the old rope with a nice shiny new halyard.

It's also the first time I'll be towing the boat anywhere, so I also needed to get a couple of towing essentials for the trip. (Fortunately the family car has a towbar). I've decided to play it safe and get a new Jib Halyard as well, as it is also about 8 years old. A trip to the LaserDirect and £100 pounds later, I've got a bunch of stuff coming, hopefully by Wednesday this week, which will give me enough time to fit before setting off on the weekend.

It's also the first time that we'll be taking the L2K onto the open water. The place we've rented is in a sheltered cove, so we don't HAVE to go to sea, but it might be fun to go out for a laugh.

If this is my last ever blog post, you can guess what happened.

I've made some minor modifications to share_isos. I realised that add_install_client creates entries in /etc/dfs/dfstab for the "boot" filesystem if not already mounted. Unfortunately, it only checks the dfstab, rather than checking the actual shares. The result is that I need to populate the /etc/dfs/dfstab file.

The complication is that now that it's in dfs, the stubs get shared when the server boots, so I need to unshare them before doing the lofi mounts.

 I've updated the share_isos script to do that. I've decided I'm simply going to make it an add_on JET module that can be installed if that feature is required/desired. (Look out for a JetISO module any day soon).

Finally, while testing all of this I decided to so something pretty obscure. I'm in the process of building a JET.vdi and I'm trying to make the .vdi as small as possible. (A virtual disk image of the smallest Solaris I can use, with JET and its modules installed so that it is nice and easy to distribute (so far its about 800k). I obviously don't want to prepopulate with any Solaris media as that will simply make my vdi too large. The cool thing about iso images is that I can NFS mount a directory containing them, and then I can lofiadm and mount them locally.

So I ended up with a JET server running in a virtualbox, NFS mounting a Sol 10 0508 iso from my home server, lofimounted and shared with share_isos, and I used that to jumpstart another Solaris virtualbox machine on the same PC (which was running XP). The really good news was that it worked. The not so good news is that it took over a day to install. I think there are a few too many levels of indirection and virtualisation between the iso and the virtual disk.

I should really do a bit more testing and figure out the bottleneck. Ideally I'd prefer to keep the isos on the host and use shared folders to allow the virtual JET server to access them, but that won't be possible if the performance is as bad as my original test.

tags: Solaris Jumpstart JET virtualbox

Things have been slightly problematic on the JET server running on Nevada front mainly due to 2 issues.

1. There is/was a bug in cpio (See BugID 6686818) which meant that setup_install_server didn't copy the media correctly.
2. setup_install_server on earlier Solaris releases didn't understand zfs, therefore it was difficult to copy onto a zfs filesystem.

Why not just use iso images?

Why not indeed? I'm in the habit fo simply downloading and keeping the .iso images for all releases, and I used to then "expand" it by lofimounting and then running setup_install_server. I realised I really didn't need to do that, I could just use the lofi mounted image, and get away from having to ever run setup_install_server, and not need to use up double the space per O/S image I wanted to keep.

 So.. what was missing?

I needed a way to lofiadm, mount and nfs share the Solaris media. Also because x86 builds also lofi mount the boot directory and mount it into the /tftpboot directory, I'd also need to lofimount a lofimounted filesystem as well.

Firstly I did everything manually to ensure that everything works fine, then I wrote a script, which reads a config file (which lives in /opt/SUNWjet/etc).

Inline here:

#!/usr/bin/ksh -p
#
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
################################################################################

# sanitize PATH, and ensure required components are in front
PATH=/sbin:/usr/sbin:/bin:/usr/bin:${PATH}
export PATH

BINDIR="`cd ${0%/*}; pwd`"
ETCDIR="`cd ${BINDIR}/../etc; pwd`"

PATH=${BINDIR}:${PATH}
export PATH

MEDIALOC=${ETCDIR}/solaris_iso_locations

if [ "$#" != "0" ]; then
	echo "Usage: $0"
	echo " "
	exit 1
fi

#
# Check to see if the media locations file exists.
#

for iso in `cat ${MEDIALOC} | grep -v "^#" | awk '{print $1}'`
do
	lofiname="`lofiadm | awk '$2 == iso {print $1;}' iso=$iso`"
	if [ -z  "${lofiname}" ]; then
		lofiname="`lofiadm -a $iso`"
		echo "Added $iso as lofi filesystem: $lofiname"
	fi
	mp="`cat ${MEDIALOC} | awk '$1 == iso {print $2;}' iso=$iso`"
	mounted="`df -k | awk '$1 == lofiname && $6 == mp {print $6;}' lofiname=
$lofiname mp=$mp`"
	if [ -z "${mounted}" ]; then
		mount -F hsfs $lofiname $mp
		mounted=$mp
		echo "Mounted $lofiname at $mp"
	fi
 	shared="`share | awk '$2 == mp {print $2;}' mp=$mp`"
	if [ -z "${shared}" ]; then
		share -o ro,anon=0 $mounted
		echo "Shared $mounted"
	fi
	# Deal with lofi mount in x86
	mountboot="`cat /etc/vfstab | awk '$1 == mpboot {print $3;}' mpboot=\"$m
p/boot\"`"
	if [ -n "${mountboot}" ]; then
	bootmounted="`df -k | awk '$1 == mpboot {print $6;}' mpboot=\"$mp/boot\"
`"
		if [ -z "${bootmounted}" ]; then
			mount $mountboot
			echo "Remounted $mountboot"
		fi
	fi
done

It parses the solaris_iso_locations file which looks like this (I've attached my filled out one)

# This file is used to specify the mountpoints for iso images. You need
# to create the mountpoint, update this file, and then run /opt/SUNWjet/bin/
# share_isos. This will lofimount all isos listed here. You will then
# need to run add_solaris_location for each image as normal.
# You should run the share_isos command everytime the JET server is
# rebooted.
/backup/Solimages/s10sparc_0508.iso    /export/install/iso/Solaris_10_0508s
/backup/Solimages/s10x86_0508.iso    /export/install/iso/Solaris_10_0508x
/backup/Solimages/solarisdvdb90x.iso    /export/install/iso/Solaris_11_b90x
/backup/Solimages/solarisdvdb90s.iso    /export/install/iso/Solaris_11_b90s
 

What it does.

In short, it reads the config file and mounts (if not already mounted) the iso images on the path specified. It will then NFS share the mountpoint, and parse the /etc/vfstab and do the /tftpboot lofi mounts if necessary. (NOTE: add_install_client creates those entries in /etc/vfstab the first time it is run for that O/S. You MUST edit the /etc/vfstab and change the mount_at_boot flag to "no", otherwise your booting your JET server will fail.)

Running share_isos produces the following:

# /opt/SUNWjet/bin/share_isos
Mounted /dev/lofi/2 at /export/install/iso/Solaris_10_0508s
Shared /export/install/iso/Solaris_10_0508s
Mounted /dev/lofi/3 at /export/install/iso/Solaris_10_0508x
Shared /export/install/iso/Solaris_10_0508x
Remounted /tftpboot/I86PC.Solaris_10-1
Mounted /dev/lofi/4 at /export/install/iso/Solaris_11_b90x
Shared /export/install/iso/Solaris_11_b90x
Remounted /tftpboot/I86PC.Solaris_11-6
Mounted /dev/lofi/5 at /export/install/iso/Solaris_11_b90s
Shared /export/install/iso/Solaris_11_b90s
# /opt/SUNWjet/bin/share_isos
# 

 Note: if you run share_isos again, it does nothing as everything is already mounted and shared, so its easy to re-run it whenever you want just to confirm it's all good. (I considered having JET call it automatically, but I'm not completely sure how I want to do that yet. Well thought out suggestions welcomed.)

The bottom line is that I can now easily use my laptop running Nevada and a ZFS filesystem and use it as a JET server for any version of Solaris without ever needing to run "setup_install_server". And the bonus is I've now got about 12GB of space back!!

tags: Solaris Jumpstart JET

I was intrigued by Gordon Johnston's blog article about creating a 2 node Cluster Express cluster using VirtualBox, and I decided I had to try it.

Here's the final result:

(You can click on the image for a fullsized picture.)

It shows a VNC connection to my laptop which has 4 windows up: The 2 VirtualBox instances showing cxnode1 and cxnode2, a terminal window on the laptop itself, showing the quorum configuration and the iSCSI share, and in the background, the VirtualBox console showing the network configuration of cxnode1. Finally, in the main screen, I'm running Sun Cluster Manager and connected to the cluster, showing the topology and the Resource Group configuration of apache.

Rather than simply duplicating what Gordon did, I decided to go with Solaris Cluster 3.2 U1 as my cluster software. This meant that I had to use an internally available package of 32 bit kernel drivers, as SC 3.2 is shipped as a 64bit only product. (Cluster Express has both 32 and 64 bit variants.) The reason for going with Solaris Cluster is that there are instances where I want to duplicate a scenario that a customer is seeing, and in most cases, that's on Solaris Cluster.

I'm hesitant to duplicate Gordon's article, as I pretty much followed his guide, so I urge you to read his article as it is quite detailed, I'll only highlight where I changed things.

1.  I only really wanted a working cluster, went for the simpler set up of HA Apache
2. Cluster node O/Ss would be Solaris 10 05/08
3. Host O/S is Solaris Nevada Build 93 (as that was what was already running on my laptop)
4. I wanted to Jumpstart the nodes using JET.

Preparing your physical host

I already had VirtualBox 1.6.2 installed, so there wasn't much to do there. The main trick was configuring my laptop to allow Host Interface networking. This bit is Solaris specific. You basically need to do 2 things.

Firstly, you need to set up a bunch of virtual nics, and assign MAC addresses to them. As this needs to be redone on every reboot, I followed Dave Tong's advice  and set up an rc3.d script. (and no, I didn't bother to use SMF either, you can bite me too if you want.)

#!/sbin/sh
state="$1"
case "${state}" in
'start')
    echo 'Plumbing VNICs'
    for MAC in 10 20 30 40
    do
        VNIC=`/usr/lib/vna nge0 c0:ff:ee:0:0:$MAC`
        ifconfig $VNIC plumb
    done
    ;;
'stop')
    ;;
*)
   echo "Usage: $0 { start | stop }"
    exit 1
    ;;
esac

(I did use Gordon's cute fake MAC address c0:ff:ee 'cos its easy to remember)

Secondly, you REALLY REALLY need to make sure that you give the VBoxSVC privileges to configure the interface, otherwis your virtual machines will fail when starting. Whenever you start VirtualBox, you need to get the PID of the VBoxSVC, and run: ppriv -s AEI+net_rawaccess <pid>

Creating your Virtual hosts

Basically I followed Gordon's instructions for this, with TWO exceptions. I needed to select Network in the Boot Order menu in Settings-->General-->Advanced as PXE is not enabled by default. I also found that I couldn't do a PXE boot if I used the Intel driver, so I used the PC-net FAST III driver throughout which worked fine.

Installing Solaris on the Virtual hosts

I've already got JET installed on my laptop, so preparing to install Solaris was an amazingly simple process:

1. /opt/SUNWjet/bin/make_template cxnode1 base_config
2. Edit the template
3. /opt/SUNWjet/bin/make_client cxnode1
4. Start the cxnode1 Virtual Machine, Press F12, Select Lan Boot, and watch it install.
5. /opt/SUNWjet/bin/make_template -T cxnode1 cxnode2
6. Edit the cxnode2 template and change 3 variables.
7. /opt/SUNWjet/bin/make_client cxnode2
8. Start the cxnode2 Virtual Machine. Press F12, Select Lan Boot and watch it install.
   

Of course, the devil is in the detail, I'd already configured the Solaris media, and these are the changes I made to a default template for the build to work:

base_config_ClientArch=i86pc
*base_config_ClientEther=c0:ff:ee:0:0:10
base_config_ClientOS=10_0508x
base_config_client_allocation="dhcp"
*base_config_sysidcfg_ip_address=192.168.1.201
base_config_sysidcfg_netmask=255.255.255.0
base_config_profile_fdisk="alldisk"
base_config_profile_cluster=SUNWCXall
base_config_profile_root=free 
base_config_profile_swap=2048
base_config_profile_s6_mtpt="/globaldevices"
base_config_profile_s6_
base_config_ipmp_networkifs="pcn0_pcn1"
*base_config_ipmp_networkif_pcn0_pcn1="prod sp 192.168.1.211 192.168.1.212 24 cxnode1 192.168.1.201"

(the ones with * are the variables I had to update when I created the cxnode2 template) 

Installing Solaris Cluster on the nodes

As the nodes were running in 32bit mode, (VirtualBox 1.7 should have 64bit support, which means this part won't be necessary in the future) I needed to use an internally available set of 32 bit kernel drivers in a packages called SUNWscka. Basically, install sc3.2 as usual using the installer, but don't configure anything, and don't configure a quorum device. Then pkgadd SUNWscka on both nodes, and reboot them both.

The rest

After that, I just followed Gordon's instructions for setting up the Quorum server, setting up the iSCSI devices on the hosts, mounting them on the cluster nodes, and creating a zfs pool using them.

Sadly, this now means I can play with and configure all sorts of things in a Sun Cluster environment from the comfort of my laptop!

tags: Solaris Jumpstart JET xvm virtualbox