So I admit I haven't been very productive on the the blog front. As you can see my last posting was in July last year... there's always something happening...

And like a new years resolution I promise I'll do better this year... any way it's always good to have something to get me into gear, so now I have something.

The fact that Virtualization is hot right now and Solaris Containers are interesting to customers can be measured in many ways. For example that Menno Lageman's BluePrint on what Solaris Containers are and how to use them was in the top three downloaded BluePrints last year, or the amount of attention the zones project pages and discussion groups are getting on OpenSolaris. And yesterday we launched a Solaris Containers Learning Center (as well as a ZFS learning Center).

However the real measure is in how customers and partners are adopting the technology. So, for example, we have some really nice Solaris Container customer references in INEEL (a secure government HPC customer), UGS Corp. (they make product lifecycle management software), Ning.com (a Marc Andreesen Web 2.0 company), and NewEnergy Associates also in this interesting article (they make software for gas and electric power companies).

We've also been working with our solution partners like EDS to get them up to speed with the technology and get them all the information they need to deliver a solution they can sell to their customers, in this case consolidation solutions. These solutions take all the aspects into account that a full enterprise class solution would need, considerations like deployment, monitoring, patching, backup and restore, filesystem layout et cetera. As a part of this two of their leading architects Keith Bates and Ron DeMena have written a whitepaper that is now available from our EDS partner site. Very cool!!!

So I'll really try to post stuff more frequently, but there always something happening...

Back from the break digging though my email I found this interesting article on Solaris Containers.

To me the most intersting thing was not the way Solaris Containers are explained nor its relevance wrt Linux and its virtualization capabilities, but more the quote from a customer who explains how they are using Solaris Containers to save money in their development department. Plus that this technology that is not only usefull on the big boxes but also on the small 1-2 processor systems.

I just bumped into this interesting comparison chart on Solaris Zones vs FreeBSD Jails, Linux/BSD chroot, User Mode Linux, and VMware. Created by James Dickens it give a quite complete list of the thing you would care about if you're an systems administrator.

Very nice.

Technorati Tag: Solaris

I've been working on Resource Management in the Solaris group for some years now and have been lucky to be involved with quite some interesting Solaris technologies like Solaris 9 Resource Manager, IPQoS, Solaris Zones, Solaris Service Manager (smf(5)),... All pretty cool stuff and used by Solaris admins the world over.

But why did we build it the way we did?

Well we talk to a whole bunch of customers, we try to understand what their challenges are and what we need to build to help the meet these challenges and how they change with time. However there is always a chance we don't actually build what you need, or we initially focus on other aspects than you need...

So what if we open the source code and allow other developers to also contribute? Well you get to join in, there is no more direct way to tell us what you need. This is one of the reasons I'm really enthusiastic about OpenSolaris, you get really cool results. Either making the stuff we built better or just coming up with completely new things. An example of what I mean is ZoneBSD, an opensource initiative with as goal "will be to factor as much Solaris code as necessary into FreeBSD in order to support running a FreeBSD container within a Solaris 10 Zone", really cool stuff... or as they say around here "Sweeet"

So come on in, the water is mighty fine...

Technorati Tag: OpenSolaris

Technorati Tag: Solaris

Just a quick entry to make sure you're all aware of the execlent new BluePrint on Solaris Containers by Menno Lageman. It goes through all the different technologies, commands, and how to use them with full blown examples that you can follow. It even has an example smf(5) manifest and method you can use for to put an Oracle database into Solaris Predictive Self Healing.

I thought I'd mention it eventhough it's been out for a few weeks now and it's been linked to quite a few times by others too.

Thinking I needed a good story as a reason to start writing a blog I hadn't joined the many friends who have been blogging for quite some time, but now I found a good reason. Btw. For those of you who like short blog entries I definitely don't plan to have entries this long too often either ...

I guess an introduction is in place for my first Sun Blog. My name is Joost Pronk van Hoogeveen (yes I had to write my full surname on every sheet on every exam I ever took), although I normally only use Joost Pronk to make it easier for myself and others. I work in the Solaris group and work on the resource/workload management tools (Solaris Resource Management, Solaris Zones, Solaris Containers, Solaris Container Manager, ...). I expect that's probably what I'll blog about most... Oh, and I'm Dutch so I might throw some of that in too.

Anyway a few weeks ago I was at a Sun customer conference discussing the fact that we currently don't support running Solaris Containers on NFS. More precisely we don't support the ability to have a Solaris Zone with its root filesystem on an NFS mounted directory.

Before I go further I should discuss the Solaris Container versus Solaris Zones nomenclature. In short: Solaris Containers = Solaris Zones + Solaris Resource Management.

The longer version: Solaris Containers is the group of technologies that allow the administrator to create separate environments for their applications in Solaris. Solaris Zones is the technology that allows the admin to create a separate logical namespace environments, and the Solaris Resource Management (RM) allows them to assign amounts of system resources to their applications. To create a complete "Container" you'd want to use both technologies and not just one. Now Solaris Zones is really new and cool and it gets most of the lime-light but a good container also has the RM stuff configured too. Now when talking about it I generally default back to the "bigger" Solaris Container to not lock out the use of RM, but sometimes you need to zoom in to be precise and I'll talk about an individual piece, which is often a zone because that's "the new thing" people want info about.

So for a whole bunch of reasons running a zone's root filesystem on an NFS mounted directory doesn't work today, and most of them have to do with the fact that the global zone (who makes the initial NFS mount) has a different IP address than the non-global zone (who then wants to use this mount), and credentials, and the possible use of Kerberos, and different domain names, and ...

What if the non-global zone doesn't see any of this because it's behind some other mechanism?

At breakfast a customer (Thomas Nau from the University of Ulm) and I were talking about this when he said; "why don't we use lofi?" So we did it right then and there with two laptops and it works (big grin).

First let me say this is a workaround hack, we didn't do anything illegal, and all the interfaces we used are regular Solaris interfaces, however it comes dangerously close to the "don't do this at home folks" category. So think twice before you'd use it in a production environment, but it definitely fun to do.

In short what we do is: We mount an NFS filesystem; Create a file; lofiadm this file; newfs this new lofi block device; mount the device on de zonepath location; Define a zone with the correct zonepath; Install and boot the zone. (sounds straight forward, right?)

Make sure you have an NFS share that you can mount:

root@nfsserv # share
-               /export   rw   ""
root@nfsserv # pwd
/export/zones

This means create a mount point (/zonemount), mount the filesystem on this, create the file:

root@zoneserver # mkdir /zonemount

root@zoneserver # mount nfsserv:/export/zones /zonemount
root@zoneserver # df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0    8242397 5397348 2762626    67%    /
/devices                   0       0       0     0%    /devices
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  443904     400  443504     1%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
fd                         0       0       0     0%    /dev/fd
swap                  443560      56  443504     1%    /var/run
swap                  446320    2816  443504     1%    /tmp
nfsserv:/export/zones 17088954 3549610 13368455    21%    /zonemount

root@zoneserver # cd /zonemount
root@zoneserver # mkfile 300M zone1_file
root@zoneserver # ls -l
total 614720
-rw-------   1 root     root     314572800 Jun  1  2005 zone1_file

Now we have a 300MB file we can use lofiadm(1M) to create a new block device in /dev/lofi. In this case we choose to specify which device (/dev/lofi/1) we want. Once we've done that we use newfs(1M) to put a filesystem on our device/file:

root@zoneserver # lofiadm -a /zonemount/zone1_file /dev/lofi/1
root@zoneserver # newfs /dev/lofi/1
newfs: construct a new file system /dev/rlofi/1: (y/n)? y
/dev/rlofi/1:   614400 sectors in 1024 cylinders of 1 tracks, 600 sectors
        300.0MB in 64 cyl groups (16 c/g, 4.69MB/g, 2240 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
 32, 9632, 19232, 28832, 38432, 48032, 57632, 67232, 76832, 86432,
 518432, 528032, 537632, 547232, 556832, 566432, 576032, 585632, 595232,
 604832,

Now we have the device, we need to mount it where we plan to have our zonepath's. So first we mkdir(1M) a new mount point /zones/zone1, and then mount the new filesystem on top of this mount point:

root@zoneserver # mkdir /zones/zone1
root@zoneserver # mount /dev/lofi/1 /zones/zone1
root@zoneserver # chmod 700 /zones/zone1

Note: Because we created the filesystem instead of zoneadm doing if for us we need to use chmod(1) to make the directory only accessible to the global root user.

So now we'll quickly create a fairly standard non-global zone and tell it to install:

root@zoneserver # zonecfg -z zone1
zone1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:zone1> create
zonecfg:zone1> set zonepath=/zones/zone1
zonecfg:zone1> add net
zonecfg:zone1:net> set physical=hme0
zonecfg:zone1:net> set address=192.168.1.101/24
zonecfg:zone1:net> end
zonecfg:zone1> verify
zonecfg:zone1> exit

root@zoneserver # zoneadm -z zone1 install
Preparing to install zone .
Creating list of files to copy from the global zone.
Copying <2568> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <946> packages on the zone.
Initialized <946> packages on zone.                                
Zone  is initialized.
The file  contains a log of the zone installation.
root@zoneserver # zoneadm list -cv
  ID NAME             STATUS         PATH                          
   0 global           running        /                             
   - zone1            installed      /zones/zone1                  

root@zoneserver # df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0    8242397 5397356 2762618    67%    /
/devices                   0       0       0     0%    /devices
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  442520     400  442120     1%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
fd                         0       0       0     0%    /dev/fd
swap                  442176      56  442120     1%    /var/run
swap                  445136    3016  442120     1%    /tmp
nfsserv:/export/zones   17088954 3549875 13368190    21%    /zonemount
/dev/lofi/1           288239   64799  194617    25%    /zones/zone1

root@zoneserver # zoneadm -z zone1 boot
root@zoneserver # zoneadm list -cv
  ID NAME             STATUS         PATH                          
   0 global           running        /                             
   1 zone1            running        /zones/zone1                  

Voila! It works

Note: df(1M) now shows us both mounts where the zone install only used 64799 KB out of the 288239 KB available on our lofi device-file-thing.

So why does this work? Well all of the NFS credential stuff that would normally break booting a non-global zone from NFS is now hidden from the zone. All of the NFS related stuff is done by the global zone and its credentials, the NFS server side never even sees any of the local zone files nor does it need to know about the IP addresses the non-global zone is using. On the other side the non-global zone never really knows its files are not on the local system, all its file are encapsulated in the filesystem living in the lofi block device. That this block device is actually a file on an NFS mount is not visible to it, nor does it care.

Well that's up to you. First as you can see above this is a fairly laborious process, and the more steps the more opportunities to mess up. Of course you can automate most of this, but still the debugability and measurability for someone who hasn't seen your setup isn't very high.

And then there is the version compliance. Having the zone root on NFS begs you to use it to migrate zones from one system to another, but be aware every zone has its package and patch list/revision embedded in it. So when you move them around they should only move to a system with an equal patch level, or you could get very strange behavior. what's going on there is worth a whole series of blogs and this entry is getting pretty long as it is...

Anyway, have fun.