I hate computers. No, that's not true. They hate me. Probably because I derive pleasure from pushing them to the hirsute edge of their design specifications. They interpret this as sadistic behavior, and think that I am mean for no reason. But that's not true. I have a reason.

Actually, I have two reasons:

1. I am curious, and enjoy measuring things
2. I want to help you understand the limits of Solaris so you can safely maximize the benefits you obtain from using it, without performing all of these measurements yourself.

Over the past few weeks I have wondered how realistic Solaris' upper bound of 8,191 non-global zones (per Solaris instance) is. So I "acquired" access to a Sun Fire X4600 which has a pre-release build of Solaris on it. This build includes several useful features from Solaris 10 11/06 including:

• zone migration: the ability to move a non-global zone from one computer to another
• zone clone: make an identical copy of an existing non-global zone
• integration of zone clones and ZFS clones, enabling incredibly efficient cloning of zones.

The efficiency extends to both time and space. Originally the only method to create a copy of a zone was configuring a new zone like an existing zone and installing it. This operation took 10-30 minutes, depending mostly on disk speed, and did not include any customizations made to the original zone after it was installed.

Spawn of Non-global Zone

The newer "zone clone" method is a vast improvement because it simply copies the files from the original zone to the new zone. This is much faster (2-5 minutes) and carries with it any post- installation customizations. This method, along with other new features in Solaris 10 11/06, enables a new method of provisioning virtual Solaris systems. I call this new method 'spawning a zone.'

Suppose that your data center uses five applications, but multiple computers run each application for different purposes. These might be test and production systems, for example. Each system that runs the same application has the same set of customizations applied to it.

Many data centers simplify the provisioning process by creating, testing, and internally certifying one or more Solaris bootable images, called 'gold masters,' which are then installed on their systems. This also simplifies troubleshooting because it removes the need to analyze a troubled system for customizations. In some cases, system administrators are specifically trained to troubleshoot systems installed with a gold master.

Solaris zones can be applied to this model by creating a gold master Solaris image that has "gold master zones" pre-installed on it. Each zone is customized for one of the five applications. After a new system is installed from the gold master, only the zone(s) customized for the applications that will be used by this computer are turned on. The rest are left off, merely using up a few hundred megabytes of disk space.

That model is simple and very useful, but often is not flexible enough. For example, it does not allow for multiple instances of one application on one system, each in separate zones.

Instead, a "gold master system" could be created that has one "gold master zone" installed per application, with appropriate customizations made to each zone. When an application is needed on a new or existing system, application provisioning includes these steps:

1. clone a gold master zone
2. migrate the clone to its new home system (also see my How To Guide on this topic)
   1. "zoneadm detach" the zone from the old system
   2. move the zone's files to the new system
   3. "zoneadm attach" the zone to its new system
3. destroy the detached zone on the gold master system

You might imagine the gold master system as a queen bee, spawning zones instead of drones.

The new, recently migrated zone can then be further customized. If conditions change, and you want to move it to a different computer, you can do that, too.

Part 1 - Summary: Well, this has gotten a bit long-winded, but the explanation in this entry provides the background necessary to understand the next entry, which in turn (finally!) explains this entry's title.

Footnotes:

1. If that word isn't your vocabulary word for the day, perhaps it should be.
2. Yes, I know: they can't think. But let's not tell them that. It will ruin their day.
3. Back in college I was a founding member of the theatre group's Acquisitions Committee, so I have some experience in this area.

Part 2 of Many

Another network feature that won't work in a non-global zone in Solaris 10 3/05, 1/06, or 6/06 is the service "dhcp-server". I wondered if appropriate privileges could be assigned to a zone, using Solaris 10 11/06, in order to enable that service to work properly in a non-global zone.

But how do you know which privilege(s) are needed? Although a tool to analyze an executable (and the libraries that it uses) for necessary privileges would be very useful, I am not aware of such a tool. However, there is a tool which will analyze a running program: privdebug.

I used dhcpmgr(1M) to configure the global zone of one Solaris 10 system to be a DHCP server, and told another Solaris 10 system to be a DHCP client by creating the appropriate /etc/dhcp.<interface-name> file. Then I ran privdebug to start gathering data.

After running privdebug as:

# ./privdebug.pl -n in.dhcpd -v -f

its output looked something like this (abbreviated slightly):

STAT TIMESTAMP          PPID   PID    PRIV                 CMD
USED 481061858324       7      1489   proc_fork            in.dhcpd
USED 481063008106       1489   1490   sys_resource         in.dhcpd
USED 481067169173       1489   1490   net_privaddr         in.dhcpd
USED 481067214515       1489   1490   net_privaddr         in.dhcpd
USED 481067261082       1489   1490   net_privaddr         in.dhcpd
USED 7602182665254      7      2307   proc_fork            in.dhcpd
USED 7602184084176      2307   2308   sys_resource         in.dhcpd
USED 7602195780436      1      2308   net_privaddr         in.dhcpd
USED 7602195826717      1      2308   net_privaddr         in.dhcpd
USED 7602195874362      1      2308   net_privaddr         in.dhcpd
USED 7617671777513      1      2308   net_icmpaccess       in.dhcpd
USED 7618028208673      1      2308   sys_net_config       in.dhcpd
USED 7618028224029      1      2308   sys_net_config       in.dhcpd
USED 7618028622618      1      2308   sys_net_config       in.dhcpd
USED 7618937845453      1      2308   sys_net_config       in.dhcpd
USED 7618937861126      1      2308   sys_net_config       in.dhcpd
USED 7786427652239      1      2308   net_icmpaccess       in.dhcpd
USED 7786782253121      1      2308   sys_net_config       in.dhcpd
USED 7786782266742      1      2308   sys_net_config       in.dhcpd
USED 7786782417242      1      2308   sys_net_config       in.dhcpd

With that list, it was easy to check each of the privileges that in.dhcpd used against the list of privileges that are allowed in a non-global zone.

Although proc_fork, sys_resource, net_privaddr and net_icmpaccess are in a non-global zone's default list of privileges, sys_net_config is not allowed in a non-global zone. Because of that, a non-global zone cannot be a DHCP server using Solaris 10 11/06.

That was a fun experiment, but in order to make a non-global zone a DHCP server we must wait for the Crossbow project to add sufficient IP instance functionality, along with its new sys_ip_config privilege. The latter will be allowed in a non-global zone.