It's time for another guest blogger.

Solving a Corner Case

One of my former colleagues, Joe Yanushpolsky (josephy100 -AT- gmail.com) was recently involved in the movement of a latency-sensitive Linux application to Solaris as part of platform consolidation. The code was old and it required access to kernel routines not available under BrandZ. Using VirtualBox as a virtual x86 system, the task was easier than expected.

Background

VirtualBox enables you to run multiple x86-based operating system "guests" on an x86 computer - desktop or server. Unlike other virtualization tools, like VMware ESX, VirtualBox allows you to keep your favorite operating system as the 'base' operating system. This is called a Type 2 hypervisor. For existing systems - especially desktops and laptops - this means you can keep your current setup and applications and maintain their current performance. Only the guests will have reduced performance - more on that later.

Here is Joe's report of his tests.

The goals included allowing many people to independently run this application while sharing a server. It would be important to isolate each user from other users. But the resource controls included with VirtualBox were not sufficiently granular for the overall purpose. Solaris Containers (zones) have a richer set of resource controls. Would it be possible to combine Containers and VirtualBox?

The answer was 'yes' - I tried two slightly different methods. Each method starts by installing VirtualBox in the global zone to set up a device entry and some of the software. Details are provided later. After that is complete, the two methods differ.

1. Create a Container and install VirtualBox in it. This is the Master WinXP VirtualBox (MWVB) Container. If any configuration steps specific to a WinXP environment are needed, they can be done now. When a Windows XP environment is needed, clone the MWVB Container and install WinXP in the clone. Management of the Container can be delegated to the user of the WinXP environment if you want.
2. Create a Container and install VirtualBox in it. This is the Master CentOS VirtualBox (MCVB) Container. Install CentOS in the Container. When a CentOS environment is needed, clone the MCVB - including the copy of CentOS that's already in the Container - to create a new Container. Management of the Container can be delegated to the user of the CentOS environment if you want.

In each case, resource controls can be applied to the Container to ensure that everyone gets a fair share of the system's resources like CPU, RAM, virtual memory, etc.

When the process is complete, you have a guest OS, shown here via X Windows.

Not only did the code run well but it did so in a sparse root non-global zone

Well that was easy! How about Windows?

Now, this is interesting. As long as the client VM is supported by VirtualBox, it can be installed and run in a Solaris/OpenSolaris Container. I immediately thought of several useful applications of this combination of virtualization technologies:

• migrate existing applications that are deemed "unmovable" to latest eco-friendly x64 (64-bit x86) platforms
• reduce network latency of distributed applications by collapsing the network onto a large memory system with zones, regardless of which OS the application components were originally written in
• on-demand provisioning, as a service, an entire development environment for Linux or Windows developers. When using ZFS, this could be accomplished in seconds - is this a "poor man's" cloud or what?!
• eliminate ISV support issues that are currently associated with BrandZ's lack of support for recent Linux kernels or Solaris 8 or 9 kernel
• what else can you create?

Best of all, Solaris, OpenSolaris and VirtualBox can be downloaded and used free of charge. Simple to build, easy to deploy, low overhead, free - I love it!

Performance

The advantage of having access to application code through Containers more than compensated for a 5% overhead (on a laptop) due to having a second kernel. The overall environment seems to be disk-sensitive (SSDs to the rescue!). Given that typical server load in a large IT shop is 15-20%, a number of such "foreign" zones could be added without impacting overall server performance.

Future Investigations

It would be interesting to evaluate scalability of the overall environment by testing different resource controls in Solaris Containers and in VirtualBox. I'd need a machine bigger than the laptop for that .

Installation Details

Here are the highlights of "How to install." For more details, follow instructions in the VirtualBox User manual.

• Install VirtualBox on a Solaris x64 machine in the global zone so that the vboxdrv driver is available in the Solaris kernel.
• Create a target zone with access to the vboxdrv device ("add device; set match=/dev/vboxdrv; end").
• In the zone, clean up the artifacts of the previous VirtualBox installation in the global zone. All you need to do is to uninstall the SUNWvbox package and remove references to /opt/VirtualBox directory.
• Install VirtualBox package in the zone.
• Copy the OS distro into a file system in the global zone (e.g. /export/distros/centos.iso, and configure a loopback mount into the zone ("add fs; set dir=/mnt/images; set special=/export/distros; set type=lofs; end").
• Start VirtualBox in the zone and install the client OS distro.

What advantages does this model have over other virtualization solutions?

• The Solaris kernel is the software layer closest to the hardware. With Solaris, you benefit from the industry-leading scalability of Solaris and all of its innovations, like:
  • ZFS for data protection - currently, neither Windows nor Linux distros have ZFS. You can greatly improve storage robustness of your Windows or Linux system by running it as a VirtualBox guest.
  • SMF/FMA, which allows the whole system to tolerate hardware problems
  • DTrace, which allows you to analyze system performance issues while the apps are running. Although you can use DTrace in the 'base' Solaris OS environment to determine which guest is causing the performance issue, and whether the problem is network I/O, disk I/O, or something else, DTrace will not be able to "see" into VirtualBox guests to help figure out which particular application is the culprit - unless the guest is running Solaris, in which case you run DTrace in the guest!
• Cost: You can download and use Solaris and OpenSolaris without cost. You can download and use VirtualBox without cost. Some Linux distros are also free. What costs less than 'free?'

What can you do with this concept? Here are some more ideas:

• Run almost any Linux apps on a Solaris system by running that Linux distro in VirtualBox - or a combination of different Linux distros.
• Run multiple Windows apps - even on different versions of Windows - on Solaris.

Additional notes are available from the principal investigator, Joseph Yanushpolsky: josephy100 -AT- gmail.com .

Attention Zonestat Fans: three bugs have been found in Zonestat 1.4. Two of them only happen if a zone boots or halts while using Zonestat. The third only happens if a zoneID is larger than 999.

I fixed all three bugs and posted v1.4.1 on the web site: http://opensolaris.org/os/project/zonestat/

Specifically:

• Bug: if a zone with dedicated CPUs is booted between poolcfg and output, zonestat can get confused or halt.
• Bug: if a zone is halted between "zoneadm list" and kstats, zonestat can get confused or halt.
• Bug: zones with a zoneID number > 999 are ignored.