The View from the Moon

20070905 Wednesday September 05, 2007

Project Etude, Revealed

Well, it has been a while since I last wrote anything here.  I have been working on a team developing a new project, which we named Etude.  Marc Hamilton, VP of Solaris Marketing, has written about it here.

[As an aside, it's weird to have a code name you selected cross the lips of senior executives several months later] 

In a nutshell, we've built a Solaris Container (or Zone) which is capable of running the Solaris 8 user environment.  We have also created a capability to perform P2V (or Physical-to-Virtual) transformation of existing Solaris 8 systems into containers running on a Solaris 10 host.  This is an enabler for rapid migration of legacy Solaris 8 environments onto modern, environmentally friendly, cost effective hardware.  And onto Solaris 10.  The idea is to break up the upgrade tasks into chunks, allowing the hardware and OS to be upgraded, while continuing to run legacy environments.  Next, the legacy environments can be used until they are retired, or redeployed into Solaris 10 containers, or into logical domains.

I will write more about the why and the how of this project in a future blog entry.  (And Marc does a good job of explaining some of the why in his blog, so go there for more background).  But for now, I want to expand upon the what.  The notion of a Solaris 8 container was not a new one when we began to look at this problem early this year.  But with the completion of the BrandZ project, we had the tools in hand to make a serious attempt to realize this idea.  BrandZ was originally developed so that we could have a "Linux Zone" but the core "brand" framework is really very flexible (and is now more so, thanks to our work).  It allows the development of a variety of OS personalities atop Solaris 10.

So who built Etude? I'm very lucky to be leading a group of seasoned and incredibly talented engineers: Bill, Ed, Jerry, and Steve (no blog), and Penny on documentation.  We worked at something of a breakneck pace to assemble a prototype, which we had in hand by late April. Since then, we have worked to convert the prototype into a production quality offering.  I'll try to say more about that later.  Along the way, we've had a lot of help from many corners of the company-- people too numerous to list.  I especially want to thank Bill Franklin, Jerri-Ann, Joost, Susan, Allan, Richard, Tim and Liane.

Ok, so I'm anxious to show off our creation. First, let's archive a Solaris 8 system using the Flash Archiving tools. If your S8 system is patched up to date, you already have these tools installed. Alternatively, you could use CPIO or ufsdump or some other tool to create a suitable archive:

s8-system # uname -a
SunOS s8-system 5.8 Generic_108528-29 sun4u sparc SUNW,UltraAX-i2
s8-system # flarcreate -S -n s8-system /net/s10system/export/s8-system.flar
Determining which filesystems will be included in the archive...
Creating the archive...
Archive creation complete.
s8-system #

[Note to our beta customers!  In the beta version of Etude, support for Flash Archives is not present, so if you use this blog entry as a substitute for reading the documentation, you will be disappointed.  Please ensure that you carefully read the supplied instructions] 

In the above example, I've simply used NFS to place the flash archive onto my Solaris 10 system, but you could use any method for moving files around. Now, we'll head over to the Solaris 10 system and create a container suitable for use as a Solaris 8 environment...

s10-system # zonecfg -z s8-system
s8-system: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:s8-system> create -t SUNWsolaris8 zonecfg:s8-system> set zonepath=/aux/zones/s8-system zonecfg:s8-system> add net zonecfg:s8-system:net> set address=10.6.62.226/24 zonecfg:s8-system:net> set physical=e1000g0 zonecfg:s8-system:net> end zonecfg:s8-system> add dedicated-cpu zonecfg:s8-system:dedicated-cpu> set ncpus=2 zonecfg:s8-system:dedicated-cpu> end zonecfg:s8-system> commit zonecfg:s8-system> exit s10-system # s10-system # zoneadm list -vc ID NAME STATUS PATH BRAND IP 0 global running / native shared - s8-system configured /aux/zones/s8-system solaris8 shared

The last part (setting ncpus=2) is not strictly necessary, but is a good example of how easy it is to allocate CPU resources to containers starting in Solaris 10 8/2007. Jerry (who led that work) and Jeff Victor have written more about this capability here and here.  (By the way, Solaris 10, 8/07 was launched yesterday, and includes BrandZ and many other enhancements to the containers infrastructure.)

Now that we've configured the zone, we need to install it.  We'll use the flash archive we made to do so:

s10-system # zoneadm -z s8-system install -a /aux/flar/s8-system.flar
      Log File: /var/tmp/s8-system.install.104490.log
        Source: /aux/flar/s8-system.flar
    Installing: This may take several minutes...
Postprocessing: This may take several minutes...

        Result: Installation completed successfully.
      Log File: /aux/zones/s8-system/root/var/log/s8-system.install.104490.log
s10-system # 

That's it! We've now installed a Solaris 8 container! Let's boot it up, and log onto the console, and see what happens:

s10-system # zoneadm -z s8-system boot; zlogin -C s8-system
[Connected to zone 's8-system' console]

SunOS Release 5.8 Version Generic_Virtual 64-bit
Copyright 1983-2000 Sun Microsystems, Inc. All rights reserved

Hostname: s8-system
The system is coming up. Please wait.
NIS domainname is mpklab.sfbay.sun.com
starting rpc services: rpcbind keyserv ypbind done.
syslog service starting.
Print services started.
The system is ready.

s8-system console login: root
Password:
Sep 5 17:55:52 s8-system login: ROOT LOGIN /dev/console
Last login: Wed Sep 5 13:11:37 on console
Sun Microsystems Inc. SunOS 5.8 Generic Patch February 2004
s8-system # uname -a SunOS s8-system 5.8 Generic_Virtual sun4v sparc SUNW,Sun-Fire-T200 s8-system # psrinfo 0 on-line since 08/29/07 13:32:21 1 on-line since 08/29/07 13:32:23

We could have also performed a sys-unconfig(1m) on the Solaris 8 image during the container installation (by passing -u to the installer). In that case, we would have been asked to answer the usual system identity questions. This zone can be cloned, moved around, renamed, attached/detached and manipulated like any other. You can even install it atop a ZFS filesystem, and from the global zone, use DTrace against the applications running inside of it.

Most importantly, you can run real workloads. Building on the Solaris Application Binary Compatibility Guarantee, we have done the difficult work to make sure that your applications will work successfully inside of these containers. This includes software such as databases, application servers, Java programs, web servers, and more. We've also utilized the amazing array of test suites we have available in-house.  You can even do software development inside of Solaris 8 Containers, building binaries which will run on any Solaris 8 (and 9, and 10) systems.

You can also patch these containers using the same tools you use to patch Solaris 8 (and using the same patches). We've even pulled down hundreds of Blastwave packages to test gnome, KDE, and lots of other applications available there. You can run your favorite ancient desktop environment: Solaris 8 was the last version of Solaris to include OpenWindows. So here's the obligatory screenshot from the wayback machine: Netscape 4.x, Java 1.3 and some other miscellaneous stuff all running atop the OpenWindows environment.  Running on emulated Solaris 8, on Solaris 10, on a T2000. A weird blast from the past mixed up with the present day.  (It's interesting to think that today's whizzy desktop applications will in a few years look just as antiquated as OpenWindows and Netscape 4.x do today).  Click the image to zoom in.

 


That's all I have time for tonight.

If you're interested in hearing more about this technology, talk to your local Sun Representative. Or if that fails for any reason, send me an email (my-firstname dot my-lastname @sun.com will do the trick) and I'll try to hook you up with someone helpful (please put "Etude" in the subject line).


(2007-09-05 21:11:14.0) Permalink Comments [14]
Tags:
Trackback: http://blogs.sun.com/dp/entry/project_etude_revealed
 

Comments:

This is awesome Dan! Are you planning to do the same thing for Solaris 9? This is definitely something my company will use!!!

Thanks for the cool blog post,
- Ryan

Posted by Matty on September 06, 2007 at 04:05 AM PDT #

This is very interesting, Dan, and something I had just been briefed on a week or so ago. I'm a little confused by your blog, in that you seem to suggest this is currently possible with the BrandZ capability of 08/07. Is that true, or is there additional software that needs to be installed in order to make use of this?

-spp

Posted by Stephen Potter on September 06, 2007 at 07:09 AM PDT #

I would love to see this for a Solaris 9 environment too.

Adam

Posted by Adam Zimmerman on September 06, 2007 at 09:04 AM PDT #

Awesome!

Posted by Jonathan Adams on September 06, 2007 at 09:54 AM PDT #

Stephen: Solaris 10 8/2007 will not have this capability "in the box". However, we are trying to place this in the hands of customers as soon as we can.

This is another brand (a "sister" perhaps, to the 'lx' brand) which we are developing. My bosses say I'm not to say too much about the exact form of (or date of) delivering this thing in a form usable by customers, so I had to be intentionally vague. I think that this is because they are still hashing out some of the details.

Posted by Dan Price on September 06, 2007 at 10:43 AM PDT #

Adam, Matty: Thanks for the input about Solaris 9!

Jonathan: Thanks man! We even managed to cope with your changes to doors :)

Posted by Dan Price on September 06, 2007 at 10:45 AM PDT #

If you need help testing, please feel free to contact me, you can get my email address from my blog uadmin.blogspot.com I also have a client that would be interested in testing this as well, they have midrange (fujitsu primepower) and t2000's that could be used in testing.

Let me know.

Posted by James Dickens on September 06, 2007 at 11:20 AM PDT #

Will BrandZ be ever capable of running FreeBSD as guest OS?

Posted by Juraj Lutter on September 06, 2007 at 03:12 PM PDT #

Juraj: I don't personally plan to do this work (unless someone shows up with a business case), but there's nothing stopping someone else from taking on this challenge.

When you say FreeBSD, do you mean on FreeBSD on SPARC, x86, AMD64, or all three?

Posted by Dan Price on September 06, 2007 at 03:27 PM PDT #

Congratulations on a job well done. Will this by chance work with Trusted Solaris 8 (TSOL8)? While cpio and flar can’t handle labels, TSOL8 ufsdump and tar can so I could transfer using these methods and possibly dd. Would I be wasting my time trying?

Posted by David Gaines on September 07, 2007 at 03:37 PM PDT #

It's great, and what about Solaris 7 or 2.6, i think this systems are also used at many place.

Posted by Cni on September 12, 2007 at 05:20 AM PDT #

One of the key concepts at Blastwave is to port and build on Solaris 8 such that any legacy Solaris user will have almost everything that they expect to get in the Linux world. I really don't want a Solaris legacy user to ever go from Solaris 8 to Linux simply because they could not get up to date ( and maintained ) Apache and GIMP. That is just a throw away example because we both know that these apps really come with a huge tree of dependencies. A massive dependency tree that also needs to be maintained. Those are dependecies that we need in Solaris 8 but end up being duplications or updates to those in Solaris 10 and Solaris Nevada. Regardless of this uncomfortable burden we are able to at least manage the revisions of those dependencies en-masse and perform grand updates that may touch hundreds of software packages. Boom. Instantly you get hit with "Blastwave bloat" and your /opt filesystem gets chewed up. But now you can run Scribus and Stellarium :

http://www.blastwave.org/dclarke/stuff/scribus-1.2.1-openwindows.png

http://www.blastwave.org/dclarke/stuff/stellar/stellarium_006.png

In any case ... I think that Blastwave will keep doing what it does for the legacy users *and* the community at large. Noe eTude allows those legacy Solaris 8 users to finally move forwards in some measurable way and release old hardware. Excellent!

As an aside, with a bit of work we can convert some ( or all ) of those old SVR4 format packages to something digestable by Indiana next month.

Fascinating times we live in.

Dennis

Posted by Dennis Clarke on September 13, 2007 at 05:28 PM PDT #

This is great news

Posted by Kalanthes on September 20, 2007 at 12:49 AM PDT #

"OS Personality"... 'now that's harkening back to
Taligent's use of the term! ....they had their
OO-based OS that would allow for pluggable
OS Personalities.... and therefore running any kind
of OS on the microprocessor.

Posted by andrew on September 24, 2007 at 07:58 AM PDT #

Post a Comment:

Comments are closed for this entry.
Dan Price's Weblog
[about me]