Hal Stern's thoughts on the economy, software, services, technology, and snowmen. Hal Stern: The Morning Snowman

We deal with nesting in many different contexts: VPNs (IP networks inside of other IP networks), filesystems (directories of directories, or mount points on top of other mount points), containers (instances of the operating system inside of other instances) and the more common functional recursion that's taught in 101-level coding classes. The common thread is that nesting (or recursion, in a more generalized form) lets you build complex structures more simply (in some cases), provided you do the necessary design work to allow the structures to nest in the first place: ensure there's a basic order of operations so the system can recover from failure, avoid mangling global state when there's more than one instance, and identify (and avoid) infinite nesting. And if you do a good job, you can treat everything below you as a black box.

Today, we have a hierarchy of nesting structures that give data center architects a wide variety of choices and control points. The announcement of Sun's Project Blackbox is just the highest-level abstraction in this stack:

CPU-level virtualization. Define the instruction set and system interfaces, and run multiple operating systems on slices of a CPU. Sun is previewing Logical Domains for our UltraSPARC T1 processor; you can also use VMware or Xen on your favorite x64 processor at the lowest level of virtualization. We've also announced Solaris support for Xen, making Solaris both a guest and a host operating system.

OS-level virtualization. Containers, or the ability to run an instance of the OS inside itself. Useful for isolating application code that you don't know or trust, or for providing a complete system image view for applications that are somewhat anti-social and assume they're in the one-app-per-box world of a few years ago. If you agree on the OS interfaces, and can isolate and aggregate at that level, then you're good to go.

Stack-level virtualization. Define the interfaces to which you deploy code, and ensure that you don't slide outside that bounding box, and you can virtualize up the stack. It's not a foreign concept; it's basically how salesforce.com is built, as well as any number of web site hosting companies that give you an httpd server, a PHP engine, and a set of packages like WordPress that you can drop on top of their stack. Want to use something not on the package list? Find someone who provides (and manages) at the OS level. Being able to compress your interface requirements into a statement of stacks, though, gives you incredible flexibility and power in terms of how you deploy. I had first-hand experience with this last month -- one of my volunteer projects was running the mail and web server for a local religious organization. The entire system could be described as IMAP4, Apache, and majordomo. After seven years of nearly flawless operation, the system died an ugly death -- so we simply shipped the user data to another hosting provider that gave us those three interfaces and we were back on the air the next day. The long pole in the time tent wasn't system configuration but rather moving our DNS entries.

Data center level virtualization. How do you manage it? Capture events? Do physical security? Identify the basic unit of work, deployment, and cost for power, cooling, space, and compute/storage density? These are all questions that get asked as part of virtualization at a lower level of abstraction (anyone who wants to implement CPU-level virtualization but hasn't built a model for allocating applications to CPU resources is making work, not progress). Answer these questions about your data center and you can build a box -- literally -- around it.

That's Project Blackbox: stimulating conversation about data center abstraction. Generating some interest in literally boxing it up and moving it to where the space, power, cooling, or environmentals are more friendly.

With all due respect and apologies to Jon Anderson, the word is live: Innovating@Sun is now available.

Tuesday, October 17, marks the first of our Innovating@Sun podcasts, a kind of funky, edgy, view of what's going on at Sun and why it's relevant -- to employees, customers, passers-by and anyone else who can spare 10 minutes of airtime. My simple goal was a rough stylistic mash-up of Stephen Colbert, James Governor, and Mark Cuban, without straying from technical accuracy or overloading the volume compander on our recording console.

The last time I had this much fun with a microphone was when I was the voice of Sunday Morning Jazz on WPRB 103.3 FM, in the mid-80s, and I had a limited but loyal crew of listeners (including my parents who were sometimes able to pick up the signal with appropriate hand-holding of the FM antenna). Some of the faithful were incarcerated in the Trenton state correctional facility, and it seems their choice of Sunday morning listening was made for them by someone who liked my show and had popular (or declarative) control over the dial at that time.

Here's hoping you like the thoughts, whether or not you're forced to listen to them.