The Glamor in Mass Transit (?)

Given a choice, few consumers would pick a double decker bus over an Italian sports car.

But if you were in the business of moving people at maximum efficiency, buses are hard to beat. Their mileage per passenger mile is >20x your average sports car. One way they achieve this, in the language of IT, is that buses parallelize transportation. They optimize for multi passenger performance, versus single passenger performance (largely the opposite of what most consumers do, parents in minivans notwithstanding).

That same focus on industrial efficiency (and divergence from consumer preference) has been washing over the datacenter for a few years. While consumers turn to Dolce and Gabbana phones, datacenters are increasingly pining for high performance buses - infrastructure optimized for utilization, efficiency, and overall performance, not just simple component speed.

That's a focus we initiated years ago - when we dove into chip multi-threading, shipping the industry's first octal (8) core microprocessor - each core with 4 threads of execution, making for a sub-$4,000 server capable of doing 32 threads of work simultaneously (with great mileage). (Click here to try one free.) We deliberately prioritized efficiency over single thread performance - and it was a good bet. Intel, AMD, IBM and Sun are now all investing heavily in multi-core platforms - with Sun farthest ahead, having taped out the world's only mainstream hexa-deca (16) core microprocessor. (Which I embarassingly referred to as a "hexacore" chip on our last earning's call... I've since been practicing, hexadecacore. Say it with me, hexa-deca-core.)

Hardware without software isn't worth much, and happily, Solaris knows how to "scale," or take advantage of all those separate lanes - so applications can take immediate advantage of the innovation without any extra work. Customers can run one application on each thread on the chip, or one app over many threads, or even assign different OS's to each core on the chip - with every permutation in between (now fashionably referred to as "server virtualization"). The result? Customers buy fewer, bigger boxes, consume less space and power, and generally get way better mileage. Traditionally, if you wanted to guarantee application performance, you gave an application its own server. Server virtualization products like VMWare and Solaris 10 let you condense lots of apps onto a single box, giving each the illusion it has its own box - while a policy engine automates how precious cpu and memory are allocated to guarantee the same performance. Again, radically reducing boxes, power, space, heat - and spend.

We're bringing that same virtualization to the storage world - having released a file system, ZFS, that knows how to "scale," eliminating volume management and abstracting away the complexities of dealing with herds of disk drives. Even when those drives fail (after all, there are only two types of disk drives: those that have failed, and those that are about to do so). With some of our customers deploying thousands and even tens of thousands of disk drives, ZFS allows for big pools to be simply aggregated together, delivering reliable service from cheap parts - with incredible simplicity and data integrity.

And that leaves only one segment of the datacenter untouched by a focus on parallelism and virtualization. And for the company that's always said, "the network is the computer," it's been an obvious gap. What about the network itself?

As you may know, most networking devices are single threaded - they parallelize work via physical ports. You want more network? Buy more ports. Yielding all kinds of cabling messes, waste, management headaches and even weight problems (copper weighs a lot, and raised floor datacenters are now hitting their weight limits - no, I'm not joking). The networking world has, for the most part, failed to keep pace with the brutal efficiency parallelization of the computing world.

In a perfect world, you'd want computers, not people, to dynamically allocate scarce networking resources, like we now see with server resources - assigning, say, lots of bandwidth and guaranteed service levels to high value customers, and less of both to low value (or 'best effort') services. You'd want a simple policy engine, not a human being, making such decisions - responding to demand or business rules without physical intervention (eliminating cable swaps and port proliferation, for example). That is, you'd want to virtualize the network, too.

That's why we just introduced Project Neptune - a silicon project that marries the parallelism of the microprocessor (for Intel, AMD and SPARC systems), with the parallelism of the underlying operating system (Solaris, Linux or Windows), with parallelism in the network itself. Which in concert with some software magic (which goes by the name of the Crossbow project) allows enterprises to collapse cabling, ports, cards and spending - by bringing parallelism to basic network infrastructure (for geeks, you can take multiple TCP streams and allocate them to different processor threads, spreading out load and freeing up CPU's/ports). Ports become a physical convenience, just like a server - what's happening inside depends upon rules or policies set by the user/administrator to automate such decisions. Like I said, the network is the computer, and the computer's virtualized, so why not the network?

Now who would find Project Neptune appealing? Anyone's whose spending, for either software licenses, administrative effort, NIC cards, cabling or hosting charges, is related to the number of ports in their datacenter. After all, the future of network computing looks a lot more like a greyhound bus station than a Formula 1 race - as unglamorous as that might be, buses are a lot more efficient, and a lot better for the planet. Fewer, more powerful ports, like fewer, more powerful servers, is a good thing (for us, anyways :).

If you'd like to try a Neptune card for free, just click here. And before you ask, yes, that is a Project Blackbox mocked up like a bus, and no, it's not a new form factor.

For far more technical details and insights (on Project Neptune and Crossbow), here are some great links and blogs:

Sunay's summary of Project Crossbow in Solaris 10.
Another great blog entry, this one from Ariel Hendel.
And here's another interesting read.

Update:

And best of all, here's a truly great podcast, brought to you by some of the engineers responsible for the innovations in Solaris and Neptune.

And lastly...

Today marks Sun's 25th birthday - there will be lots of folks focused on the proper celebrations (throughout the year - stay tuned). Having been at Sun for only 10 years, I'm very much in awe of our history, and still feel like a newbie (the podcast, above, features four individuals who have each, even the youngest, been at Sun for longer than I).

To me, the best way of celebrating Sun's history - is to celebrate the future we're helping to create. Neptune's a timely example. The network is the computer... indeed.

Happy Birthday, everyone!

Posted on 10:15AM Feb 24, 2007 | Comments[29]

Share this post  | | | | | |

My Family Photos - and ODF

I was staying with my parents a few years ago, and looking through a shoe box of old family photos. It was great, I was really enjoying them - until it occurred to me most of the photos were singletons. That is, they were the only copies. On earth. And of at least one individual from my family's past, there were only two or three photographs in existence. Yipes.

A shoe box, I thought. How archaic, right? What if there were a flood, or heaven forbid, a fire? These are photographs I want to share with my family, and to pass along for generations. I want my children to know their history. And their children and their children.

So I did what any good son would do, I convinced my parents to let me abscond with the box, I returned home, and I scanned the photos (and returned the box).

And then the scanned photos were sitting on my hard drive. On my laptop. In my kitchen (that's where my laptop lives).

Given what daily happens in my kitchen, that was probably less safe than a shoebox. So much for archaic. Strike one.

So I made a few DVD's. And distributed them around my house, and gave some to other family members. Suffice it to say, most non-professional system administrators are non-professional for a reason - most of the DVD's were lost. Strike two.

But good news, someone bright once said the network is the computer - I decided a while back to upload them to my on-line photo service. If you're going to watch a shoe box, you may as well hire someone who's watching other people's shoe boxes, and may in fact be the best in the world at such a task.

And then I thought...

How do I guarantee the service will be around, or that I'll be able to render the images I've stored there - not just in a year, but in five or fifty? What if the images outlive the technology?

And with that as a backdrop, now you understand at least one real world motivation behind something called the Open Document Format.

Imagine you're a legislator that writes a law, or a doctor that drafts a patient's record, or a student that writes a novel. And that five years or fifty years from now, you want to return to review your documents. Except the vendor that created the application used to draft those documents, the company that created the word processor, has either gone out of business, or decided to charge you $10,000 for a version capable of reading old file formats. Either scenario makes the point: Information always outlives technology.

What do you do?

First, you grumble. After all, the information you created is your information - not the vendor's. Just like your family photos, the last thing you'd want is a camera company demanding payment before you could see your photos. And that's the danger created by applications without open file formats. Remember, information outlives technology.

That's why we, alongside some of the industry's most important technology companies, and a bevy of governments and agencies around the world, created something called the Open Document Format (known affectionately as 'ODF'). ODF defines an open format for document based information that's independent of the applications used to create documents stored in ODF.

Which is a fancy way of saying if you write a law or a medical history or a regulatory filing in a word processor that supports ODF today, and need to gain access to it at any point in the future, you'll have the freedom to do so on your terms. Without being held up by an application provider. ODF is a true open standard, adopted and implemented by a diversity of vendors (from IBM and Sun, to Google, Red Hat and now even Microsoft), and embraced by an amazing spectrum of the planet. And it's royalty free.

Durability of information and file formats is exceptionally important to institutions or businesses with document retention policies that extend beyond the useful life of the software (and employees) creating the documents - and ensures the availability of information well into the future. The same applies to the photographs in the shoebox - as the CIO of my home, I want the images to outlive me.

And just in case you missed the menu item, we're working with Google to ensure interoperability between Google's office documents and OpenOffice documents - leveraging ODF as an exchange mechanism. Any document created in Google's office can be trivially exported to (and soon imported from) OpenOffice (see the screenshot). Together, the two products allow businesses and individuals to preserve access, across the globe and across generations, for laws, legal contracts, patient records, diaries and strategic plans. Along with spreadsheets and presentations.

Finally, for those new to OpenOffice, it's a free office productivity suite that will forever be free - to corporations and end users alike. As best we can count, we've distributed hundreds of millions of copies across the world (download here). And now that Microsoft has announced support for the Open Document Format, users can feel comfortable that OpenOffice can be added to any environment, home or office, not just across the developing world, but the developed. In a few weeks, you'll be able to download an ODF plug-in here, which will enable Microsoft Word, by default, to save to/read from ODF. Once installed, you'll see this in Word's Options panel:

(I'll provide a pointer when the plug-in is ready.)

From then on, ODF becomes your default format. Whether you're an oil company or a high school student - ODF will enable seamless interoperability between open source and closed source environments - for as long as the standard, not the technology or product, exists.

From a corporate perspective, this also allows a very natural migration to occur across large institutions - front office staff might stay on Microsoft Word, but the rest of the organization can move to an interoperable alternative (say, Google's word processor or OpenOffice - or both). Affordability and interoperability are a good thing for the internet - and for the successive generations we expect to use it.

Posted on 09:54PM Feb 12, 2007 | Comments[35]

Share this post  | | | | | |

Sun Analyst Conference

We just had our annual analyst conference - where we bring together financial and industry analysts for a day, and engage them in a dialog around our direction and perspective. Presentations are here, for those interested in what we had to say.

I thought the best presentation of the day, by far, was Greg's - on the coming explosion in our industry, and the separation of a class of customers that care about scale and efficiency in ways historically unseen in the IT industry. We call it the "Redshift."

For those that didn't know, Greg's a (former?) card carrying MIT professor. You can watch him here, well worth the time. And you'll definitely get a sense for where we're applying R&D for the next 10 years...

Posted on 04:00PM Feb 08, 2007 | Comments[20]

Share this post  | | | | | |