Wednesday January 16, 2008

The Three Most Important Applications

Around 1995, I gave a series of academic talks that tried to capture what I had learned at Sun during my first couple of years away from teaching at M.I.T. My biggest lesson was that, in the world of enterprise computing, there were three applications that really mattered: Databases, Big Databases, and Really Big Databases. I actually went way out on a limb predicting that, by 2000, we'd likely see terabyte-sized production databases (imagine that!).

The punch line being how databases were shaping key aspects of server and storage systems design at Sun: large memory, lots of I/O and memory bandwidth, RAS, symmetric multiprocessing and, of course, an operating system (Solaris) that could grok it all. We ended up creating systems that were naturally very well-suited for running, well, really big databases from the likes of  DB2, Informix, Oracle, Sybase. We also worked very closely with all of these folks to continually tune performance and bolster availability.

Good for us at the time, a bunch of people found many of these system design values --- especially around memory bandwidth and I/O --- made great Web 1.0 machines, too.

A decade later, databases matter even more. They are to storage what application containers are to computing. That isn't to minimize the importance of file systems --- those are the foundational storage abstractions, just as threads and processes are to application containers like Apache and Glassfish. Databases have continued a primary influence over big swaths of  our systems design (and so has high performance computing). The overall system center now being scaled-out network assemblies of web/application and database tiers.

In the contemporary web era, not only have the enterprise databases grown in force (I'd rightfully add SQL Server to the list today) but open source databases (OSDBs) have come into their own: MySQL, PostgreSQL and Derby (to name but a few). These have wonderful affinity with the modern application containers, especially PHP and Java. And, indeed, MySQL has become foundational to the web, the M in LAMP.

And guess what? We've been targeting big swaths of our $2B R&D budget to engineer systems that run these workloads really well, too. The exciting part for thousands of engineers at Sun is that now we  get to rub shoulders with the great engineers at MySQL. We are champing at the bit to optimize and scale systems in a myriad of ways: from microelectronics to memory systems to storage to kernel engineering. In the magic transparency of open source, these optimizations will lift all boats.

And that is the truly exciting part. We now get to openly develop a new wave of very deep innovation in hardware and software systems. Ones that will continue the movement of  customer's capital to be invested in those who sustain in truly adding value, rather than adding to switching costs.

 A big open embrace to everyone at MySQL and welcome to the Sun family. This is going to be fun!