Which brings me to our announcement yesterday. Keeping data for long periods of time is important these days. Of equal importance is truly destroying data as required by internal corporate erasure or regulatory policy. That's why we developed a new on-site service to help customers with this challenge. Our experts delivering the new Sun Data Protection Data Erasure service will work with customers to ensure their erasure policies are compliant.

Now back to Thanksgiving: tomorrow we will give thanks for our baker's pie-making expertise while we erase those four pies.

The thing about SAM and Q is that their attributes have been required for the medical, military, and oil&gas industries for over a decade now, which is why they are so widely deployed in those market sectors. But the need to store and retrieve large volumes of data quickly and cost-effectively is no longer a requirement limited to those markets. Heck I've got a terabyte of data at home - think about what's going on in media & entertainment, manufacturing, financial services, education...

SAM-QFS was originally developed by LSC Inc, which was purchased by Sun in 2001. I had the opportunity to work closely with the SAM-Q team when they first joined Sun: back then there was Harriet (who has had a fascinating career in high tech), the Matthews brothers and the Intern, Bob, Ted, Tom, Harold, John, Margaret, Clay, Robert, Dave, ... who did I forget? I have lots of crazy Eagan Minnesota memories with the team - like the last slot in the soda machine, the oven at the side of the road, the bratwurst barbeques. The first time I went to Minnesota to meet with them - as I was pulling out of the airport -the Hertz guy said to me "Ya ready for the snow?" Two feet by the morning! Boy was it cold, and that was in the spring! And I have warmer memories of meeting with their customers - like Robert Cecil, PhD, Cleveland Clinic’s network director. Dr Bob gave us a great tour through radiology where SAM-Q was being used to show that a tumor was shrinking, through surgery where SAM-Q provides patient data right in the operating room, and through the data center with huge tape libraries, where SAM-Q was helping to increase the quality of patient care while decreasing costs. And I remember Dr Bob speaking on a panel at a storage conference - when asked about the importance of data availability, he quietly stated that access to data is the difference between life and death. No one can express the need for data availability and integrity better than Dr Bob.

Open sourcing SAM-Q is a key step for Sun and the developer community. It's now easy for people facing large data management challenges to try something that has worked for years in large scale, mission-critical deployments. And in case you're wondering how a business can make money while making such a key asset freely availably, remember that SAM-Q runs on servers, needs to be supported in product environments, stores data on disk and tape, ...

Most of my previous blogs have been about our Storage strategy at Sun. This past summer I moved into a similar role in Sun's Software group. What does that mean? Well even though I've got a new job, a new title, a new boss, and a new BU, it all feels pretty familiar.

While storage is all about data...keeping it, replicating it, archiving it, retrieving it, it's the software that provides the special sauce in managing data efficiently. Which is why when we talk about data integrity, scalability, and performance for unstructured data...ZFS is the answer. ZFS ROX. It must be true. There's even a license plate to prove it.

But what about structured data....how 'bout a database? How 'bout the world's most popular open source database? Last week, Sun announced we are acquiring MySQL. Rich Green blogged about how this is a good match on a company level - we're both big believers in open source, both have active contributing communities, both focused on the web economy. That's all good. Again, big changes, new stuff, but also somehow familiar.

But how do we match up on software? A good fit? ZFS and MySQL databases are a lot alike. Twins separated at birth? Well, customers use the same adjectives to describe the two: easy to use, reliable, flexible/scalable, great performance. Nice. Happily the differences between the software, and they are different, not twins but rather siblings who actually get along (does that happen?), complement each other: ZFS managing the storage of unstructured data and MySQL managing the use of structured data.

Kinda like my two teenagers - Danielle the structured just about taking over her college; Tara the unstructured defining her own home schooling plan... Absolutely love 'em both!

Cluster File Systems' use of ZFS as the disk filesystem inside Lustre is a perfect example of how open source can be used to advance the state of data management. Lustre is a distributed filesystem that can run across thousands of clustered server nodes, all sharing potentially petabytes of data. And like the challenges faced by anyone trying to manage large data sets these days, Lustre needed better scalability, reliability and storage management features than those available in their current internal disk filesystem. So the makers of Lustre had a few choices: start developing enhancements to their existing internal filesystem, build a new one, or pick one up from another open source community. Turns out to be an easy choice - ZFS is a 128 bit filesystem (more scale than any of us can use these days), has built in data integrity through its checksumming algorithms, and handles storage management internally via RAIDZ (no need to define RAID stripes and disk pools separately). Most requirements Lustre had and probably some they haven't even thought of yet are satisfied by ZFS. It's pretty cool that ZFS is solving out-of-this-world problems.

A friend emailed me this picture she took in the Palo Alto Fish Market parking lot of this weird green-glowing thing about to take off...

Click on image for more information.

Seems like ZFS might really be out-of-this-world... [yeah, I know, maybe her camera phone just doesn't take clear shots ]

But I wonder if George and Brad know just how awesome the SL8500 actually is. Do they know, for example, an SL8500 can hold a petabyte of data - about two hundred thousand copies of their Oceans Thirteen movie? Do they know that if the Bank Casino used 1000 cameras to gather their surveillance data and stored that data for 30 days, they would fill the tapes in an SL8500? And in interests of saving the planet, do they know tape is about 25 times less expensive to power and cool than disk because it uses that much less energy? All great news for the IT budget and the planet.

I spent some time this week with our Media and Entertainment sales team - to say the data in M&E is exploding is a complete understatement. One customer digitizing TV shows is expecting to have 50 petabytes of metadata to enable all the searches they need to handle - never mind the raw entertainment itself! And the M&E industry is heading full steam ahead into complete digitization, consumer mashups, affiliate communities... Data, data, and more data. No wonder why many cool web sites are using SL8500s to help store that data.

So, sure Brad and George were cast for the Ocean's movies because they're so hip, but our SL8500 certainly fit right in with the Ocean gang on their latest caper.

Let's set the background: First, we actually did grow up with respect to our storage portfolio (certainly a lot had to do w/ Sun's acquisition of STK). Check out the storage products we announced at SNW: our Low Cost Array, the library partitioning (ha – the only library on the market that can be shared between mainframes and open systems), our incredible file browser that lets you see across tiers of storage. New conversations are always easier when people believe you understand the old conversations. Second, the stage continues to change with exploding data growth (exabytes, exabytes, exabytes) and extensive retention periods (decades, decades, decades). When it comes to data, it's just not the same world it was last year, or the year before, or the year before that... And last but certainly not least, general purpose technology has caught up with the performance and reliability demands of storage - there's little need to build the world's storage from scratch anymore. Out with the custom ASICs and boards, out with the real-time embedded kernels. In with x64 servers, volume CPUs, Ethernet, and the general purpose operating system. So if you can build storage from general purpose hardware and software (check out the Thumper), what's next?

Open source, of course. Once you move from the proprietary to the general purpose, you can take advantage of all that world has to offer. Which is why on April 10th we announced an open source community for storage – because we're making the entire storage I/O stack within Solaris available via the opensolaris.org storage community. So customers and partners working to solve the world's toughest data management problems now have access to the source code and the developers of the world's best operating system.

Now that we've hatched our storage open source community, interest from storage analysts, press, customers, and partners is overwhelming. The ugly duckling has truly become the swan.

A few years ago I lost a wallet and spent about a week hunting down #s, dealing with nerves over what might show up on one of my accounts, and driving carefully so as not to get pulled over sans-license-on-hand. Wed night the only thing I was nervous about was the fact that I wasn't nervous about the wallet loss, only the game loss (all right, so we're overly emotional in Boston when it comes to the Sox).

I bonded with my online world that nite - those faceless people helping in my time of stolen-wallet/bad-baseball distress - geez, sounds like the theme of a crass TV commercial. But no TV for me, I'd rather go read some interesting blogs (like this one, or this one, or this one), see what's up in open storage land, IM my teenagers... just generally basking in the warmth of my digital world.

And talk about using the proverbial can opener on storage. We're opening up storage today with our new open source community for storage developers. You'll find filesystems, data services, drivers - all in an open source operating system that scales, is secure, highly available, and reliable.

Too much more to talk about now. I just heard those Opening Day fighter jets fly over our Burlington campus. While I'm checking out our opensolaris storage community, I gotta tune into MLB.com for updates on the Red Sox. BTW, ever check out how MLB.com actually feeds us all that real-time data?

Earlier this week I was with an account team discussing Sun's whole portfolio with a global bank. First the server guys pitched... I love listening to SEs talk with customers - each one has a unique and interesting twist on their area, with cool tools, interesting data [not to mention insightful opinions that we can wrap back into product strategy] and excellent technical explanations that get to the meat of a customer's decision. Oh, the delicate balance between chip frequency and memory speed leading into CMT was a delightful discussion! Then the software dudes took over - Java, Solaris, open source, more Java, more Solaris, more open source...

At 4:58 PM the software dudes contritely ceded the platform to me, leaving me just 2 minutes to explain our storage strategy. A few years ago I would have been in miserable trouble. But we are building storage from general purpose computers and general purpose operating systems - Sun's x64 and UltraSPARC systems and Solaris 10! And of course, that's what we had just spent the last few hours discussing with this customer.

Granted it took me a few more than 2 minutes to explain the changes in the storage industry that are driving us in this direction. Data is exploding - multiple exabytes of new data being created every year! And the tradition in the storage industry has been to build special purpose hardware and software to manage that data. This custom and proprietary strategy worked because the performance and reliability requirements of storage just couldn't be met with off the shelf computing. Not anymore! When we think about what we need in a storage OS - it's reliability, availability, security, performance, data integrity, ... When I listened to my software colleague talk yesterday about Solaris, it was all about reliability, availability, security, performance, data integrity, ... And a few more cool things thrown in for good measure, like observability (DTrace), virtualization (containers), "better than RAID6" software RAID (in ZFS) and so on. All with a developer community consisting of Sun Storage, Sun Software, and Sun Server engineers, along with our partners and the world participating in our open source efforts. Why would I want anything but Solaris to manage my data?

And what about storage controllers and their processors? We've got them coming out the you-know-what... Fast, cool, space-saving, really sweet storage controllers. We just happen to also call them servers.

People in every division of Sun are working on very interesting storage solutions, which makes it was easy to explain our storage strategy. Good thing, because I hate being the person between a hungry group of people and their dinner.

I'm sure Riley was trying to achieve a state of Zen-ness with his hard work at this garden... look at the dinosaurs intertwined with the soldiers... the piles of rocks... even a pen cap thrown in for good measure... (is that uncapped pen lurking in the cushions of my chair?) And while the contemporary artists among us might relax with the results, the more conventional like me were just exhausted by the time Riley and dad left for the day.

And how is this related to storage you might ask? This is exactly what happens when you take special purpose storage technology from the 90s, running single threaded operating systems on specially built boards, and you put it into today's world. How can you run your Web 2.0 application on your storage system? Once you move aside the dinosaurs and pen caps, you're left with nothing but a mess of firmware trying to handle sectors, slices, and spindles.

That's why we're building our storage systems today with general purpose hardware and software. That's why we think Thumper matters. Because our world today is different. We've got to rake the dinosaurs, pen caps, and soldiers out of storage and give today's developers a better way to manage their data.

Congrats to all the great engineers that made this happen over the past 10 years: Jim², John, Mark, Simon, Phil, Marcus, Jeff, Paul, George... who did I forget? And we're getting really good feedback from interesting customers building large web infrastructures. Developers that need to do new and interesting things to protect the volumes of data their customers are creating.

We're open sourcing more and more software all the time. Join our developer network to get in on the fun.

Check out the Big is Bad emblem on the tow-hitch - a perfect statement. But this big bad thumper isn't nearly as eco-responsible as our hybrid data storage server at Sun. Our x4500 data storage server holds 24 TB of data in a 4U of rack space, starting at $2/GB. A data storage server that can run any application - right on the storage system itself!

We're building new storage systems like this just because there's too much data in the world. Too much to just shove away on a block storage device and forget about. Too much to simply spool to a tape and ship offsite. Data today matters to more and more people over longer and longer periods of time. Data that has to be accessible when it's needed. You know what that means? It's not so much about how you store it - sure there are tons of options for disk and tape to meet your performance, reliability, scale, etc needs - its really about the applications you use to retrieve it for use again and again...

I did a double take. The guy driving did have a ponytail, but he definitely wasn't Jonathan...