Cluster File Systems' use of ZFS as the disk filesystem inside Lustre is a perfect example of how open source can be used to advance the state of data management. Lustre is a distributed filesystem that can run across thousands of clustered server nodes, all sharing potentially petabytes of data. And like the challenges faced by anyone trying to manage large data sets these days, Lustre needed better scalability, reliability and storage management features than those available in their current internal disk filesystem. So the makers of Lustre had a few choices: start developing enhancements to their existing internal filesystem, build a new one, or pick one up from another open source community. Turns out to be an easy choice - ZFS is a 128 bit filesystem (more scale than any of us can use these days), has built in data integrity through its checksumming algorithms, and handles storage management internally via RAIDZ (no need to define RAID stripes and disk pools separately). Most requirements Lustre had and probably some they haven't even thought of yet are satisfied by ZFS. It's pretty cool that ZFS is solving out-of-this-world problems.

A friend emailed me this picture she took in the Palo Alto Fish Market parking lot of this weird green-glowing thing about to take off...

Click on image for more information.

Seems like ZFS might really be out-of-this-world... [yeah, I know, maybe her camera phone just doesn't take clear shots ]

But I wonder if George and Brad know just how awesome the SL8500 actually is. Do they know, for example, an SL8500 can hold a petabyte of data - about two hundred thousand copies of their Oceans Thirteen movie? Do they know that if the Bank Casino used 1000 cameras to gather their surveillance data and stored that data for 30 days, they would fill the tapes in an SL8500? And in interests of saving the planet, do they know tape is about 25 times less expensive to power and cool than disk because it uses that much less energy? All great news for the IT budget and the planet.

I spent some time this week with our Media and Entertainment sales team - to say the data in M&E is exploding is a complete understatement. One customer digitizing TV shows is expecting to have 50 petabytes of metadata to enable all the searches they need to handle - never mind the raw entertainment itself! And the M&E industry is heading full steam ahead into complete digitization, consumer mashups, affiliate communities... Data, data, and more data. No wonder why many cool web sites are using SL8500s to help store that data.

So, sure Brad and George were cast for the Ocean's movies because they're so hip, but our SL8500 certainly fit right in with the Ocean gang on their latest caper.