Half of World's Fastest Supercomputers Running Lustre
It has been a whirlwind few days and I am just getting around to talking about the SC07 show being held this week in Reno, Nevada. While today's announcements at Oracle Open World kept me away from the show itself, I did spend the weekend in Reno at Sun's HPC Consortium user group meeting. We had a record number of Sun HPC customers attend, including more commercial customers than ever from financial, energy, and manufacturing industries. While the often quoted
Top500 list doesn't list file systems used, we did a bit of our own research and found that at least 5 of the top 10 and at least 14 of the top 30 supercomputers in the world are running Sun's Lustre cluster file system. That easily represents many petabytes of data being stored on Lustre.
At the HPC Consortium, ex CFS CEO and new Sun employee Peter Braam shared his plans for continuing Lustre's development. The Solaris version of Lustre's Object Storage Server is now in beta with general availability planned for the first half of 2008. That will follow later in the year with the Solaris client. Users of the current Linux version of Lustre won't be left out either, as all versions of Lustre will soon use a ZFS disk format to store their data. Sun's new HPC Community Site has more info on what's going on in Reno as does Josh Simons very complete blogging on most of the conference sessions.
Hello,
one thing that would be nice is to see how lustre could be used in other places than hpc. With qfs, lustre, and soon pnfs, there are quite a few possibilities, but everything is very confusing, and except in hpc, people usually end up with a simple nfs server because that is all we are able to understand and configure easily...
There is also one situation I was shown recently, and I don't know if it fits any of these products. There are say a hundred workstations, all have a large disk and use only a small part for the system. The home directories are on a nfs server. Now we would like to use the rest of these disks, but any machine is likely to disappear anytime (crashed, broken, upgrade) so whenever we store anything, we store in on three different machines. Now what would be great is a system that shares all the disks of all the workstations and automatically handles enough redundancy to make this useful.
And I guess looking at p2p like techniques for storage is more a research topic.
Posted by Marc on November 15, 2007 at 06:01 AM PST #