Tuesday November 20, 2007

Peter Braam, founder of Cluster File Systems Inc. and now VP of Lustre at Sun, gave an update on Lustre to the 150+ Sun HPC customers who attended the HPC Consortium meeting in Reno prior to the Supercomputing '07 conference.

He spoke briefly about Lustre's place in the HPC market, citing example of its use in the TOP500. Subsequent to his talk we learned that the most current list (which was released at Supercomputing) shows that Lustre is used on 7 of the 10 largest supercomputers in the world. While it is used at the very high end, Lustre also has a strong presence in Oil & Gas, in digital animation, in EDA (electronic design automation), and at several large ISPs to name a few areas.

The current Lustre release is 1.6, which has undergone some major usability improvements from 1.4, which is still in use by some customers today. Version 1.8 is targeted for the 2nd quarter of 2008 and will include support for ZFS and Solaris on the storage server. Version 2.0 is scheduled for the 4th quarter of 2008 and will add a clustered metadata capability and server network striping. These are the plans. Insert standard caveats about engineering plans here.

Peter talked about the post-acquistion integration into Sun and described it as smooth. While it was disruptive to some (notably, Peter himself) the team has continued largely structured in the same way. I do know that some Sun managers and engineers have joined the Lustre team, which I think is a great way to help the Lustre team continue to transition into Sun. We've also paired Lustre engineers and managers with old Sun hands as mentors to help easy the transition. It helps, I'm sure, that some of these mentors themselves came into Sun from small companies through acquisition. From all I've heard, the integration seems to be going quite well.

In terms of the business ramifications of the acquisition, continuity is the theme. Still open source, still the same model for customer support, and we will continue business with Lustre's various OEMs. And, of course, Linux continues to be a focus while we also work to expand Solaris support.

So, what's up with ZFS and Lustre? Lustre servers today are built on the Linux ext3 local filesystem and CFS was able to achieve extreme performance with it. Version 1.8 will add support for ZFS with the intent of hardening Lustre and driving for even higher levels of scalability. The servers will be in user space using user space ZFS code and there will be server migration tools available for those customers wishing to migrate from an ext3-based server to one based on ZFS.

Version 1.8 will also see the additional of a network request scheduler to improve I/O scheduling, based on work done at Oak Ridge National Laboratory (ORNL), a Lustre Center of Excellence, on Jaguar, their 8000-client HPC cluster.

One funny point Peter made: Sun is back in Phase III of the DARPA HPCS program. Recall (perhaps) that Sun was not selected to proceed from Phase II to Phase III--but CFS was to supply the file system for Cray's solution, and so Sun is back. :-) As part of our involvement in HPCS Phase III, there will be significant future enhancements to Lustre to support some fairly daunting requirements on file creation rates, client bandwidths, and extremely large file counts. All good news for the HPC community at large.