Random footnotes of Engineering

Reminder the Open Storage Summit is on this weekend, Sunday September 21.

I guess it never really occurred to me to provide an outline of what I see Open Storage as being. Clearly this is an incredible oversight with my new role as "Open Storage Software Community Guy" ;-) So I'll attempt to outline below what I see Open Storage as being.

As you'd expect Jonathan has actually already beaten me to the blogging punch. However, I will endeavour to provide some additional value to the subject.

So what is Open Storage? It's an effort here at Sun to provide storage technology through open source, specifically through OpenSolaris. Essentially:

• Open Standards + Open Source = Open Storage
  

This area is ripe for disruption. There are many proprietary solutions in this area, but the technology in the OpenSolaris storage stack is compelling. We're seeing examples of the following:

• Explosive data growth
  
• Increasing CPU power
  
• Parallelising of CPUs
  
• Ever increasing data access rates
  

ZFS is a piece of technology that allows us to take advantage of commodity hardware. It provides the reliable backing store. Among many features it provides guarantees about ondisk consistency (Note it's not the only filesystem that does this). This provides a strong building block for the OpenSolaris Open Storage stack. The stack looks like:

As you can see there are a huge number of components in Open Storage. We provide all sorts of initiators and significant ability to function as a target through COMSTar. There are many different filesystems too. This all provides significant flexibility in using Open Storage to solve the problem the way _you_ want to solve it. You are not constrained by the box that Sun or anyone else sells. If OpenSolaris runs on it you can use it.

It is the ability to provide storage technology on any equipment that OpenSolaris runs that is disruptive. It allows the flexibility to choose the hardware to match the task at hand and scale that as necessary.

So in my view the above comprises what is Open Storage and highlights some of the problems it allows us to solve. Moving ahead I'll dive into more details of some of these components. I'll also blog about random other things to keep everyone on their toes too ;-)

I'd been working on this blog entry for a while, but it seems that Jonathan beat me to the publish button. He is covering something more XvM specific, I've included what I was working on below just for some comparison...

We hear a lot about "Cloud computing" all over the place. Often it's used to refer to to web 2.0 type resources that are out there. Things like Amazon S3 or Google's multitude of applications. What I'd like to discuss today is some thoughts I've got around how a storage "cloud" would look like inside a data center. What are the interesting pieces to the problem?

• Horizontal Scalability
  
• Managability
  
• Reliability
  

This is likely to be a gross oversimplification, but heh, that's never stopped me before! A big requirement I've heard from a number of different places is that people want to store their data, they don't particularly care where, and just know when they come back to ask for it that it will be there. A key consideration for these folks is that the storage system is easy to manage, the fact that they don't care about anything other than if their bits are there implies that they would like as close to no required management as possible. The fact that they want their bits to be there when they come back to ask for them implies data reliability to me. It also seems implicit that as the storage requirements grow the horizontal scalability needs become drastically apparent.

So why do I raise any of this since it seems incredibly obvious? I think that we need to think about the way we approach building a 'Storage Cloud' within our customers data centre. There will be a lot of folks who aren't prepared to trust a cloud out there on the internet somewhere. If we can come up with solutions that create these clouds in the data centre we'll be onto a winner.

Yesterday I gave a presentation at Community One on Storage. Sun has just recently announced it's OpenStorage Initiative. Part of that technology is a project called COMSTAR. The presentation I did talked through the way to create a multiprotocol storage device. This used ZFS to export things using NFS and CIFS, but to also use COMSTAR to export over fiber channel. In the future COMSTAR will support iSCSI, iSER, FCoE and SAS. It is a really unique technology that will challenge the current market of disk arrays.

Update: you can see a video of this presentation.