So you want to build a storage server...

Wednesday Jun 18, 2008

You have lots of company. Storage Servers have been around since the days when Sun first put NFS on their workstations. Administrators would load up one workstation with disks and share it with the others. Even today, a large percentage of our "servers" end up as storage servers when deployed.

The acquisition rules for some companies actually favor building a storage server over the purchasing of a separate appliance. The server budget is typically separate from the storage budget for most organizations. Administration is more familiar as well. Administering a storage server is little different from managing any other server, save for the additional commands required to expose the internal storage to a separate host.

So when building a storage server, you want to understand your requirements of course. File or Block protocols? Ethernet or Fibre Channel or even SAS to the hosts? How will you protect the data: snapshot, backup, remote replication? How much CPU power do you need? How much storage space to match that? All of these factor into which server you should buy or repurpose for this deployment and which OS you install.

Sun does have servers that make a good choice for a storage server. Of course the classic example is the 4500 "Thumper" storage server. Customers have been buying these from us in droves, which is the best proof of the value of this notion. With the OpenSolaris storage stack they have been able to make them into NFS servers, CIFS server and even iSCSI arrays. Now Sun has released software that will take storage servers to the next level: Multi-protocol storage arrays.

The new software, part of the latest build of OpenSolaris, is affectionately called COMSTAR and boy is it the STAR of the storage COMmunication you want to be able to do with a storage server. Fibre Channel, SAS, iSCSI, iSER, OSD, you-name-it, if it's a storage communication protocol, COMSTAR handles it.

Now it may seem silly to think about putting a bunch of SAS/SATA drives in a box with a CPU and then just re-exporting them as SAS targets to hosts. What value does this have? Of course it's what you do in that storage server before you re-export them that make this a storage server and not just an enclosure. You want mirroring, disc scrubbing and automatic failover and repair - load up ZFS. You want remote replication - load up Availability Suite. You want to be able to diagnose bottlenecks and performance issues - load up DTrace. Then export the resulting highly available highly performant virtualized disks.

I think our current stars: ZFS and DTrace, should make room for the new star. Let's hear it for ZFS, DTrace and COMSTAR. Hooray!

XAM Java Binding Overview

Monday Jun 16, 2008

This article was originaly posted to the Royal Jelly blog: http://weblogs.java.net/blog/macsun/ The eXtensible Access Method (XAM) is new kind of storage interface for storage devices that can handle a special type of data, called "Fixed Content". Version 1.0 is soon to be released, but there is a working draft available of the Architecture Specification, the C Language Binding and the Java Language Binding. It is this last document that this post covers. A good introduction to XAM is the beginning chapters of the Architecture specification, and that document covers the normative behavior required for implementations of the interface.

The primary Java Object that applications deal with to store and retrieve data is called an Xset. It is the object that is the container for both the data as well as the associated metadata. XSet metadata goes beyond the typical filesystem metadata, and adds fields for retention and other data management policies. Applications can also add their own metadata as well. This is a key requirement in the fixed content storage market. Any of the metadata in the XSet can be queried for later, in order to find the XSet.

But before we dig into the XSet, let's talk about the other primary objects of the XAM Interface. First is the notion of a system that will contain the XSets. We call this storage system an XSystem in the abstract. The XSystem object is also the factory for XSet objects (via create and open methods). Now any given host may have access to any number of XSystem storage devices, so we need a way to create XSystem objects for each system we need to access. We do this through an object called the XAMLibrary. The XAM Library manages all the connections to the storage devices and acts as a factory for XSystem objects. It does this through the connect() method:

public XSystem connect(String XRI) 

The XRI (XSystem Resource Identifier) string is a "handle" that is used to load the correct Vendor Interface Module (VIM) for the device and locate it, perhaps on the network. It has the following syntax:

"snia-xam://" [vimname] xsystemname [params]

The XSystem object that is returned from the connect() operation can now be used to talk to the storage device. The first thing we will need to do is to authenticate with the device, to prove that we are who we say we are. This is done through the authenticate() method:

public byte[] authenticate(byte[] buffer) 

See the Architecture Specification for a more detailed description of the steps required to authenticate to an XSystem.

So we have the three primary objects of the interface: XAMLibrary, XSystem and XSet. All of these objects have properties which are accessed in the same way. These properties can be of several different types and can be created at runtime. Thus we need a way to do this in a type safe manner. The key to this capability is something called an XIterator:

import java.util.Iterator;
public interface XIterator extends Iterator 
{ 
    public void close() throws AuthenticationExpiredException, XAMException; 
};

Many of the basic operations on the fields these primary objects contain are in common, so the XAM interface defines a superclass for the primary objects called FieldContainer:

public interface FieldContainer 
{
 XIterator openFieldIterator(String prefix) 
	throws AuthenticationExpiredException, InvalidFieldNameException, 
       ObjectInUseException, XSetAbandonException, 
       XSetCorruptException, XSystemAbandonException, 
       XSystemCorruptException, XAMException; 
...
}

The XIterator returned from openFieldIterator() can contain all properties, or just a subset (those having a name starting with "prefix"). The FieldContainer Java interface also contains methods for: field interrogation; property creation, accessors, and mutators; field operations, and XStream operations. An XStream is an object that can contain the actual content associated with an XSet. XStreams have operations to read and write the content using various means both synchronously and asynchronously. The type of data an XStream holds can be any valid Mime type.

This post covers most of the basic objects that are part of the XAM interfaces. Future posts will delve into specific use cases.

Preservation and Archiving

Thursday Jun 05, 2008

It was an exciting week for Preservation and Archiving activities.

First, I was introduced to several key academic researchers (and software developers) in the digital repository space who had come early to get the inside scoop on the latest ST 5800 (Honeycomb) developments. Dave Tarrant (University of Southampton), Ben O’Steen (University of Oxford), and Neil Jeffries, R&D Project Manager, Oxford University Libraries met with us in the Honeycomb development facility in San Francisco. I have discussed Open Archives before, but these folks are working on some great software that starts to fulfill that promise. They are also customers of the ST 5800 and are thrilled that we just Open Sourced the latest (1.1.1) product code and can contribute some of their ideas to that code base in the Honeycomb community.

Which brings us to the event this last week that I attended. Sun sponsors a regular set of meetings of what is called the Preservation and Archiving Special Interest Group (PASIG). The latest event was held last week in San Francisco and boasted attendance of over 130 non-Sun attendees from across the industry. There were lots of great talks given, and I highlight a few below:

Neil Jefferies talked about the Oxford Digital Asset Management System (slides). They use FEDORA-Commons in combination with the ST 5800 and other open source projects. They are ingesting the output of the Google Library Project into DAMS, something they call Mass Digitisation Ingest Components (MDICS).

David Tarrant talked about his work with EPrints at the University of Southampton (slides). His emphasis is on supporting small science as well as big science projects. He also talked about some of the requirements for long term storage. He has combined the open source EPrints repository software with the ST 5800, utilizing the self managed storage to extends the value of the repository. EPrints has an extensible plug-in architecture that allows for any storage device or database to be plugged in. He also discussed the Preserv interoperabilty project that is working towards standards that separate the reository software from the storage controller module and from the actual physical storage.

Ray Clarke gave a great talk (slides) on some of the best practices for long term data retention. He also gave a talk (slides) on some of the activities at the SNIA in this area including XAM, the Data Management Forum (DMF) and the new Long Term Retention Technical Work Group (LTR TWG). SNIA is working on a format for encapsulating data along with its metadata that will allow it to preserved in a storage system independant manner over its lifetime. Most of the attendees of the conference expect to retain and preserve their data essentially forever.

There were lots of other great talks which you can find here. Many thanks to Art Pasquinelli (and many others) who did a bang up job as usual organizing the event. The next meeting is tentatively scheduled for November in Europe. If you want to join the Sun PASIG community, join here.

Don Deel: Newest SNIA TC member

Sunday May 18, 2008

I first met Don in the early days of the Bluefin effort when we were both helping start what would eventually become the SMI-S standard. He had his own SAN management software startup called Prisa Networks with a pretty comprehensive Java based framework for management. At the time, I was trying to convince him to adopt Jiro, as I recall.

Now Don has been appointed to fill the seat recently vacated by David Black on the SNIA Technical Council. Most recently Don and I were co-chairs for the Management Frameworks TWG, which then produced a reference architecture.

I recently interviewed Don for the Technical Council Podcast series. You can subscribe (iTunes) or just download the .mp3.

Welcome Don! I know you will make some valuable contributions in your new role.

Microsoft Manages Solaris!

Wednesday Apr 30, 2008

Wow! This is big. Microsoft just announced that they will be able to manage Solaris 10 systems from their Systems Center product. This is in addition to support for various Linux systems as well.

This cross platform support uses the latest management standards from the DMTF, including WS-Management and CIM. Microsoft actually installs a CIMOM on the OS to be managed. This common infrastructure is used to host the Microsoft created CIM provider that instruments the operating system.

The CIMOM that is used is the Open Pegasus CIMOM from The Open Group . As part of the announcement, Microsoft pledged to work with the Open Pegasus community to put back their changes. They will also take part in the steering committee. This is great news, and I want to welcome them to the open source community. The folks that are participating even have their own blog.

XAM 1.0 major milestone

Monday Apr 07, 2008

The eXtensible Access Method (XAM) 1.0 specifications are now available as work in progress drafts on the SNIA web site. This includes the XAM Architecture spec (details all the semantics of the XAM interface), the XAM Java API (Java language binding) and the XAM C API (C language binding).

SNIA also issued a press release announcing this and the code donations by Sun, EMC and HP to the XAM SDK effort. The XAM SDK is a toolkit and source code that implements XAM and will be made available after XAM 1.0 finishes the SNIA Architecture vote by it's members later this year.

I am here at Storage Networking World in Orlando this week to give a XAM tutorial and help with a multi-vendor XAM demonstration in the Tech Edge area of the show floor. Stop by and say hi if you are attending.

Also, EMC has announced that they will be using the XAM SDK in their own developer platform. This is great news and follows on Sun's own announcement last month.

DMTF and SNIA sponsor Alliance Symposium, Green Summit

Friday Mar 21, 2008

Next week in San Jose, the Alliance Partner Technical Symposium will held at the Dolce Hayes Mansion from Tuesday 25th of March through the 28th of March 2008.

This year marks the first Green Summit that will be part of the agenda. The Green Summit is the first technical summit of the multiple organizations collaborating to manage the power and cooling aspects of data center equipment through standard models and interfaces. This day-long summit will include presentations from each group and include time for creative discussions and organization of further collaboration. The participants will include the Storage Networking Industry Association (SNIA), the Distributed Management Task Force (DMTF) and The Green Grid (TGG).

On of the hats I wear at the DMTF is the Vice President of Alliances, so I am very excited at the broad participation we are getting for this conference. It should be a week of great cooperation and information sharing between all the groups. Thursday is dedicated to presentations from all the alliance partners including the Open Grid Forum, the Trusted Computing Group, the Printer Working Group and the Blade Systems Alliance.

If you are interested in attending, registration is still open and there are even hotel rooms available.

Open Archive

Thursday Feb 28, 2008

\ˈō-pən\ \ˈär-ˌkīv\

What does it mean to keep data around for generations to come? It's an interesting problem. Were the Egyptians thinking about this when they created their documents on Papyrus? When that data becomes digital, the challenges seem daunting. Changing technology soon obsoletes the data that we have created even within a decade. Applications change the format of the documents that they create. If you had a VisiCalc spreadsheet with important information in it, how would you read it today without the VisiCalc application, the operating system and perhaps even the hardware that supports that OS? What if that spreadsheet was on a 5 1/4 inch floppy?

The idea of an Open Archive means that we can think about what it means to keep data around for generations and make it accessible by the technology that will exist 100 years from now. The elements that I think will be necessary to consider include:

• An Open Document Format
• Open Application Interfaces to the repositories
• Open Export and Import of Data between repositories
• Open architecture for data services that will keep data accessible
• Open Source implementations of the above elements

It is the combination of all these elements that makes up an Open Archive in my mind. Many of these elements are already being addressed by industry efforts, and of course Sun is an active participant in these efforts.

Open Document Formats are being addressed by standardization efforts such as the OASIS Open Document Format for Office Applications standard. OpenOffice.org has an open source implementation of this standard that Sun integrates into it's StarOffice suite. Of course we need to go beyond just office documents over time.

Open Application Interfaces is an emerging area that is being addressed by the Storage Networking Industry Association (SNIA) with it's eXtensible Access Method (XAM) standard. I have blogged about this in the past, but the standard is now nearing completion. What's new (announced here) is that Sun has donated source code to the SNIA to allow it to create a reference implementation of the XAM API. This will be software that implements the XAM standard and uses a standard filesystem as the repository. This will be released by SNIA as an SDK for XAM once the standard and the software work is completed.

Open Export and Import is needed in order to move your data and it's associated metadata from storage system to storage system over the life of that data. The XAM Standard also addresses this with an XML based format for this purpose. This allows you to not only be agnostic to vendors, but the changing technology over the years.

An Open Architecture for Data Services allows those data services to periodically read data in old formats, convert them to newer versions of standard formats, and then write them out via newer, standard interfaces. This is an essential element of an active archive that can keep data accessible without having to archive the machines, operating systems and application version that you used to create the data. Sun has donated source from it's StorageTek 5800 to a new project on Java.net called Royal Jelly. We are working here in an open community to create the architecture for the data services that will implement this vision of an active archive.

I have mentioned some Open Source Implementations above, but I want to also highlight Open Solaris itself as community where this work is going on. We also posted the source to Project HoneyComb on it's Project Page. We envision adding support for the standard XAM interface to Open Solaris with this code and that being developed in the other two communities (SNIA and Royal Jelly) mentioned above. In addition to supporting application access to fixed content storage systems attached to Open Solaris based host machines, we expect Open Solaris as a Storage Platform to provide XAM capability natively.

If you care about the issue of keeping data around for generations to come, if you care about openness as the way to achieve this, you should be asking potential archive vendors some pointed questions. If you really care about the evolution of open archives, please consider participating in one or more of the communities mentioned here.

Storage Developer Conference 2008

Wednesday Feb 27, 2008

The 2007 Storage Developer Conference, put on by the SNIA, was full of great content for developers. As evidence of how Open Solaris is getting traction as a storage platform, many of the talks last year came from Sun developers.

The talks which Sun Developers gave (now with presentations) include:

• Lori Alt on ZFS as a Root File System [Slides] [Video]
• Spencer Shepler on NFSV4.1 [Slides], Filebench [Slides] and pNFS [Slides] [Video]
• Rob Thurlow on NFS Namespace Futures [Slides] [Video]
• Sumit Gupta on Fibre Channel Target for Solaris [Slides] [Video]
• Aaron Dailey on Xen and Storage [Slides] [Video]
• John Forte on the iSCSI Management API [Slides] [Video]
• Jim Dunham on SAN Target Mode Boot Server [Slides] [Video]
• Ken Davis and Sajid Zia on Open Source iSCSI [Slides]
• Hyon Kim on Multipath API for SCSI Devices [Slides] [Video]
  And of course my own presentations:
• Management Framework Reference Architecture [Slides] [Video]
• eXtensible Access Method Software Development Kit [Slides] [Video]

Update: The 2008 agenda has been uploaded: http://www.snia.org/events/storage-developer2008/agenda2008

Highlights include: Andy Bechtolsheim is giving a keynote on Tuesday.

Jeff Bonwick and Bill Moore have updated their very popular ZFS Tutorial for Monday.

As usual, we have had many storage developers from the OpenSolaris Storage community step up to do presentations at this years conference again.