by Mark A. Carlson

Perhaps you have an application that generates data that doesn't need to change once it is created. Likely that data will need to be retained for some period of time, and will also need to be deleted at some point. This data could be medical records, media such as images, video or audio, engineering documents, and so on. An important concept associated with this data is the metadata - or data about this data - that also needs to be retained. For example, the metadata for medical records could include patient info, type (x-ray, CAT scan, etc.), doctor, radiologist, disease diagnosis, and so forth. The traditional way to store the data and metadata might be to put the metadata in a database and the actual data in a file, but then how do you ensure that  the same retention policy is applied to the data stored in these two disparate storage mechanisms? Note that retention policies can be influenced by regulation (Sarbanes-Oxley), corporate policy, and even legal action.

So this is where Fixed-Content Aware Storage comes in. The idea is that the storage device itself implements the retention policies for both the stored data and the stored metadata. You can specify for each piece of information how long it must stick around, whether it should be absolutely (positively) deleted after that time (to prevent legal subpoena), and whether there is a hold (too late, already subpoenaed) on the deletion. The storage device can also allow you to use the metadata to find the data later by performing searches and queries. There are several different manufacturers of these types of devices. The Sun StorageTek 5800 System (perhaps better known as Project Honeycomb) is one such device.

So if you have such an application that would like to use this type of system, how does that work? Well, you need to save your data to the device, and that will involve some level of unique coding for each device since there is no standard API as yet. Each storage system vendor has an API that you will need to use to save data to their system. The STK 5800 API is here, and you can find out more about the API on the Open Solaris Project's page with the description.

So, OK, now you're thinking: "What? I have to code to N number of APIs for N different storage system vendors?" This is where the standard comes into play. The standard is called XAM (a retronym for eXtensible Access Method). This is the industry's proposed solution to the multi-vendor interoperability problem I just outlined above. Sun is actively involved in the standards effort, which includes not just the XAM Specification itself, but also the development of a XAM SDK that can be used by application developers to start coding to the new standard, and storage vendors to start implementing to it. Of course standards take time, but it is interesting to see the progress on this so far. In the meantime, you can develop your application to one of the vendors who has an API. Be sure to ask about open standards and open source when you contact that vendor so that you don't get locked in.

be sure to drop by Mark's blog, Management Monogatari