NPIV for Opensolaris is delivered

NPIV (N Port ID Virtualization) for Opensolaris is delivered. NPIV allows one FibreChannel physical port to support mulitple virtual ports, and is especially useful for virtualized environments, such as Xen.

We actually delivered the core kernel changes a couple builds ago. This allows you to create and delete NPIV ports using the new fcadm command.

But, what's really useful is combining that with code with just put back to support xVM to build 97.  With xVM, administrators will be able to associate virtual ports with a specific DOMU. Disk devices that are visible (via LUN Masking and switch zoning) on that port will appear in the DOMU.

You'll need 4Gb/S (or faster) QLogic or Emulex HBAs, and the latest firmware on your 4Gb/S or faster switches.  Targets (for example storage arrays) don't need to do anything special to support NPIV.

There's an except from the to be updated SAN configuration guide on the Opensolaris NPIV project page that explains how to use the new kernel functionality.

NPIV Beta release

Our team released a beta of NPIV support for Solaris. See previous posts for what NPIV can do for you. This release contains the device driver changes and a utility to add and delete ports. Future releases will contain integration with xVM.

SNIC SDC NPIV video

This morning I posted slides that I gave at my talk, I just found that the video is up now. 

SNIA Developer Conference Report

Here's a link to the presentation I did at SNIA SDC about NPIV. The conference was well done; I reconnected with some old friends and heard some good talks.

The list of all talks is here. I didn't attend a lot of the talks because I had meetings at Sun's office in Menlo Park, but I did like "Storage on the Lunatic Fringe," about the next generation of storage devices. I also liked the ZFS presentation, explaining how new technology trends (fast CPUs with lots of cores) make ZFS attractive, even for doing RAID.  And, I enjoyed hearing Richard Stallman, because he's been such a big influence on the industry.


 

SNIA Developer Conference

I'll be at SNIA Storage Developer Conference next week (September 10-13) talking about NPIV and the work we're doing in Solaris. If you're there, come stop by.

More NPIV Reading

I found a couple more links that I like about NPIV. The first is from IBM Journal of Research and Development. IBM did the initial development of NPIV several years ago. One reason I like this article is that it actually talks about the engineering trade offs they went through in developing NPIV. I've read a lot of specifications in my career, but it's rare to see the decisions behind the specification. Here's the link.

The second is from the team at Solution Technology. They provide a solid explanation of how it all works at the protocol layer. The link is here. 

NPIV and Solaris FibreChannel

What do I do all day at the office? Lately, I've been working on adding NPort ID Virtualization (NPIV) to our Leadville FibreChannel stack.

At a high level, you can think of NPIV as allowing one physical FibreChannel HBA to log in multiple times to the SAN, and so you have many virtual HBAs.

Why is this interesting?

The first thing I thought of when I heard about this is hypervisor applications, like Xen. If you have one world wide name per DOMU (in Xen terminology), you can do the same zoning/lun masking that you've always done per server, but this times it's per DOMU.

Another use is that if your HBA breaks, and you have to replace it, you can use the old WWN on the new HBA, and you won't have to rezone your SAN.

Storage Magazine has some more good background reading here on other possible uses.

I'm looking at how this will work in the existing Solaris FibreChannel stack, and Solaris Xen implementation (although it's not dependent on Xen).

Watch this space for more details in the future.

Sun Developer Network and Storage

My department wants to explain why Solaris is a good platform for doing storage development. We'll be regularly publishing articles on Sun Developer Network. The first covers why you'd want to use the FibreChannel drivers built into Solaris, the second talks about some of the many storage standards that Solaris implements. I worked on the first one with my colleague Reed Liu (who I was just reminded is actually Dr. Reed Liu, although he is so down to earth I forget), and wrote the second with help from many colleagues.

Solaris FibreChannel Stack

On opensolaris-storage, there have been some discussions about the Solaris FibreChannel stack. Here's a pretty good picture of how it fits together.  From the top:

1. Fibre Channel HBA-API and Multi Path API are industry standard ways to communicate with storage software. luxadm and cfgadm are utilities to control and report status on storage.
2. sd/st/ses are SCSI target drivers.  Target drivers in Solaris are a driver to control a particular type of SCSI peripheral. We'll talk about IP/ARP and the other networking portions later.
   
3. SCSA is thin layer that connects HBAs and target drivers
4. MPXIO is the Solaris multipathing solution. It allows for several physical paths to one device, for fault tolerance and higher performance.
5. FCP is the SCSI to FibreChannel layer. FCSM is a small device driver for management applications. FCIP is network device driver, which allows IP over FibreChannel
   
6. FC Transport is the fctl and fp device drivers.  This keeps track of which ports are where. It also generally does the FP-2 protocol, handling device discovery and changes in the SAN.
   
7. At the bottom, we have each vendor's specific library, to run their particular HBA.
8. On the right is FibreChannel doing networking.  FibreChannel was designed as a general purpose transport. As well as SCSI, it can carry TCP/IP networking protocol, and Solaris supports IP over FibreChannel.
   
9. On the bottom left are the FibreChannel drivers that run without the Solaris FibreChannel stack.