SUMIT GUPTA's BLOG

Here is a demo on how to take a machine running Solaris and convert it into a Fibre Channel Array using COMSTAR and ZFS. The demo covers all the steps starting right after a fresh install of the OS to seeing the FC LUNs from different initiators (Solaris, Windows and Linux) connected to the Solaris Host acting as a FC storage array. But still this is around basics. There are some concepts like LUN mapping and masking and using some advanced zfs concepts which are not covered in this demo. I hope to add more content in future which will cover those aspects.

 This Demo is also posted to the Videos section of the COMSTAR project page.

COMSTAR + ZFS Demo

Basic steps to setting up a Solaris host as storage array using COMSTAR and ZFS

Last week (5/5/08) I did the putback of final COMSTAR bits into Network Storage Consolidation for release into Solaris Nevada B90. The entire team was very excited to have made the target as it was not easy. Too many things to cover before integration. The following features have been delivered into build 90.

• SCSI target mode framework (stmf driver and stmf service).
• Management library for stmf, libstmf and the related headers.
• 2 CLIs, stmfadm and sbdadm
• Common Fibre Channel port provider (fct) and Qlogic driver for target mode (qlt)

The above is same as what was already being deliverd as a part of opensolaris package SUNWstmf on the COMSTAR project page. However there are some features in the opensolaris package which are still being worked for Solaris integration (and hence are not included in this B90 integration). These are:

• MDBmacros.
• SAS target support.

For all of us in the COMSTAR team this is really a beginning of a new roadmap of exciting features. With a multiprotocol SCSI target mode framework, the list of things which can now be done is pretty long and is growing every day. I hope to post more on Storage Discuss and in this blog as we make further strides.

(To download all the patches listed below go to SunSolve).

Until very recently there used to be a single patch for all the FC drivers (excluding Emulex) on solaris 10. This patch was 119130 for sparc and 119131 for x86. Emulex always had a different patch which was 120222 for sparc and 120223 for x86.

That changed recently and the change was triggered by some open source activity by HBA vendors. It changed the patch numbers and also it broke the one big patch into 2 patches with a small caveat. Now if you were downloading and installing 119130 (or 119131), to get the latest driver, you will now do the following:

1. Install 119130-33 for sparc and 119131-33 for x86 platforms. This is the last rev of this patch and is needed for anything below Solaris 10 Update 4 (07/07).
2. Install the latest rev of 125184 for sparc or 125185 for x86.
3. Install the latest rev of 125166 for sparc or 125167 for x86. This is for the Qlogic driver (qlc). Or if you dont need qlc i.e. you are only using Emulex cards, install the latest rev of 120222 (sparc) or 120223 (x86).

So basically the Qlogic driver is no longer part of the same patch as the FC framework drivers but has its own patch now. Also starting with Solaris 10 Update 4 (07/07) there wont be any need to install 119130-33 as it will be a part of the OS.

Here is a list of all the patches used by SAN Foundation. Thanks to Charles Baker for compiling that list. 

No matter how hard you try to make a software smart and adaptive, there is no replacement for the human element. In the end an administrator can come and tune a few things and set a few variables in a bunch of conf files and do something in 15 minutes which the developer was trying to automate for the whole past year.

While it has obvious benefits, Tuning SANs can be very dangerous specially if the limits and impact of tuning is not well understood. Thats where folks here in SUN have done a really good job of putting together years of experience in the form of a document which describes what all things can be changed and what all are the possible values. The tunable document can be found at:

Tunables for the Sun StorEdge SAN Foundation Suite: Optimizing for Performance and Failover

I recently had the opportunity to directly take questions on SUN's Fibre Channel Solution from customers through BigAdmin . Needless to say it was very insightful because there are things everyday developers assume to be well known but they are not. One of the questions was

What are the actual limits for "LUNs per Target", "Number of Targets" and "Number of MPxIO nodes" supported by Leadville in conjunction with emlxs and qlc drivers?

And I replied, assuming, people know that these limits are on a per HBA port basis and not system wide. I received a few follow ups E-mails asking for the same information. And while I am trying to fix the answer on BigAdmin, I thought it will be a good idea to have that data in my blog. So here it is again:

 

• LUN limits on a per HBA port basis.
  • Qlogic: 16K per target
  • Emulex: No limit based on LUNs
• Target limits:
  • Emulex: 507 targets per HBA Port
  • QLogic:
    • 8 targets for the low-cost HBA (QLA210 aka Prism)
    • 256 for the 2G HBAs
    • 2048 for the 4G HBAs

In future when NPIV will go in i.e. customers can buy a HBA and create many virtual ports per physical HBA port, the LUN limits will be on a per virtual port basis.

And in case you are wondering about the BigAdmin session, the link can be found here.