SPARC Enterprise M-class Servers The Secrets of Olympus

To make service a bit easier, each physical disk bay in the Sun SPARC-Enterprise M-class line of servers has two LEDs: one for power, the other for fault indications. We used the fault LED in blinking mode to act as an indicator so the system administrator can identify and locate a specific disk.

The disk LEDs are accessed using cfgadm(1M). The cfgadm PCI plugin always supported LED manipulation, using a -x option. So we expanded the cfgadm SCSI plugin to use the same syntax. We wanted to add the ability to directly control the disk LEDs, so we added "-x led=LED[,mode=MODE]" (where LED could be power, fault, active or attn, just like PCI, and mode can be on, off or blink).

But we also felt there was a general problem being solved here, that of a "locator" indication. In the case of the M-class servers, the "locator" indication happens to be blinking the fault LED, but in the future, other platforms may have separate locator LEDs, or use some other method (for example, a separate locator LED, an annoying wable sound like a car alarm coming from the disk, or an LCD display on the from of the server that draws a hand pointing to the disk you want to remove). Our solution should not pre-suppose that the fault LED, or any LED for that matter, must be used as the locator.

So another -x option was added to the cfgadm SCSI plugin: "-x locator[={on|off}]". This allows a user to turn on or turn off the locator indication, regardless of the underlying implementation.

Here's a snippet from the new cfgadm_scsi(1M) man page showing the two new -x options:

 -x hardware_function   Some of the following  commands  can 
                        only  be  used with SCSI controllers
                        and some only with SCSI devices.

                        In the  following,  controller_ap_id
                        refers  to  an ap_id for a SCSI con-
                        troller,    for     example,     c0.
                        device_ap_id  refers to an ap_id for
                        a   SCSI   device,   for    example:
                        c0::dsk/c0dt3d0.

                        The  following   hardware   specific
                        functions are defined:

                        locator [=on|off] device_ap_id
                            Sets or gets the hard disk loca-
                            tor  LED,  if  it is provided by
                            the platform.  If  the  [on|off]
                            suboption  is not set, the state
                            of  the  hard  disk  locator  is
                            printed.

                        led[=LED,mode=on|off|blink]
                        device_ap_id
                            If  no  sub-arguments  are  set,
                            this  function  print  a list of
                            the  current  LED  settings.  If
                            sub-arguments   are   set,  this
                            function  sets  the  mode  of  a
                            specific LED for a slot.

To give you an idea of how this works, here's some annotated output from a Solaris session:

Check the current state of the fault LED for disk c0t0d0; it is now off:

# cfgadm -x led=fault c0::dsk/c0t0d0
Disk                    Led
c0t0d0                  fault=off

Turn on the locator indication for disk c0t0d0:

# cfgadm -x locator=on c0::dsk/c0t0d0
# cfgadm -x locator c0::dsk/c0t0d0
Disk                    Led
c0t0d0                  locator=on

Check the current state of the fault LED; it is now blinking:

# cfgadm -x led=fault c0::dsk/c0t0d0
Disk                    Led
c0t0d0                  fault=blink

Turn off the fault LED:

# cfgadm -x led=fault c0::dsk/c0t0d0
Disk                    Led
c0t0d0                  fault=off
# cfgadm -x led=fault,mode=off c0::dsk/c0t0d0

And the locator indication is also off:

# cfgadm -x locator c0::dsk/c0t0d0
Disk                    Led
c0t0d0                  locator=off

The intention is certainly that the system administrator would use '-x locator' to locate and replace internal SAS disks. But once SCSI FMA is supported in Solaris, we'll have the ability to set the fault LED on disks, and administrators can use '-x led' to view the state of the fault LED.