Out of Sync: Maddy's Blog

The fencing mechanism of SCX has been enhanced to make it optional.  Though it is recommended to use strong fencing mechanisms, some user environments require flexibility and in such cases, fencing can be turned off. 

Basics

* The fencing can be set a) globally or b) Individual disks. 

* Fencing type cannot be changed for disks configured as Quorum devices.

* Fencing type set at individual disk level will take precedence over global settings.

By default, the cluster will use Device ID (DID) pathcount  to determine the type of reservation protocol to be used.  For devices connected to 2 nodes, it will be PGR emulation and for 3 or greater connections, SCSI will be used.

How to determine what is the fencing type globally?

bash-3.00# scdidadm -G

The cluster is currently configured to use DID path count algorithm to determine fencing protocol for shared devices.

Now let us change the fencing type globally to nofencing:

bash-3.00# scdidadm -G nofencing
Warning: Device instance d1 is a quorum device - fencing protocol remains PATHCOUNT for the device.

Now to change the fencing algorithm to prefer scsi,

bash-3.00# scdidadm -G prefer3

** If scsi cannot be used for a subset of disks, the warning message will be displayed.

To return fencing to default settings,

bash-3.00# scdidadm -G pathcount

To change fencing at per disk level, use scdidadm with -F option:

To change the fencing type to noscrub,

bash-3.00# scdidadm -F nofencing-noscrub d2

** noscrub  switches fencing off without scrubbing the scsi keys. Keys are scrubbed only for scsi and not for PGRe.

For using global fencing settings,

bash-3.00# scdidadm -F useglobal d2

One of the questions that our team frequently encounters is regarding the type of scsi reservation Solaris Cluster uses.   This can be determined in a very simple way.  How many nodes does a shared disk connected to?  

 If the shared disk is connected to 2 nodes, then it uses SCSI-2 reservations.  If it is connected to more than 2 it uses SCSI-3 reservations.  Where can you find more details regarding this?   You should be checking out the scdidadm man page.

 bash-3.00# man -M /usr/cluster/man scdidadm
Reformatting page.  Please Wait... done

System Administration Commands                       scdidadm(1M)

NAME
     scdidadm - device identifier configuration  and  administra-
     tion utility wrapper

SYNOPSIS
     /usr/cluster/bin/scdidadm -C

     /usr/cluster/bin/scdidadm -c

     /usr/cluster/bin/scdidadm -F {pathcount | scsi3 | useglobal} instance

     /usr/cluster/bin/scdidadm -G

     /usr/cluster/bin/scdidadm -G {pathcount | prefer3}

     /usr/cluster/bin/scdidadm {-l | -L} [-h] [-o fmt]... [path | instance]

     /usr/cluster/bin/scdidadm -R {path | instance | all}

     /usr/cluster/bin/scdidadm -r

     /usr/cluster/bin/scdidadm -T remote-nodename -e replication-type

     /usr/cluster/bin/scdidadm -t source-instance:destination-instance -e replication-type
     [-g replication-device-group]

     /usr/cluster/bin/scdidadm [-u] [-i]

     /usr/cluster/bin/scdidadm -v
.

.

   pathcount           Determines the fencing  protocol  by
                             the  number  of  DID  paths that are
                             attached to the shared device.

                                 o    For  a  device  that   uses
                                      fewer than three DID paths,
                                      the command sets the SCSI-2
                                      protocol.

                                 o    For  a  device  that   uses
                                      three  or  more  DID paths,
                                      the command sets the SCSI-3
                                      protocol.

         scsi3               Sets the  SCSI-3  protocol.  If  the
                             device  does  not support the SCSI-3
                             protocol, the fencing protocol  set-
                             ting remains unchanged.

To find out the method being used, you can execute the following command:

bash-3.00# scdidadm -G

The cluster is currently configured to use DID path count algorithm to determine fencing protocol for shared devices.

To find the reservations keys on a disk, you should use the scsi command:

For SCSI-2, a sample output from a 2 node cluster:

bash-3.00#  /usr/cluster/lib/sc/pgre -c pgre_inkeys -d /dev/did/rdsk/d3
key[0]=0x47cd37ab00000001.
key[1]=0x47cd37ab00000002.

For SCSI-3, a sample output from a 4 node cluster:

 bash-3.00#  /usr/cluster/lib/sc/scsi -c inkeys -d /dev/did/rdsk/d10s2
Reservation keys(4):
0x47f5da4400000003
0x47f5da4400000001
0x47f5da4400000002
0x47f5da4400000004

WARNING:  Do not administer any of these commands without understanding the implications - do it only on a test cluster lest you're willing to land in trouble!!