3 Node Cluster Setup ********************** _____________________________________________________ Trivia.. a. Shared devices in the 3 node setup are shown below.. All these devices are LUNs from SAN storage (Sun StorEdge 3510), mapped to the Guest Operating Systems through RDM. b. The transport paths displayed below, show the "vmxnet" drivers loaded. These drivers are much more stable than the "pcn" drivers and come with the VMware Tools for Solaris. Single NIC Private Interconnect can also be seen in the output below. c. The setup has a NFS resource setup done on top of SVM. d. Quorum setup is shown later here. _____________________________________________________ bash-3.00# cluster status Cluster Nodes === --- Node Status --- Node Name Status --------- ------ prasam1 Online prasam2 Online pdosa1 Online Cluster Transport Paths === Endpoint1 Endpoint2 Status --------- --------- ------ prasam1:vmxnet1 prasam2:vmxnet1 Path online prasam1:vmxnet1 pdosa1:vmxnet1 Path online prasam2:vmxnet1 pdosa1:vmxnet1 Path online Cluster Quorum === --- Quorum Votes Summary --- Needed Present Possible ------ ------- -------- 2 3 3 --- Quorum Votes by Node --- Node Name Present Possible Status --------- ------- -------- ------ prasam1 1 1 Online prasam2 1 1 Online pdosa1 1 1 Online Cluster Device Groups === --- Device Group Status --- Device Group Name Primary Secondary Status ----------------- ------- --------- ------ dg1 prasam2 prasam1 Online dg2 prasam2 prasam1 Online --- Spare, Inactive, and In Transition Nodes --- Device Group Name Spare Nodes Inactive Nodes In Transistion Nodes ----------------- ----------- -------------- -------------------- dg1 pdosa1 - - dg2 pdosa1 - - --- Multi-owner Device Group Status --- Device Group Name Node Name Status ----------------- --------- ------ Cluster Resource Groups === Group Name Node Name Suspended State ---------- --------- --------- ----- nfs-rg prasam1 No Offline prasam2 No Offline pdosa1 No Online Cluster Resources === Resource Name Node Name State Status Message ------------- --------- ----- -------------- tdosa2-0b prasam1 Offline Offline - LogicalHostname offline. prasam2 Offline Offline - LogicalHostname offline. pdosa1 Online Online - LogicalHostname online. nfs-rs prasam1 Offline Offline - Completed successfully. prasam2 Offline Offline - Completed successfully. pdosa1 Online Online - Service is online. Cluster DID Devices === Device Instance Node Status --------------- ---- ------ /dev/did/rdsk/d1 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d10 pdosa1 Ok /dev/did/rdsk/d11 prasam2 Ok /dev/did/rdsk/d12 prasam1 Ok /dev/did/rdsk/d13 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d14 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d2 prasam1 Ok /dev/did/rdsk/d3 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d4 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d5 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d6 prasam2 Ok /dev/did/rdsk/d7 pdosa1 Ok prasam1 Ok prasam2 Ok /dev/did/rdsk/d8 prasam1 Ok prasam2 Ok /dev/did/rdsk/d9 pdosa1 Ok /dev/rdsk/c2t1d0 pdosa1 Ok NFS resource created on Cluster with SVM as Volume Manager. *********************************************************** bash-3.00# clrg status Cluster Resource Groups === Group Name Node Name Suspended Status ---------- --------- --------- ------ nfs-rg prasam1 No Offline prasam2 No Offline pdosa1 No Online Exercising State changes for the NFS resource and verifying. The commands execute below are self-explanatory of the actions. bash-3.00# clrg switch -n prasam1 nfs-rg bash-3.00# clrg switch -n prasam1 nfs-rg bash-3.00# clrg status Cluster Resource Groups === Group Name Node Name Suspended Status ---------- --------- --------- ------ nfs-rg prasam1 No Online prasam2 No Offline pdosa1 No Offline bash-3.00# share - /space ro "" bash-3.00# clrg offline nfs-rg bash-3.00# share bash-3.00# clrg status Cluster Resource Groups === Group Name Node Name Suspended Status ---------- --------- --------- ------ nfs-rg prasam1 No Offline prasam2 No Offline pdosa1 No Offline bash-3.00# clrg online nfs-rg bash-3.00# share - /space ro "" bash-3.00# clrg switch -n pdosa1 nfs-rg The resource gets switched over to node "pdosa1" which is confirmed by the NFS share. bash-3.00# rsh -l root pdosa1 share - /space ro "" Rebooting node "pdosa1" now. The NFS resource should failover from that node. bash-3.00# rsh -l root pdosa1 reboot bash-3.00# clnode status Cluster Nodes === --- Node Status --- Node Name Status --------- ------ prasam1 Online prasam2 Online pdosa1 Offline The resource fails over to node "prasam1" from node "pdosa1". bash-3.00# clrg status Cluster Resource Groups === Group Name Node Name Suspended Status ---------- --------- --------- ------ nfs-rg prasam1 No Online prasam2 No Offline pdosa1 No Offline bash-3.00# hostname prasam1 bash-3.00# share - /space ro "" Adding quorum devices to the cluster (SCSI and Quorum Server type Quorum Devices shown here) ******************************************************************************************** bash-3.00# clq list prasam1 prasam2 pdosa1 bash-3.00# cldev list -v d14 DID Device Full Device Path ---------- ---------------- d14 pdosa1:/dev/rdsk/c2t8d0 d14 prasam2:/dev/rdsk/c2t8d0 d14 prasam1:/dev/rdsk/c2t8d0 Adding SCSI quorum device "d14" bash-3.00# clq add -t scsi d14 bash-3.00# clq status Cluster Quorum === --- Quorum Votes Summary --- Needed Present Possible ------ ------- -------- 3 4 5 --- Quorum Votes by Node --- Node Name Present Possible Status --------- ------- -------- ------ prasam1 1 1 Online prasam2 1 1 Online pdosa1 0 1 Offline --- Quorum Votes by Device --- Device Name Present Possible Status ----------- ------- -------- ------ d14 2 2 Online Adding quorum server type quorum device bash-3.00# clq add -t quorum_server -p port=9000 -p qshost=sczone-2 sczone-2_qserver_qdevice bash-3.00# clq show Cluster Nodes === Node Name: prasam1 Node ID: 1 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000001 Node Name: prasam2 Node ID: 2 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000002 Node Name: pdosa1 Node ID: 3 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000003 Quorum Devices === Quorum Device Name: sczone-2_qserver_qdevice Enabled: yes Votes: 2 Global Name: sczone-2_qserver_qdevice Type: quorum_server Hosts (enabled): prasam1, prasam2, pdosa1 Quorum Server Host: sczone-2 Port: 9000 Quorum Device Name: d14 Enabled: yes Votes: 2 Global Name: /dev/did/rdsk/d14s2 Type: scsi Access Mode: scsi3 Hosts (enabled): prasam1, prasam2, pdosa1 Removing the quorum devices added (Note that quorum devices are mandatory only for 2 node clusters, and this is a 3 node setup. So it can do without a quorum device) bash-3.00# clq remove d14 sczone-2_qserver_qdevice bash-3.00# clq show Cluster Nodes === Node Name: prasam1 Node ID: 1 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000001 Node Name: prasam2 Node ID: 2 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000002 Node Name: pdosa1 Node ID: 3 Quorum Vote Count: 1 Reservation Key: 0x45CF545400000003