| 75. |
 |
A3x00
Command Line Procedures
INDEX
2 Setting Path
2 Checking the whole Array
3 Checking the Controllers
4 Checking the Disks
5 Checking the Logical Devices
7 Recovering Disk Problems
7 Recovering Controller Problems
8 Recovering Logical Devices
9 Upgrading Firmware
Appendices:
A Decoding rmlog.log
2 A3500 Trouble Shooting from the Command Line
Setting up the path
# ksh
# PATH=$PATH:/usr/lib/osa/bin
# export PATH
This will need to be run every time you log in, unless it is added to the /.profile file.
Looking for errors
- Check /var/adm/messages
- LUNs offline
- SCSI errors
- Check rmlog.log
- # logutil /usr/lib/osa/rmlog.log
- See Attachment A for notes on how to decode the log
- See /etc/raid/raidcode.txt for more on decoding
- Look for patterns of errors in the logs
Here is some examples of useful commands, and some outputs that you might expect to see:-
# healthck -a
**********************************************************************************
Health Check Summary Information
p4u-5500a_002: Multiple Unresponsive Drives at Drive [2,2];[2,3];[2,5];[2,8];[2,9];[2,12];[2,13]
healthck succeeded!
**********************************************************************************
3
To Check Controllers
# rdacutil -i p4u-5500a_002
************************************************************************************
p4u-5500a_002: dual-active
Active controller a (c1t5d0s0) units: 0 2
Active controller b (c2t4d1s0) units: 1
rdacutil succeeded!
************************************************************************************
# lad
************************************************************************************
c1t5d0s0 1T81410687 LUNS: 0 2
c2t4d1s0 1T82623877 LUNS: 1
************************************************************************************
# storutil -c c1t5d0s0 -d
************************************************************************************
p4u-5500a_002:
Controller A: 1T81410687 ( c1t5d0s0 )
Controller B: 1T82623877 ( c2t4d1s0 )
Independent Controller Configuration: OFF
************************************************************************************
4
To Check Disks
# drivutil -i c1t5d0s0
************************************************************************************
Drive Information for p4u-5500a_002
Location Capacity Status Vendor Product Firmware Serial
(MB) ID Version Number
[1,0] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G074
[2,0] 4094 Optimal SEAGATE ST34501WCSUN4.2G 0558 LG433750
[3,0] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G072
[4,0] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G086
[5,0] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G053
[1,1] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G083
[2,1] 4094 Optimal SEAGATE ST34501WCSUN4.2G 0558 LG461369
[3,1] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G075
[4,1] 17274 Optimal FUJITSU MAA3182S SUN18G 1907 00H22912
[5,1] 17274 Optimal FUJITSU MAA3182S SUN18G 1907 00H22718
[1,2] 17274 Optimal FUJITSU MAA3182S SUN18G 1907 00H24391
[2,2] 0 Unresponsive
[3,2] 17274 Optimal FUJITSU MAA3182S SUN18G 1907 00H24741
[2,3] 0 Unresponsive
[2,4] 4094 Optimal SEAGATE ST34501WCSUN4.2G 0558 LG461444
[2,5] 0 Unresponsive
[2,8] 0 Unresponsive
[4,8] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G083
[5,8] 17274 Optimal FUJITSU MAA3182S SUN18G 1705 00G082
[2,9] 0 Unresponsive
[2,10] 4094 Optimal SEAGATE ST34501WCSUN4.2G 0558 LG432787
[2,11] 4094 Optimal SEAGATE ST34501WCSUN4.2G 0558 LG461468
[2,12] 0 Unresponsive
[2,13] 0 Unresponsive
drivutil succeeded!
************************************************************************************
5
To Check the Logical Devices (LUNs and Drive Groups)
# drivutil -d c1t5d0s0
************************************************************************************
Drives in Group for p4u-5500a_002
Group Drive List [Channel,Id]
Unassigned [3,2]; [4,8]; [5,8];
Group 1: [1,0]; [3,0]; [4,0]; [5,0];
Group 2: [2,0]; [2,1]; [2,2]; [2,3]; [2,4]; [2,5]; [2,8]; [2,9]; [2,10]; [2,11]; [2,12]; [2,13];
Group 3: [1,1]; [3,1]; [4,1]; [5,1]; [1,2];
drivutil succeeded!
************************************************************************************
# drivutil -I p4u-5500a_002
************************************************************************************
Group Information for p4u-5500a_002
Group No. of RAID No. of Total Remaining
LUNs Level Drives Space(MB) Space(MB)
Unassigned 0 - 3 51704 51704
1 1 0 4 68938 68937
2 1 1 12 24325 0
3 1 5 5 68938 0
drivutil succeeded!
************************************************************************************
6
# drivutil -p 0 c1t5d0s0
************************************************************************************
p4u-5500a_002 unit 0: optimal
drivutil succeeded!
************************************************************************************
# drivutil -l p4u-5500a_002
***********************************************************************************
Logical Unit Information for p4u-5500a_002
LUN Group Device RAID Capacity Status
Name Level (MB)
0 1 c1t5d0s0 0 1 Optimal
1 2 c2t4d1s0 1 24325 Optimal
2 3 c1t5d2s0 5 68938 Optimal
drivutil succeeded!
************************************************************************************
7 Recovery
Disks
To try and ’unfail’ a disk in the state Replaced
Failed
Unresponsive, do
# drivutil -u <drive number> <controller id>
Where <drive number> is the tray number, followed by the disk number, with no separator, and the
<controller id> is the name of the array, or the ID of the specific controller
e.g. # drivutil -u 210 c2t4d0s0
(This will ’unfail’ disk 10 on tray 2, on the controller c2t4d0s0)
If this does not work, try and manually fail the drive, and then try and ’unfail’ it again afterwards. To
fail the drive, do:
# drivutil -f <drive number> <controller id>
e.g. # drivutil -f 210 c2t4d0s0
Controllers
To ’unfail’ a controller. Normally this would succeed if the controller was held in reset (all lights on
the controller on).
# rdacutil -u <raid module specifier> or
# rdacutil -U <raid module specifier>
e.g. # rdacutil -u p4u-5500a_002
This will bring the controller back online, and should move the LUNs so that they are balanced in the
same way as before the failure. If you use the lower case ’u’ as the argument then normal checks will
be made before the controller is put back on line, and if you use the the uppercase ’U’ then the controller
will come back on line without any checks being made.
8
If required you can manually fail one of the controllers. This will automatically move the LUNs on the
controller that we are trying to fail on to the controller that will remain ’optimal’.
# rdacutil -f <id of the controller you want to fail>
# rdacutil -F <id of alternate controller to the one you want to fail>
Logical Devices
To revive a LUN from a dead state:
# drivutil -r <LUN number> <raid module specifier>
e.g. # drivutil -r 2 p4u-5500a_002
You must now run fsck on the file systems on the LUN, and then mount it, and check the data. The sanity
of this data is not guaranteed. (In fact it may be complete insane!)
9
Upgrading Firmware
To check the current version of the firmware that is installed:
# raidutil -i <controller id>
e.g. # raidutil -i c1t4d0s0
***********************************************************************************
LUNs found on c1t4d0s0
LUN 0 RAID 0 1 MB
LUN 2 RAID 5 68938 MB
Vendor ID Symbios
ProductID StorEDGE A3000
Product Revision 0205
Boot Level 02.05.01.00
Boot Level Date 12/02/97
Firmware Level 02.05.02.15
Firmware Date 09/08/98
raidutil succeeded
***********************************************************************************
To quisce the one of the controllers, so that it can be upgraded.
If the controllers are currently active/active, then run the following command
# raidutil -m 1 <raid module specifier>
e.g. # raidutil -m 1 p4u-5500a_002
This will make controller A active, and controller B passive. You can now upgrade controller B.
Run the command again, and the currently active controller will become passive, and vice versa. This
will allow you to upgrade controller A
10
Then run the following command to put the controllers back into an active/active state.
# rdacutil -m 2 -b <raid module specifier>
e.g. # rdacutil -m 2 -b c1t4d0s0
(The -b will rebalance the LUNs to a ’nominal’ configuration)
To actually do the upgrade, you will need to load the relevant patch (currently 106513), which will put
the firmware files in the usr/lib/osa/fw/ directory.
The files will be proceeded with the firmware level, and end with a suffix, such as .apd or .fcd.
e.g. /usr/lib/osa/fw/02050211.apd
All the files with the relevant firmware number prefix should be installed.
To install the firmware run the command:
# fwutil <file> <raid controller specifier>
e.g. fwutil /usr/lib/osa/fw/02050211.apd p4u-5500a_002
After the upgrades, check the firmware levels, with the command:
# raidutil -i <controller id>
e.g. # raidutil -i c1t4d0s0
Attachment A
Decoding rmlog.log
First run the command:
# logutil /usr/lib/osa/rmlog.log
Record # 409:
Host: bart
Date: 09/28/97
Time: 16:22:18
Device: c2t5d6s0
Controller: 1T1234567JT
Error Type: 06
LUN: 03
LUN Status: 01
Drive: 2E
Error Number: 3F80
Sense Data:
7000 0600 0000 0098 0000 0000 3F80 2E00
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0008 0500 0000 0000 0000 0000
0000 0000 0F05 3154 3731 3332 3230 3736
2020 2020 2020 0204 1D00 0003 0100 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0050 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 000
The first line tells you the event number, followed by the
name of the host. Next is the date, time of the event. The drive
invloved, the controller serial number of the A3000 controller invloved.
Next would be the type of error, the LUN number, the location of the
actual drive, the error number and then the sense data.
LUN Status Definition:
00 => Optimal condition
20 => Optimal with parity scan in progress
01 => Degraded LUN- waiting for repair action
41 => Degraded LUN - replaced drive being formatted.
02 => Degraded LUN - replaced drive being reconstructed
04 => Dead LUN
44 => Dead LUN - format in progress
54 => Dead LUN - creation in progress
74 => Dead LUN - wrong drive removed/replaced
Error Number Definition:
This information is in the raidcode.txt on the sonoma page or
in RM6 6.1
Error Type:
This will always be byte 2 in the sense data, they are defined as
follows:
00 => No Sense
01 => Recovered Error
02 => Not Ready
03 => Medium Error
04 => Hardware Error
05 => Illegal Request
06 => Unit Attention
07 => Data Protect
0B => Aborted Command
0E => Miscompare
In the example above, you can see that it is telling you that the error
has to do with "Unit Attention:.
Drive Location:
The 362x controller reports drive location start with byte offset 0, and
going from bottom to top, it would be 0,1,2,3 and 4. This would correspond
with drive channels 1,2,3,4 and 5 that RM6 uses. The drives are targeted
from 8 thru E in the firmware. In our example above of drive location
2E, that would be the third tray from the bottom and the last drive in the
tray. RM6 would see this drive 3,14.
Sense Data Decoding:
Byte 2 is the sense key inforamtion
byte 7 is Additional Sense Length, this value will indicate the number
of additional sense bytes to follow.
Bytes 8-11 will always be zero unless the has been unsuccessful Reassign
blocks.
Bytes 12-13, this information is in the raidcode.txt file or on the sonoma
web page.
Byte 14, Field Replaceable Unit Code, this is also in the raidcode.txt file.
Bytes 37-40, Error Detection Point, will indicate were in the software
the error was detected.
Bytes 52-53, Host Descriptor:
LSB:
bit 0 => data is being transfered 16-bit wide
bit 1 => Reserved
bit 2 => wide negotiation completed successfully
bit 3-7 => Reserved
MSB:
0 => Message using host
1 => Reselectable host
2 => data is being transferred synchronously(bit off means
asynchronously data transfer)
3 => synchronous negotiation successful
4 => Reserved
5 => AEN supported
6 => Polled AEN supported
7 => Reserved
Bytes 54-69, Controller serial number
Bytes 70-73, Array software Revision
Byte 75, LUN Number
Byte 76, LUN status
Byte 101, Raid Level, a value off 255 indicates that the LUN Raid
level is undefined
To break down on sense data a little:
byte 2 7 1213 14
7000 0600 0000 0098 0000 0000 3F80 2E00 sense data
0000 0000 0000 0000 0000 0000 0000 0000 sense data
byte 37 38
0000 0000 0008 0500 0000 0000 0000 0000 sense data
byte 5253 5455 5657 5859 6061 6263
0000 0000 0F05 3154 3731 3332 3230 3736 sense data
byte6465 6667 6869 7071 7273 75 76
2020 2020 2020 0204 1D00 0003 0100 0000 sense data
0000 0000 0000 0000 0000 0000 0000 0000 sense data
byte 101
0000 0000 0050 0000 0000 0000 0000 0000 sense data
0000 0000 0000 0000 0000 0000 0000 000 sense data
Oracle 10g RAC on Solaris 10 With SVM Install
1. Mirror on boot disk: (All nodes)
# metainit -f
d100 1 1 c0t0d0s0
d101 1 1 c0t0d0s1
d104 1 1 c0t0d0s4
d105 1 1 c0t0d0s5
d106 1 1 c0t0d0s6(or d16, d116 on node 2)
d0 -m d100
d1 -m d101
d4 -m d104
d5 -m d105
d6 -m d106 (or d16, d116 on node 2)
# metaroot d0
edit /etc/vfstab
change swap to /dev/md/dsk/d1
change /globaldevices to /dev/md/dsk/d6 (d16 on node 2)
change /var to /dev/md/dsk/d4
change /var/crash to /dev/md/dsk/5
turn on UFS logging on / and /globaldevices
# reboot
# metainit -f
d200 1 1 c1t0d0s0
d201 1 1 c1t0d0s1
d204 1 1 c1t0d0s4
d205 1 1 c1t0d0s5
d206 1 1 c1t0d0s6 (d216 on node 2)
# metattach d0 d200
# metattach d1 d201
# metattach d4 d204
# metattach d5 d205
# metattach d6 d206 (d16 d216 on node 2)
Change devalias for mirror: (both nodes)
halt
devalias mirror /sbu@7,0/QLGC,ips@0,10000/sd0,0
setenv use-nvramrc? true
boot
#1. Solaris 10 Patches
#2. Sun Cluster 3.1 Update 4
#3. RAC Support Packages
SunCluster_Oracle_RAC_CVM_3.1
SunCluster_Oracle_RAC_FRAMEWORK_3.1
SunCluster_Oracle_RAC_HWRAID_3.1
SunCluster_Oracle_RAC_SVM_3.1
#4. UDLM 3.3.4.8 for S10:
#5. Oracle 10g RAC Pkage Install +++++++++++++++++++++++++++++ -> examples ...SC3.1U4 with SVM
Mirror Boot disk
metadb -a -c3 -f /dev/rdsk/c0t0d0s7
metainit -f d150 1 1 c0t0d0s0
metainit -f d151 1 1 c0t1d0s0
metainit -f d152 1 1 c0t0d0s1
metainit -f d153 1 1 c0t1d0s1
metainit -f d154 1 1 c0t0d0s3
metainit -f d155 1 1 c0t1d0s3
metainit d100 -m d150
metainit d101 -m d152
metainit d102 -m d154
metattach d100 d151
metattach d101 d153
metattach d102 d155
metainit -f d156 1 1 c0t0d0s4
metainit -f d157 1 1 c0t1d0s4
metainit -f d158 1 1 c0t0d0s5
metainit -f d159 1 1 c0t1d0s5
metainit d103 -m d156
metainit d104 -m d158
metattach d103 d157
metattach d104 d159
===================================================
Local Filesystem (Archive etc) setup
metaset -s db1arch -a -h intdb1 intdb2
metaset -s db2arch -a -h intdb2 intdb1
metaset -s db2arch -a -h intdb2 intdb1
metaset -s db1arch -a /dev/did/rdsk/d4
metaset -s db1arch -a /dev/did/rdsk/d5
metaset -s db2arch -a /dev/did/rdsk/d21
metaset -s db2arch -a /dev/did/rdsk/d22
metainit -s db1arch d110 2 1 /dev/did/rdsk/d4s0 1 /dev/did/rdsk/d5s0
metainit -s db2arch d210 2 1 /dev/did/rdsk/d21s0 1 /dev/did/rdsk/d22s0
metainit -s db1arch d111 -p d110 60g
newfs /dev/md/rdsk/db1arch /d111
metainit -s db2arch d211 -p d210 60g
newfs /dev/md/rdsk/db2/arch d211
scrgadm -a -g intdb1-rg -h intdb1,intdb2
scrgadm -a -g intdb2-rg -h intdb2,intdb1
scrgadm -a -g intdb1-rg -j db1arch-rs -t SUNW.HAStoragePlus -x FilesystemMountPoints=/p1_ar1 -x GlobalDevicePaths=db1arch -x AffinityOn=TRUE
scrgadm -a -g intdb2-rg -j db2arch-rs -t SUNW.HAStoragePlus -x FilesystemMountPoints=/p2_ar2 -x GlobalDevicePaths=db2arch -x AffinityOn=TRUE
root@intdb1 metaset -s intrac -a -h intdb1 intdb2
intdb2: RPC: Program not registered
root@intdb1 metaset -s intrac
Set name = intrac, Set number = 3
Host Owner
intdb1
intdb2
root@intdb1 metaset -s intrac -a -m intdb1
root@intdb1 metaset -s intrac -a -m intdb2
root@intdb1 medstat -s intrac
Mediator Status Golden
intdb1 Ok No
intdb2 Ok No
===========================================
RAC Volume
metaset -s intrac -a -M intdb1 intdb2
-- Multi-owner Device Groups --
Device Group Online Status
------------ -------------
Multi-owner device group: intrac -
metaset -s intrac -a /dev/did/rdsk/d6 ~ d 30
metainit -s intrac d300 15 1 /dev/did/rdsk/d6s0 1 /dev/did/rdsk/d7s0 1 /dev/did/rdsk/d8s0 1 /dev/did/rdsk/d9s0 1 /dev/did/rdsk/d10s0 1 /dev/did/rdsk/d11s0 1 /dev/did/rdsk/d12s0 1 /dev/did/rdsk/d13s0 1 /dev/did/rdsk/d14s0 1 /dev/did/rdsk/d15s0 1 /dev/did/rdsk/d16s0 1 /dev/did/rdsk/d17s0 1 /dev/did/rdsk/d18s0 1 /dev/did/rdsk/d19s0 1 /dev/did/rdsk/d20s0
======= Redo Log ============
metainit -s intrac d301 -p d300 255m
metainit -s intrac d302 -p d300 255m
metainit -s intrac d303 -p d300 255m
metainit -s intrac d304 -p d300 255m
metainit -s intrac d305 -p d300 255m
metainit -s intrac d306 -p d300 255m
metainit -s intrac d307 -p d300 255m
metainit -s intrac d308 -p d300 255m
metainit -s intrac d309 -p d300 255m
metainit -s intrac d310 -p d300 255m
metainit -s intrac d311 -p d300 255m
metainit -s intrac d312 -p d300 255m
metainit -s intrac d313 -p d300 255m
metainit -s intrac d314 -p d300 255m
metainit -s intrac d315 -p d300 255m
metainit -s intrac d316 -p d300 255m
===System Engin ===========
metainit -s intrac d317 -p d300 55m
metainit -s intrac d318 -p d300 155m
metainit -s intrac d319 -p d300 155m
metainit -s intrac d320 -p d300 155m
metainit -s intrac d321 -p d300 155m
metainit -s intrac d322 -p d300 4005m
metainit -s intrac d323 -p d300 4005m
metainit -s intrac d324 -p d300 105m
metainit -s intrac d325 -p d300 105m
metainit -s intrac d326 -p d300 2005m
metainit -s intrac d327 -p d300 2005m
metainit -s intrac d328 -p d300 2005m
metainit -s intrac d329 -p d300 2005m
=======Data Table Spabe ========
metainit -s intrac d330 -p d300 2005m
metainit -s intrac d331 -p d300 2005m
metainit -s intrac d332 -p d300 2005m
metainit -s intrac d333 -p d300 2005m
metainit -s intrac d334 -p d300 2005m
metainit -s intrac d335 -p d300 2005m
metainit -s intrac d336 -p d300 2005m
metainit -s intrac d337 -p d300 2005m
metainit -s intrac d338 -p d300 2005m
metainit -s intrac d339 -p d300 2005m
metainit -s intrac d340 -p d300 2005m
metainit -s intrac d341 -p d300 2005m
metainit -s intrac d342 -p d300 2005m
metainit -s intrac d343 -p d300 2005m
metainit -s intrac d344 -p d300 2005m
metainit -s intrac d345 -p d300 2005m
metainit -s intrac d346 -p d300 2005m
metainit -s intrac d347 -p d300 2005m
metainit -s intrac d348 -p d300 2005m
metainit -s intrac d349 -p d300 2005m
metainit -s intrac d350 -p d300 2005m
metainit -s intrac d351 -p d300 2005m
# /etc/rc3.d/s30initucmm start
scrgadm -at SUNW.rac_framework
scrgadm -at SUNW.rac_udlm
scrgadm -at SUNW.rac_svm
scrgadm -a -g rac-rg -y maximum_primaries=2 -y desired_primaries=2 -y nodelist=intdb1,intdb2
scrgadm -a -j rac-framework-rs -g rac-rg -t SUNW.rac_framework
scrgadm -a -j rac-udlm-rs -g rac-rg -t SUNW.rac_udlm -y resource_dependencies=rac-framework-rs -x port=7000
scrgadm -a -j rac-svm-rs -g rac-rg -t SUNW.rac_svm -y resource_dependencies=rac-framework-rs
=================================================================================================
Solaris Cluster software provides the framework and API required to make applications highly
available on Solaris OS. The software application, designed for solaris, does not have to
be modified to use Solaris Cluster API. Instead, you write an agent which acts as the interface
between Solaris Cluster core software and the application. Solaris Cluster comes bundled with a
rich portfolio of agents, also called data services, which make applications highly available on
Solaris Cluster. These agents, designed and developed by the Solaris Cluster Engineering team,
have undergone rigourous testing by the internal Quality Assurance engineers.
Agents for several software products are available in Solaris Cluster 3.1 and the Solaris Cluster 3.2
releases. For details on the supported applications and how to configure and administer the agents
for the supported applications please refer to the Data Service administration guides, available
at the following location:
http://docs.sun.com/app/docs/coll/1574.1/
Writing Custom Data Services
-----------------------------
If an agent for any software application is not already available then, you can write an agent
for that application by using the Data Service Development tools available in the product.
First, check if the application can be made highly available on Solaris Cluster software. The
following chapter in the Solaris Cluster Data Services Developer's Guide lists everything you need
to verify your application's cluster readiness.
http://docs.sun.com/app/docs/doc/819-2972/6n57ngipe?a=view
Most of the applications can be integrated with Solaris Cluster software right out of the box.
Sometimes you might have to enhance the application to be able to integrate with Solaris Cluster.
Once the application is ready for integration, use the Solaris Cluster Agent Builder to
generate an agent for you. The Solaris Cluster Agent Builder not only generates the code for you
but also generates the Makefiles to compile the code and build a nice Solaris package. This
generated package can be easily installed by using the pkgadd utility. The Solaris Cluster Agent
Builder can generate agent source code in C programming language or ksh. The following chapter
in the Data Services Developers Guide explains how to use the Agent Builder to develop an agent.
http://docs.sun.com/app/docs/doc/819-2972/6n57ngisa?a=view
The Solaris Cluster Agent Builder and the API library is available as a package, SUNWscdsdev,
which can be downloaded on to your desktop. Download the SUNWscdsdev package on your desktop and
follow the instructions in the Developers guide to generate and compile an agent for the Solaris
Cluster Software.
If you do not want to write a seperate agent for your application, you can use the Generic Data
Service (GDS) agent to make your application HA on Solaris Cluster software. Generic Data Service,
as the name suggests, is a generic agent designed by Solaris Cluster Engineering. GDS takes as an
input, scripts to start, stop, validate, and probe an application. Instead of writing a separate agent
for your application, you just write scripts to start, stop, validate, and probe your application
and supply these scripts as extension properties to the GDS resource at the time of creation. The
following chapter in the Data Services Developers Guide explains how to use GDS to make applications
Highly Available on Solaris Cluster.
~ ~ ~
The Solaris Cluster Agent Builder can be downloaded from the following location:
http://www.sun.com/download/products.xml?id=45b160e5
To test the agent, the Solaris Cluster software has to be installed and configured. A single node cluster
can be configured to test an agent. The Solaris Cluster 3.2 Software and the Solaris Cluster agents can
be downloaded from the following location:
http://www.sun.com/download/products.xml?id=4581ab9e
Also, check out http://blogs.sun.com/SC for some cool tips and other information straight from the
Solaris Cluster engineering organization.
http://www.sun.com/software/solaris/cluster/support.xml
The majority of Open High Availability Cluster code is released under the Common Development and Distribution License (CDDL) Version 1.0.
Source based on existing open source projects will continue to be available under their current licenses. Some binary components are covered under the OpenSolaris Binary License and some are covered under other open source licenses. Specific download pages provide license information associated with the available component pieces.
Running Open HA Cluster
- Solaris Cluster Express, the release of Solaris Cluster for the OpenSolaris platform, will be made available in the near future.
- In the meantime, you can run the open-source agents on Solaris Cluster 3.2 on Solaris 10.
Building Open HA Cluster
- Download the latest Open HA Cluster source, build tools, and related binaries.
- Download the latest Open HA Cluster test source.
Building Agents
- Building Agents
Instructions for downloading, installing and building the Open HA Cluster Agents.
Building G11N
- Building Agents G11N
Instructions for downloading, installing and building the Open HA Cluster Agents G11N.
Building Tests
One the most useful new command I found in Solaris 10 is fcinfo, a command line interface that will display information on HBA ports on a host, but also many useful bits of information on connected storage remote port WWN, raid type, link status,etc.
root@node57 # fcinfo hba-port -l
HBA Port WWN: 10000000c9503663
OS Device Name: /dev/cfg/c3
Manufacturer: Sun Microsystems, Inc.
Model: LP10000-S
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 20000000c9503663
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 6
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 24
Invalid CRC Count: 0
HBA Port WWN: 10000000c9504044
OS Device Name: /dev/cfg/c2
Manufacturer: Sun Microsystems, Inc.
Model: LP10000-S
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 20000000c9504044
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 15
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 62
Invalid CRC Count: 0
HBA Port WWN: 210000e08b076df5
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Model: 375-3019-xx
Type: unknown
State: offline
Supported Speeds: 1Gb
Current Speed: not established
Node WWN: 200000e08b076df5
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
root@node57 # fcinfo remote-port -sl -p 10000000c9503663
Remote Port WWN: 216000c0ff886d4f
Active FC4 Types:
SCSI Target: yes
Node WWN: 206000c0ff086d4f
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 3
Loss of Signal Count: 3
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
LUN: 0
Vendor: SUN
Product: StorEdge 3510
OS Device Name: /dev/rdsk/c7t600C0FF000000000086D4F0F39F16600d0s2
LUN: 1
Vendor: SUN
Product: StorEdge 3510
OS Device Name: /dev/rdsk/c7t600C0FF000000000086D4F17C87C9B00d0s2
Remote Port WWN: 10000000c9504044
Active FC4 Types:
SCSI Target: no
Node WWN: 20000000c9504044
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 15
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 62
Invalid CRC Count: 0
root@node57 #
RAID-Z is a data/parity scheme like RAID-5, but it uses dynamic stripe width.
Every block is its own RAID-Z stripe, regardless of blocksize. This means
that every RAID-Z write is a full-stripe write. This, when combined with the
copy-on-write transactional semantics of ZFS, completely eliminates the
RAID write hole.
RAID-Z is also faster than traditional RAID because it never has to do
read-modify-write.
Whoa, whoa, whoa -- that's it? Variable stripe width? Geez, that seems
pretty obvious. If it's such a good idea, why doesn't everybody do it?
Well, the tricky bit here is RAID-Z reconstruction. Because the stripes
are all different sizes, there's no simple formula like "all the disks
XOR to zero." You have to traverse the filesystem metadata to determine
the RAID-Z geometry. Note that this would be impossible if the filesystem
and the RAID array were separate products, which is why there's nothing
like RAID-Z in the storage market today. You really need an integrated
view of the logical and physical structure of the data to pull it off.
But wait, you say: isn't that slow? Isn't it expensive to traverse
all the metadata? Actually, it's a trade-off. If your storage pool
is very close to full, then yes, it's slower. But if it's not too
close to full, then metadata-driven reconstruction is actually faster
because it only copies live data; it doesn't waste time copying
unallocated disk space.
But far more important, going through the metadata means that ZFS
can validate every block against its 256-bit checksum as it goes.
Traditional RAID products can't do this; they simply XOR the data
together blindly.
Which brings us to the coolest thing about RAID-Z: self-healing data.
In addition to handling whole-disk failure, RAID-Z can also detect
and correct silent data corruption. Whenever you read a RAID-Z block,
ZFS compares it against its checksum. If the data disks didn't return
the right answer, ZFS reads the parity and then does combinatorial
reconstruction to figure out which disk returned bad data. It then
repairs the damaged disk and returns good data to the application.
ZFS also reports the incident through Solaris FMA so that the system
administrator knows that one of the disks is silently failing.
Finally, note that RAID-Z doesn't require any special hardware.
It doesn't need NVRAM for correctness, and it doesn't need write buffering
for good performance. With RAID-Z, ZFS makes good on the original RAID
promise: it provides fast, reliable storage using cheap, commodity disks.
For a real-world example of RAID-Z detecting and correcting silent
data corruption on flaky hardware, check out
Eric Lowe's SATA saga.
The current RAID-Z algorithm is single-parity, but the RAID-Z concept
works for any RAID flavor. A double-parity version is in the works.
One last thing that fellow programmers will appreciate: the entire
RAID-Z implementation
is just 599 lines.
[ Read More]
Archives
| « 11월 2009 | | 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | | | | | | | | | | | | | | | Today |
|
