As Good A Place As Any

Recipe for Sun Fire X4500 RAID-Z Config with Hot Spares

This configuration was discussed briefly in an earlier entry, but I wanted to dig a bit deeper on it by way of a better explanation.

The below command configures the disks in a Sun Fire X4500 as a single ZFS storage pool with 8 RAID-Z.4+1 stripes and 6 hot spares. ZFS dynamically stripes data in the pool across all of the stripes. Note that this runs over multiple lines.

zpool create -f  bigpool  \
raidz c0t0d0 c1t0d0 c4t0d0 c6t0d0 c7t0d0 \
raidz c1t1d0 c4t1d0 c5t1d0 c6t1d0 c7t1d0 \
raidz c0t2d0 c4t2d0 c5t2d0 c6t2d0 c7t2d0 \
raidz c0t3d0 c1t3d0 c5t3d0 c6t3d0 c7t3d0 \
raidz c0t4d0 c1t4d0 c4t4d0 c6t4d0 c7t4d0 \
raidz c0t5d0 c1t5d0 c4t5d0 c5t5d0 c7t5d0 \
raidz c0t6d0 c1t6d0 c4t6d0 c5t6d0 c6t6d0 \
raidz c0t7d0 c1t7d0 c4t7d0 c6t7d0 c7t7d0 \
spare c0t1d0 c1t2d0 c4t3d0 c6t5d0 c7t6d0 c5t7d0

The server has six controllers each with eight disks and I have built the storage pool so that the hot spares are staggered across the controllers to spread I/O evenly and to enable me to build 8 RAID-Z stripes of equal length. Six hot spares may seem a lot, but it was either six spares or have fewer and make the RAID-Z stripes in the storage pool different lengths.

With 500 GB SATA-II disks this gives a usable capacity of 14 TB. Note that the capacity of a 500 GB disk as seen by Solaris is 466 GB.

root@x4500 # df -h bigpool  
Filesystem             size   used  avail capacity  Mounted on
bigpool                 14T    39K    14T     1%    /bigpool

The performance of this configuration was discussed in a previous blog entry.

Posted at 11:15AM Oct 11, 2007 in Storage  |