Joe Mocker's Weblog: How'd you know I was blogging at you if you weren't blogging at me?

Fresh back from the break I decided to give and upgrade to Solaris 10_U2 a go, and migrate my data to ZFS (now included w/ U2).

The ZFS migration was slightly painful so I figured I'd post my experience, in case anyone else might want to attempt it.

Actually migrating the data itself to a ZFS partition was simple, however I wanted to mirror the data as I have been doing with UFS/SVM which caused the problem. Apparently logged as a known bug:

6355416 zpool scrubbing consumes all memory, system hung

Even though the system was unresponsive, I decided to let it do its thing overnight and it did eventually finish resyncing/mirroring, and now everything is fine.

So for what its worth, here it is. The system involved was a SunBlade 2000 with 3G of RAM.

The goal was to combine three separate UFS partitions (/app, /work, /extra) into a single ZFS pool which would then host all those partitions again. The partitions would then use portions of the pool as need be, precluding the need to set arbitrary sizes as has been needed with UFS.

Also, the partitions were mirrored, and I wanted to continue mirroring with ZFS.

The three partitions are

/dev/md/dsk/d4       90030867 30328334 58802225    35%    /extra
/dev/md/dsk/d5       24795240 23208057  1339231    95%    /app
/dev/md/dsk/d6       20646121 16648727  3790933    82%    /work

(Notice that I was bumping up against some partition limits for a couple of the filesystems. The primary reason I wanted to convert to ZFS was to just have a big pool of space which could be used by whoever needed it.)

With the following SVM meta partitions assigned

d4 had submirrors d40 (c1t1d0s4), d41 (c1t2d0s4)
d5 had submirrors d50 (c1t1d0s5), d51 (c1t2d0s5)
d6 had submirrors d60 (c1t1d0s6), d61 (c1t2d0s6)

The method I went through was

• break the mirrors on one disk
• combine the partitions
• create a ZFS pool from the combined partition
• migrate the UFS/SVM data to the ZFS pool
• delete the remaining UFS/SVM partitions and combine them
• attach the combined partition as the second side of the mirror

Here's the procedure.

Break the mirrors on one disk

metadetach  d4 d41
metaclear d41
metadetach  d5 d51
metaclear d51
metadetach d6 d61
metaclear d61

Remove the meta db from the disk

metadb -d /dev/dsk/c1t2d0s7

Combine the disk partitions

I used the partition commands in the format utility to change the c1t2d0 disk.

Create a ZFS pool called storage and filesystems within it

zpool create storage c1t2d0s4
zfs create storage/extra
zfs create storage/work
zfs create storage/app

Migrate the UFS/SVM data to the ZFS filesystems

cd /work
find . -depth -print | cpio -pdmv /storage/work
cd /app
find . -depth -print | cpio -pdmv /storage/app
cd /extra
find . -depth -print | cpio -pdmv /storage/extra

Unmount the UFS/SVM partitions

unshareall
umount /app
umount /work
umount /extra

Remount the ZFS partitions where I expect them

zfs set mountpoint=/work storage/work
zfs set mountpoint=/extra storage/extra
zfs set mountpoint=/app storage/app
shareall

Remove the remaining SVM meta partitions

metaclear d4 d40
metaclear d5 d50
metaclear d6 d60

Remove the meta db from the disk

metadb -d /dev/dsk/c1t1d0s7

Combine the disk partitions

I used the partition commands in the format utility to change the c1t1d0 disk.

Finally, add the combined partition as a ZFS mirror

This was the painful step. Apparently ZFS decided to take all available CPU and memory in the system. The system became unresponsive. Apparently ZFS is a little to aggressive with its resyncing (they call it scrubbing/resilvering).

I found bug 6355416 which appears to describe the issue.

I would recommend before issuing this last command, that you boot the system into single user mode, and kill and processes that you don't need.

zpool attach storage c1t2d0s4 c1t1d0s4

The system will probably become unresponsive within a few minutes. If so, just walk away and let it do its thing. You can use the zpool status command to check the progress of the resilver.