Replication of an entire ZFS storage pool.
All of the volumes contained in a single ZFS storage pool need to be configured into their own SNDR replica. This list can be derived using the output of "zpool status <pool-name>"
When ZFS uses the entire volume (as is the case in most situations), all ZFS data is placed in slice 0. /dev/rdsk/c?t?d?s0 of the volume being configured. If partial volumes are configured, be careful to specify the individual slice being used.
Bitmap volumes are needed for each replica, that size of which is derived by using 'dsbitmap -r <volume-name>".
Provision bitmaps volumes on SSDs, cached arrays, RAID-1 or RAID-10 volumes. Avoiding using RAID-Z or RAID-5 storage, due to the high I/O cost of flipping one or more bits.
At SNDR create time, use the "g <group>" option, or after the fact with the "-R g <group> [set]" option with sndradm, placing all replicas for a single ZFS storage pool in a the same SNDR I/O consistency group.
NOTE: When SNDR replicas are configured before the ZFS storage pool is created, use "sndradm -E ...", verses "sndradm -e ...", avoiding the cost in both time and network resources to synchronize all volumes in the ZFS storage pool, volumes which essential contain no data.
Examples of using both forms of SNDR enable follow.
Using sndradm -E
[on primary host]
zpool create <pool-name> <vdev1> ... <vdev'n'>
zpool status
[ keep list of all vdevs configured]
zpool destroy <pool-name>
dsbitmap -r <vdev1>
[ keep output of required bitmap volume size depending on type]
[provision 'n' bitmap volumes, based on size returned from dsbitmap]
[on secondary host]
zpool create <pool-name> <vdev1> ... <vdev'n'>
zpool status
[ keep list of all vdevs configured]
zpool destroy <pool-name>
[ keep output of required bitmap volume size depending on type]
[provision 'n' bitmap volumes, based on size returned from dsbitmap]
[on primary host]
sndradm -E <primary-hostname> <vdev1> <bitmap1> <secondary-hostname> <remote-vdev1> <remote-bitmap1> ip async g <pool-name>
.
.
sndradm -E <primary-hostname> <vdev'n'> <bitmap'n'> <secondary-hostname> <remote-vdev'n'> <remote-bitmap'n'> ip async g <pool-name>
zpool create <pool-name> <vdev1> ... <vdev'n'>
[on secondary host]
sndradm -E <primary-hostname> <vdev1> <bitmap1> <secondary-hostname> <remote-vdev1> <remote-bitmap1> ip async g <pool-name>
.
.
sndradm -E <primary-hostname> <vdev'n'> <bitmap'n'> <secondary-hostname> <remote-vdev'n'> <remote-bitmap'n'> ip async g <pool-name>
[note: zpool create not done on secondary, as the ZFS storage pool from the SNDR primary is replicated here by SNDR]
[on primary host]
sndradm -g <pool-name> -nu
Using sndradm -e
[on primary host]
zpool status
[ keep list of all vdevs configured]
dsbitmap -r <vdev1>
[provision 'n' bitmap volumes based on size returned from dsbitmap]
[on secondary host]
zpool create <pool-name> <vdev1> ... <vdev'n'>
zpool status
[ keep list of all vdevs configured]
zpool destroy <pool-name>
[provision 'n' bitmap volumes based on size returned from dsbitmap]
[on primary host]
sndradm -e <primary-hostname> <vdev1> <bitmap1> <secondary-hostname> <remote-vdev1> <remote-bitmap1> ip async g <pool-name>
.
.
sndradm -e <primary-hostname> <vdev'n'> <bitmap'n'> <secondary-hostname> <remote-vdev'n'> <remote-bitmap'n'> ip async g <pool-name>
[on secondary host]
sndradm -e <primary-hostname> <vdev1> <bitmap1> <secondary-hostname> <remote-vdev1> <remote-bitmap1> ip async g <pool-name>
.
.
sndradm -e <primary-hostname> <vdev'n'> <bitmap'n'> <secondary-hostname> <remote-vdev'n'> <remote-bitmap'n'> ip async g <pool-name>
[note: zpool create not done on secondary, as the ZFS storage pool from the SNDR primary is replicated here by SNDR]
[on primary host]
sndradm -g <pool-name> -nu
Caution: Only "zpool import <pool-name>" a replicated storage pool, when SNDR replication is in logging mode. Make sure to "zpool export <pool-name>" a replicated storage pool before resuming SNDR replication. Failure to adhere to this requirement may lead to ZFS detectable corruption, or a system panic. ZFS is not a shared filesystem, and an actively replicating ZFS storage pool is being shared.
Please read the sndradm(1m)man page, and the complete AVS documentation at http://docs.sun.com/app/docs?p=coll%2FAVS4.0
Hi all,
i have one production database which replicates to secondary servers. I have read in the documentation one to many replication is possible using AVS. Each secondary host should have its own bitmaps even in the primary server. yes this has been taken into consideration. when both replication is enable the second one does not starts. it thows an error like:
Sep 17 14:00:30 sndr: sndradm -e sun1 /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 nswc
np /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3
SNDR: Cannot add sun1:/dev/rdsk/c1t5d0s0 ==> nswcnp:/dev/rdsk/c1t5d0s0 to group
pool1
please note; i am replication the same pool from production to two secondary hosts at the same time.
Please help,
sailesh
Posted by sailesh on September 17, 2009 at 07:47 AM EST #
Note that your command "sndr: sndradm -e sun1 /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 nswcnp /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3", makes no explicit reference to the SNDR I/O consistency group name "pool1", but that the error message does.
Since the SNDR primary volume "/dev/rdsk/c1t5d0s0" is already a member of the I/O consistency group "pool1", this volume when enable, must also be in the I/O consistency group "pool1". This is done by addition the "-g pool1" option to the "sndradm -e ...", enable command.
Posted by James Dunham on September 17, 2009 at 08:25 AM EST #
Hi james,
Thanks for your reply.
Here is my file on both server.
Sun1 /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 nswcnp /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 ip async g pool1
v890_live /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 v890_bk /dev/rdsk/c1t5d0s0 /dev/rdsk/c1t5d0s3 ip async g pool1
Sun1 and v890_live are same servers and nswcnp and v890_bk are two secondary server.
Ipaddress of sun1: 192.162.163.25
Nswcnp 192.168.160.10
V890_bk 192.162.162.26
Posted by sailesh on September 17, 2009 at 09:51 AM EST #