The Availability Suite product set, or AVS by those that work with only in acronyms, has been a product set at Sun Microsystems Inc. since the time of Solaris 2.6. Although AVS has supported host-based replication (using SNDR) and snapshots (using II) for UFS, QFS and other Solaris filesystems, the existence of both ZFS and AVS in OpenSolaris opens new doors of opportunity for both of these technologies.
Since ZFS brings the features and functionality of a filesystem and a volume manager together, and offers its own implementations of snapshot and data replication, these latter two ZFS features do not preclude the use of Availability Suite with ZFS. Both products offer unique snapshot and replication features and functionality that work well together, not against each other.
Unit of Replication
As with any Solaris supported filesystem, AVS provides the means to establish both snapshot and remote volume replication, and with ZFS this is no different. With ZFS being both a filesystem and volume manager, there are a couple of things that one must be aware of when using AVS and ZFS together.
The component of replication or snapshot is a 'named' ZFS storage pool, not a single ZFS filesystem within a pool. Since ZFS filesystems share all the space associated with a single storage pool, all the storage must be configured in the same I/O consistency group, a feature of both SNDR and II.
Although a ZFS storage pool consisting of one volume does not require an I/O consistency group, using one does not impact AVS. This is a good practice to establish, especially if the ZFS storage pool is likely to grow over time. Prior to adding a new volume to an existing ZFS storage pool, first configure it within AVS, adding it to the I/O consistency group.
Initial Synchronization (Resilvering)
When creating a ZFS filesystem that will be replicated with SNDR, it is best to enable the SNDR volumes first, then ZFS volumes second. Because AVS is filesystem agnostic, when enabling volumes for remote replication, the software does not know if the volume(s) being configured are near empty (a new filesystem), partially or near full. Given no further information, SNDR is required to replicate the entire volume, or volumes
Finding a means to avoid the unneeded replication makes sense, as why would one want to replicate GBs or TBs, if portions of the volumes have no valid data in uninitialized blocks.
An SNDR enable option (-E), indicates that the local and remote volumes are equal, and thus forgoes the initial volume synchronization process. The volumes are considered equal because uninitialized data = uninitialized data. When the subsequent 'zpool create ... ' and 'zfs create ... ' commands are issued, the write I/Os issued by ZFS on the local node, will cause the create operations to get replicated to the remote node.
If there are existing ZFS filesystems within ZFS storage pools, that contain valid data, one would assume that SNDR would have no choice but to replicate the entire ZFS storage pool. The good news is there is a feature of ZFS, zpool replacement, makes the sum of the parts (AVS & ZFS), greater then the whole!
To utilize this ZFS feature, one would setup a duplicate set of uninitialized volumes on the local node, sized to matching those in the pre-existing ZFS storage pool one wishes to replicate, then enabling SNDR using the (-E) equal option. By issuing 'zpool replace ... ' specifying these uninitized volumes, the internal ZFS processing of zpool replacement, moves only the valid data from the old volumes to the new volumes (a ZFS resilvering process), allowing SNDR to detect and replicate only the valid data to the remote host. When ZFS is done with its zpool replace processing, the old volumes are automatically removed from the ZFS storage storage pool and can be reclaimed.
For ZFS files systems that already consume a fair amount of the ZFS storage pool, the need to perform any synchronization processing can still be both time and resource consuming. Fortunately there is one option left that also uses the SNDR (-E) option.
Make a tape or other media backup of the physical ZFS storage pool, a pool that has been placed in a 'zpool export ...' state. (Note: II can be used to eliminate this tape backup window, but that that is a subject for another time.) Prior to performing the 'zpool import ...', enable the local and remote SNDR volumes, again using the SNDR (-E) option. Don't forget to also enable I/O consistency groups if needed. The ZFS filesystem(s) can now be used while sending the tape backup using an overnight carrier to the remote site, and restoring the data onto the remote volumes. During this unspecified period of time, SNDR has been tracking all changes being made by ZFS on the primary site. Once the backup has been restored (and verified) at the remote site, SNDR can be placed in replicating mode, and just those changes that happened while the backup tape was in transit, need to be replicated by SNDR. This by fair cuts down on the amount of data that needs to be replicated.
These aforementioned issues with initial synchronization using SNDR, do not apply equally to II. First there is no remote replication costs, as all the I/O is done locally, and for all II shadow volume types except independent shadows, there is no initial resilvering process needed. Even when using II's independent shadows, all II snapshot functionality is instant, (the II name is for Instant Image), meaning that both the master and shadow volume are instantly accessible while the background synchronization is active. The impact of performing a full ZFS storage pool copy is somewhat hidden when using II independent shadow volumes, a non-existant for dependent shadow volumes.
This is just the tip of the AVS & ZFS iceberg, as there are many features of Availability Suite, ZFS and other Solaris data path technologies, that will collectively make OpenSolaris the storage platform of choice.