An image of a tape drive in an Ultra 80

With a slight hint of cruelty (I believe) another of the things the guys wanted me to look at when joining the ZFS testing team, was to see whether existing backup programs can correctly back up the contents of ZFS file systems.

I say cruelty, because having never messed about with these sorts of enterprise backup programs before, I just happily agreed to take on the task and got on with it, without knowing what a pain in the neck these things were to get running. Apart from just the traditional tools like tar and cpio, we were interested in seeing whether Veritas NetBackup would be able to backup files on ZFS. I should point out, that this was in no way detailed, in-depth testing of Veritas NetBackup with ZFS - more that it was sanity testing on our behalf to see that nothing untoward was happening. ISVs tend to do their own rounds of qualification to ensure their software runs on Solaris (though Sun will do all we can to help those guys out: there was an article on BigAdmin about this a while back)

Anyway, after a certain amount of whinging (and a little bad language - sorry Mum!) I finally got NetBackup working and backing up files (though to this day, I can't seem to persuade it not to eject tapes after completing a backup job). I was happy to see that, yes - it works! Now, I should qualify that statement a little more : it backs up and restores both simple files and files with extended file attributes with no trouble.

Things got a bit more complex when I looked at files with ACLs (access control lists). ZFS has a new implementation of ACLs, modelled on NFSv4 ACLs that Mark, Sam and Lisa have talked about in the past. These have more features than the older POSIX-draft ACLs (but there's a tin opener on my desk here, and I'm absolutely determined not to use it on that container of annelids). To cut a long story short, I found that NetBackup wasn't able to backup/restore files with these more complex ACLs - the files are backed up and restored, but the ACLs get silently dropped, so there's a bug logged against Veritas to get them to fix this. If ACLs aren't important to you though, then you probably don't need to worry.

Of course, thankfully there's a really simple workaround for this problem - and indeed it works for any other backup mechanism that's finding ZFS a bit hard to cope with. One of the features of the zfs(1M) command is the ability to do backup/restore. ZFS writes it's own archive format, which can then be backed up normally by any backup program you like, and then restored again. There's a good example of this in the man page, which I'll quote here :

Example 10. Using backups

The following commands sends a full backup and then an incremental to a remote machine, restoring them into poolB/restored/fs@a and poolB/restored/fs@b, respectively. poolB must contain the file system poolB/restored, and must not initially contain poolB/restored/fs.

     # zfs backup pool/fs@a | \
        ssh host zfs restore poolB/restored/fs@a
     # zfs backup -i pool/fs@a pool/fs@b | ssh host \
        zfs restore -d poolB/restored/fs
   

So there you go: ZFS backup - just like the rest of ZFS, it's really simple to use. Now all you have to remember, is to actually do your backups! (as one of my college lecturers, Joe Carthy mentioned in quintuplicate several years ago during one of our first lectures with him : "Always make a backup!" - and indeed, he advised that whenever you start using a new computer network, you should find out who's responsible for doing the backups, get to know them well and always try to be in their good books, just in case! ;-)


Comments:

I'm sorry, but the built-in ZFS backup capability is just not enterprise-level-ready yet. It won't handle end-of-media on a tape, and very little has been publicly said about how to recover when one of these network pipes goes down in the middle of a backup|restore operation. Furthermore, if you really did try to use incremental backups this way, you would need a mechanism outside of the basic one-liner commands to track where you are in the cycle of full and incremental backups, and restart from a safe place. I'm sure Sun will eventually plug these and other holes, but to leave the reader with the impression that "all you have to do" is follow the sample commands is very misleading.

Posted by Glenn on December 14, 2005 at 07:29 PM GMT #

Hmm, I certainly didn't mean to give the impression that ZFS contained an <em>enterprise</em> backup mechanism : it doesn't yet - we still do need to rely on the abilities of Networker, or Netbackup - I was merely documenting the fact that you can (with workarounds) currently use Enterprise backup tools with ZFS. For simple user-initiated backups though, I believe the capabilities in ZFS are sufficient. They work for me.

Posted by Tim Foster on December 15, 2005 at 10:07 AM GMT #

This is not suprising, as NetBackup essentially uses GNU tar to back up and restore files. The ZFS ACL stuff hasn't been implemented there yet.

Posted by Craig Steinberger on December 21, 2005 at 06:13 PM GMT #

More in the release notes at http://docs.sun.com/app/docs/doc/819-5947/6n80cmc6l?a=view#gcumk

Posted by Tim Foster on August 30, 2006 at 02:56 PM IST #

If you use networker, http://www.inserve.se/edu/using-legato-networker-to-backup-zfs/ Can be used as a workaround.

Posted by Niklas Paulsson on October 27, 2006 at 07:11 AM IST #

Thanks Niklas! I believe Networker 7.32 is available now which as you mention fixes this. JamesD has more on this at: http://uadmin.blogspot.com/2006/10/emc-networker-732-supports-zfs.html

Posted by Tim Foster on October 27, 2006 at 10:05 AM IST #

I do not have a zfs backup option? I'm obvioulsy missing something big here.. is everyone having a party without me? Mark.

Posted by Mark Davies on March 04, 2007 at 10:10 PM GMT #

Good point Mark, you're absolutely right. The "zfs backup" command was renamed "zfs send", as an acknowledgment that zfs backups of this sort don't provide the complete solution that folks regularly doing backups need. However, I believe they are a good starting point that can be used to layer more complex backup/restore functionality on top of. "zfs recv" is the new name for the old "zfs restore" command. Both are documented in the ZFS admin guide, linked from the opensolaris.org zfs docs page.

Posted by Tim Foster on March 04, 2007 at 11:36 PM GMT #

Almost 4 years after this was posted, has the situation changed much? Is there now a decent backup/restore system in ZFS that avoids the need to pay loads of money for backup software?

# zfsdump
zfsdump: not found
# zfsrestore
zfsrestore: not found

Dave

Posted by David Kirkby on August 24, 2009 at 10:24 AM IST #

Is is possible to restore a ufsdump backup to a zfs system? My trusty Blade-1500 appears to have died, and all my backups are in ufsdump format.

Posted by Tim Evans on September 02, 2009 at 02:07 PM IST #

Hi Tim, you can't ufsrestore to a zfs filesystem directly: you may have some luck though ufs restoring to a UFS filesystem created on a zvol of sufficient size, then copy the files back over to zfs assuming you've got the capacity?

Posted by Tim Foster on September 02, 2009 at 03:53 PM IST #

Or NFS-share the zfs filesystem, and mount it on a system with ufsrestore?

Posted by Tim Evans on September 02, 2009 at 04:05 PM IST #

Nope, don't think that'll work - my impression of ufsdump is that it works at a raw ufs filesystem level (so writes inodes, etc.) rather than being a generic file-archiver like tar, cpio and the like.

Posted by Tim Foster on September 02, 2009 at 04:28 PM IST #

Post a Comment:
  • HTML Syntax: NOT allowed

This blog copyright 2009 by timf