ZFS Automatic Snapshots 0.10

I set up exactly the same manually 3 weeks ago with cron jobs (no SMF). Unfortunately I was not aware of the thread in the discussion groups.
From this practice comes an RFE:
'zfs list' needs a flag to exclude snapshots in the listing; or better: a flag to include them and exclude by default (the interface is documented to be evolving, right?). With automatic snapshots the list gets loooong. I have autosave-00, autosave-05 ... -55 every 5 min, hourly-00 ... -23, daily-monday-1 ... -7, weekly-00 ... -53 for every FS exept swap, that's hundredth of snapshots after 3 weeks.

Posted by Paul on January 23, 2008 at 12:15 AM GMT #

hi Paul,

I know exactly what you mean. There's a "-t" flag to the zfs command that does what you're after:

timf@haiiro[40] zfs list | wc -l
276
timf@haiiro[41] zfs list -t filesystem,volume | wc -l
34

Posted by Tim Foster on January 23, 2008 at 09:22 AM GMT #

Hello Tim,

Thanks for your continued work on this project! It has been very helpful. I upgraded from a heavily modified 0.8 to this 0.10 version with minimal changes. The prefix and separator customization is very good.

I still had to remove $SEP in line 275 because I prefer snapshot names such as daily-1, daily-2, etc. without any prefix, but that's the only out-of-the-ordinary change I did. Keep up the great work!

Posted by Siegfried Leonard on January 24, 2008 at 09:15 AM GMT #

Hi Tim,
I've just started using your excellent tool. Is there a way for me to call the GUI tool over an X tunnel? That is, I'm connecting to my Nevada box with 'ssh -X user@host' and want to have an quick and dirty method for adjusting settings, by calling something like 'gnome-snapshot-tool' or whatever.

thanks,
blake/

Posted by Blake Irvin on January 28, 2008 at 10:50 PM GMT #

Cool - glad you're finding it useful Blake. Yes, you can run:

$ ssh -X user@host /usr/bin/zfs-auto-snapshot-admin.sh simple

will give you the basic GUI that lets you configure which filesystems should have snapshots taken under the frequent, daily, hourly, weekly and monthly schedules. (the GUI behind the scenes, just sets some given zfs user-properties on the filesystems of interest)

Running:

$ ssh -X user@host /usr/bin/zfs-auto-snapshot-admin.sh <filesystem>

will start the wizard which walks through the steps of creating a new instance of the service which can take periodic snapshots of the given filesystem. The instance manifest file will be created on the remote system, but you'll need to "svccfg import" it on the remote system in order to start using it.

Finally, to enable/disable instances of the service remotely via a GUI, you can use webmin - as root on the server, run:

# /usr/sfw/bin/webminsetup

answer the questions, then connect to

https://server:10000/webmin

and select the "Service Management Facility Configuration". Hope this helps?

Posted by Tim Foster on January 29, 2008 at 11:18 AM GMT #

Thanks Tim. I've gotten the default 'frequent' and 'hourly' snapshots working. The longer-term ones aren't yet, but I'm still investigating that.

Posted by Blake Irvin on February 06, 2008 at 09:53 PM GMT #

Hi Tim,

I'm using your automated snapshot service and I find it really usefull, I'm just curious why sometimes a snapshot does not show how much space it takes.

I mean, I have it set up so that it uses the standard 30 days, 7 weeks, 12 month schedule over a filesystem. It sems to me that when more than one snapshot starts at the same time zfs does not show occupied space. Look here:

nas 260G 881G 11,2G /nas
nas/nwserv 11,2G 881G 6,80G /nas/
nwserv
nas/nwserv@zfs-auto-snap:daily-2008-02-27-00:00:00 519M - 6,99G -
nas/nwserv@zfs-auto-snap:daily-2008-02-28-00:00:00 491M - 7,02G -
nas/nwserv@zfs-auto-snap:daily-2008-02-29-00:00:00 0 - 6,91G -
nas/nwserv@zfs-auto-snap:weekly-2008-02-29-00:00:00 0 - 6,91G -
nas/nwserv@zfs-auto-snap:daily-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:monthly-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:weekly-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:daily-2008-03-02-00:00:00 241M - 6,94G -
nas/nwserv@zfs-auto-snap:daily-2008-03-03-00:00:00 242M - 6,94G -
nas/nwserv@zfs-auto-snap:daily-2008-03-04-00:00:00 538M - 6,96G -
nas/nwserv@zfs-auto-snap:daily-2008-03-05-00:00:00 499M - 6,74G -

When the weekly and monthly snapshots are taken there is no occupied space in the listing.

Is this something to report to zfs storage forum or does this depend on your service?

Thanks a lot.

Maurilio.

Posted by Maurilio Longo on March 06, 2008 at 12:12 PM GMT #

That's interesting - you're sure you're looking at space the snapshot is actually using, vs. space the snapshot is referencing: a snapshot only starts using space when the blocks it references are deleted or changed.

http://docs.sun.com/app/docs/doc/819-5461/6n7ht6qsb?a=view#gbcxc

- would that explain it ? Based on your schedule though, it looks like dates after 2008-03-01 are using space, so it's weird that 2008-03-01 isn't. Could be worth mailing zfs-discuss about...

Posted by Tim Foster on March 06, 2008 at 12:21 PM GMT #

Tim,

the header of the column says USED and for days when I have only the daily snapshot it is correct

# zfs list
NAME USED AVAIL REFER MOUNT
POINT
nas 264G 877G 11,2G /nas
nas/nwserv 11,5G 877G 6,80G /nas/
nwserv
nas/nwserv@zfs-auto-snap:daily-2008-02-27-00:00:00 519M - 6,99G -
nas/nwserv@zfs-auto-snap:daily-2008-02-28-00:00:00 491M - 7,02G -
nas/nwserv@zfs-auto-snap:daily-2008-02-29-00:00:00 0 - 6,91G -
nas/nwserv@zfs-auto-snap:weekly-2008-02-29-00:00:00 0 - 6,91G -
nas/nwserv@zfs-auto-snap:daily-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:monthly-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:weekly-2008-03-01-00:00:00 0 - 6,93G -
nas/nwserv@zfs-auto-snap:daily-2008-03-02-00:00:00 241M - 6,94G -
nas/nwserv@zfs-auto-snap:daily-2008-03-03-00:00:00 242M - 6,94G -
nas/nwserv@zfs-auto-snap:daily-2008-03-04-00:00:00 538M - 6,96G -
nas/nwserv@zfs-auto-snap:daily-2008-03-05-00:00:00 499M - 6,74G -
nas/nwserv@zfs-auto-snap:daily-2008-03-06-00:00:00 378M - 6,80G -
nas/rsync_clienti 7,90G 877G 7,90G /nas/
rsync_clienti
nas/samba 233G 877G 233G /nas/
samba
nas/test_UP 44,0K 877G 44,0K /nas/
test_UP

I'll post a comment on storage discuss.

Thanks.

Maurilio.

Posted by Maurilio Longo on March 06, 2008 at 03:32 PM GMT #

Hi Tim. I've been using your auto-snapshot service with much success. I'd like to suggest, however, that the value for 'SEP' in the svc method be changed to something other than a ':' - :'s mess up Samba clients pretty badly :) I'm using '_' instead.

cheers,
Blake

Posted by Blake Irvin on March 10, 2008 at 08:28 PM GMT #

Yeah, I am using periods for separators. Not only Samba clients have difficulties with colons, but older Mac OS clients use : as the path separator and thus either get mangled names, or do not see anything.

Posted by Siegfried Leonard on March 11, 2008 at 06:11 AM GMT #

Hello,

i found bug (maybe it isn't bug). I use this script for creating snapshots of volumes (zvol), not filesystems.

In this case script doesn't work, because function "get_snapshot_datasets" contains command: "zfs list -t filesystem -o name,com.sun:auto-snapshot:$LABEL" where is set type to filesystem.

But this is little bug in this great script :)

Thanks,

Dan

Posted by Dan on March 13, 2008 at 02:29 PM GMT #

Good catch Dan - I'll have that fixed in the next version! Glad you're finding the script useful.

Posted by Tim Foster on March 13, 2008 at 02:51 PM GMT #

Just a little thought: could you somehow detect if dataset has not been changed since the last snapshot and avoid taking the snapshot in that case ?

Posted by Vladimir on April 07, 2008 at 08:49 AM IST #

One more nit: the pkgmap sets /usr/bin to root:sys while I believe on Nevada it is root:bin by default.

Posted by Vladimir Kotal on April 07, 2008 at 09:03 AM IST #

Thanks Vladimir, - just looking, I have:

timf@haiiro[3] grep "/usr/bin d" /var/sadm/install/contents | more
/usr/bin d none 0755 root sys SUNWcsu SUNWcsl...

- admittedly, I'm still on nv_82. I'll check this out for the next release though.

As for only taking snapshots when the dataset has changed, that's hard - I don't know of a way for doing this using the current ZFS CLI - don't really have time to investigate libzfs yet, but feel free to post an update if you get a chance to look into it :-)

Posted by Tim Foster on April 07, 2008 at 09:48 AM IST #

It would be great to make the date optional. There is already a 'creation' property, so this value is basically redundant. Much better to have rotating snapshot names with simple #s. It's more legible and for most cases easier to navigate IMHO.

Posted by Randy Bias on April 20, 2008 at 04:05 AM IST #

I've been playing around with your SMF snapshot script, and I have to congratulate you on some great work. I'm a GUI kind of guy, so I really appreciate the configuration panel. By the way, can your app. be installed in Indiana? I'm not sure whether Indiana contains the necessary dependencies (such as the SMF service) for it to run?

My current backup idea would be to rsync over SSH from my source machines (Macs) to the Solaris or Indiana-based server. My backup script (running on the Mac) would then issue a create snapshot command via SSH to the Solaris server (I think this would work). This leaves the snapshot management function. I would preferably like to find something of a cross-platform solution should I migrate from Solaris to OpenSolaris (or even OS X or FreeBSD) in the future.

I haven't been able to find a script that would allow the specification of number of hours/days/weeks/months to keep, rotate the snapshots, and then delete old ones. Can your service be used solely for the snapshot management feature (and not create new snapshots)? If you know of such a script, or if your service can do strictly this feature, I would be greatly appreciative.

Posted by Jeff on May 02, 2008 at 08:46 AM IST #

Hi Jeff,

Thanks for the kind words! Yes, this service can be used in Indiana. In terms of purely managing snapshots, not taking them, this script probably isn't what you're after. It does do rotation of snapshots, but only considers the snapshots that it has taken (ie. ones that match a particular pattern) as candidates for deletion when the roll-over date arrives.

It's a pretty basic hack to make it consider all snapshots on the system as candidates for deletion though - check out the "destroy_older_snapshots" function in /lib/svc/method/zfs-auto-snapshot, around line 395 : take out the command to grep for the given pattern, and use zfs list -s to sort by you might want to use zfs list -s creation to sort by creation date.

Posted by Tim Foster on June 15, 2008 at 01:27 PM IST #

Hi Tim,
hi all,

first of all: Tim, thank you very much for your very useful auto-snapshot script.
I believe this really is what every ZFS user needs.

As a laptop user, I wanted to make sure that I get snapshots taken even if my
machine is down every night, so I added at scheduling support to zfs-auto-snapshot,
where each job re-schedules it's next execution, thus ending up with a cron-like
behaviour.

The main advantage is that also the standard Solaris cron runs at jobs retrospectively,
so we can live without anything like anacron.

The main disadvantage is that with current onnv versions, at jobs are broken in cron,
so before working on zfs-auto-snapshot, that's what I had to fix (or at least revert
to old code) first (see
http://www.opensolaris.org/jive/thread.jspa?threadID=64739 ).

I have uploaded zfs-auto-snapshot with my changes to http://www.opensolaris.org/jive/message.jspa?messageID=251976&tstart=0
because I cant post file to this blog (or can I?).

Tim, feel free to integrate my suggestions or not, I wont feel offended if you
don't, but at any rate I am very happy that you maintain this tool.

Thanks again, Nils

Posted by Nils Goroll on June 24, 2008 at 10:56 PM IST #

Tim - thanks for the snapshot script; I'm using 0.10 on a couple of x4500's; very useful.

I just pushed one into production this past weekend and noticed today some of my snapshots were not quite set up right, so I went about trying to get them properly configured.

What I'm seeing now is that after using the GUI via "zfs-auto-snapshot-admin.sh" and constructing a new snapshot setup, that when I do an "svccfg import auto-snapshot-instance.xml" and then "svcadm enable svc:/system/filesystem/zfs/auto-snapshot:vol0,daily", that this is not being reflected in cron.

The couple I've got showing up in cron can be enabled/disabled via svcadm, but new ones I create with the GUI don't seem to be showing up.

I'm not sure where to look to see about this cron issue. Any suggestions?

Posted by John Stewart on July 07, 2008 at 08:28 PM IST #

That's weird John. Any error messages in /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,daily.log ?

Posted by Tim Foster on July 07, 2008 at 09:03 PM IST #

Aha (I didn't realize where to look for logging; thank you):

[ Jul 7 12:23:51 Rereading configuration. ]
[ Jul 7 12:23:54 Enabled. ]
[ Jul 7 12:23:54 Executing start method ("/lib/svc/method/zfs-auto-snapshot start") ]
/lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,daily
crontab: error on previous line; unexpected character found in line. crontab: errors detected in input, no crontab file generated.
[ Jul 7 12:23:54 Method "start" exited with status 0 ] [ Jul 7 12:24:23 Stopping because service disabled. ]

Not sure yet what is different/new; I don't believe I've used any special characters. Trying now to start over.

Posted by John Stewart on July 07, 2008 at 09:16 PM IST #

I removed all of the snapshot services and the TIMFauto-snapshot package, then re-installed and re-added them using the GUI. Strangely, it seems to work now.

In the last few weeks, I did back out a kernel patch then install an IDR to deal with the Marvell bugs on the X4500's, so perhaps something got funked up as a result of that.

Posted by John Stewart on July 07, 2008 at 09:43 PM IST #

What's wrong with a bash script called by a cron job? I don't get setting this up as a service. I also don't get running make on something that appears to be a few shell scripts. I attempt to run make and it just errors that pkgmk is not found.. apt-cache search does not show where I would find it. Wouldn't it make sense to combine the shell scripts and ditch make?

I'm currently running a simple bash script from cron that makes daily snapshots. I'll probably just update it to include hourly, weekly, and monthly. At least I picked up the avoidscrub tip, thanks!

Posted by Michael Bushey on July 15, 2008 at 02:22 PM IST #

Everyone's a critic - ok here goes. The point of this, is that you shouldn't have to understand cron in order to have this functionality on your system. I'm using SMF to provide a single point of control, and that if the snapshotting fails for some reason, you get a useful administrative interface to help work out what's gone wrong.

SMF also separates the log files for each service and (should cron fail for some reason) it'll also mark these snapshot services as being offline.

You're absolutely right though that cron and a simple shell script can do the job, it just depends on what extra functionality you're after and how much time you've got to write and maintain your own shell script.

The makefile that's there creates SVR4 packages for Solaris - here's where I have the packaging commands:

timf@beag[367] pkg search pkgmk
INDEX ACTION VALUE PACKAGE
basename file usr/bin/pkgmk pkg:/SUNWpkgcmds@0.5.11-0.91

The shell scripts here are separate things and it doesn't really make sense to combine them - the method script which adds/removes the cron jobs and actually does the work, and the others as gui tools to allow more easy administration.

Btw, I believe the avoidscrub bug has been fixed in a recent putback. Anyway, glad you found this stuff useful in some way! :-)

Posted by Tim Foster on July 15, 2008 at 02:45 PM IST #

Just a quick note:

I too am seeing frequent transitions into maintenance, e.g. 'Stopping for maintenance due to administrative_request' with no other datapoints anywhere in the logs as to what may have caused that. Strange stuff.

Looking forward to 0.11 ;)

B

Posted by Brett on August 16, 2008 at 07:37 PM IST #

I've been meaning to post about the same issue Tim just did - more and more frequently the service is dropping to maintenance. The cause seems to be being unable to destroy a snapshot.

Relevant snippet from messages:

Aug 16 02:19:16 titan zfs-auto-snap: [ID 702911 daemon.notice] Error: Unable to destroy vol0/project@snap-hourly-2008-08-14-08-00-00
Aug 16 02:19:16 titan zfs-auto-snap: [ID 702911 daemon.notice] Moving service svc:/system/filesystem/zfs/auto-snapshot:vol0,hourly to maintenance mode.
Aug 16 02:19:16 titan svc.startd[7]: [ID 748625 daemon.error] system/filesystem/zfs/auto-snapshot:vol0,hourly transitioned to maintenance by request (see 'svcs -xv' for details) ~ ~

In all cases a "svcadm clear" on the service will fix the error.

I think actually this may have been the problem I noted above where the entries were missing from cron - the service may have been in maintenance mode and I hadn't noticed that.

Posted by John Stewart on August 16, 2008 at 08:20 PM IST #

Interesting comments - I too have seen cases where the service has been inexplicably unable to destroy a snapshot - manually trying to destroy a snapshot using exactly the same command the service uses would then succeed. John or Brett - any chance you could provide more info on what OS you're running, and whether you can discern a pattern from these indestructible snapshots ?

I'm not sure 0.11's going to fix the problem, because I'm not yet convinced it's a problem with the service, but I could change things so we don't drop to maintenance, but instead make lots of noise in the logs whenever this happens - as those snapshot start to mount up, it might start pointing to a pattern. Sound like a plan?

Posted by Tim Foster on August 16, 2008 at 10:56 PM IST #

The OS is Solaris 10 on an x4500, uname -a is:

SunOS titan 5.10 Generic_127112-11 i86pc i386 i86pc

It's got the IDR on it for the Marvell bug issue.

I'd say if you could just set up the snapshot scripts to not give up on the first failure to delete a snapshot, and try again in a minute, that would go a long way to making this a non-issue.

Since last week, no more failures... not discerning a pattern yet.

Posted by John Stewart on August 25, 2008 at 09:32 PM IST #

Hey Tim
Thank you for your work .
I just noticed that when giving for exemple <test> as instance name using your GUI, i got the output below;
I had problem trying to import .
I get rid of it by replacing the two strings '-mypool-myzone-,test' by 'mypool-myzone-test'.
It seems that svccfg don't like instance name with comas:-)
I hope this will help someone.

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type='manifest' name='-mypool-myzone-,test'>
<service
name='system/filesystem/zfs/auto-snapshot'
type='service'
version='0.10'>
<create_default_instance enabled='false' />

Posted by issa on September 24, 2008 at 11:04 AM IST #

Dear Tim

Is there a way to get zfs-auto-snapshot-0.10.tar.gz. from your site.
Link is broken now.
I feared the one i got from another site have not a good md5 because i found error like this when trying to install it:

FIRTS TEMPTATION
pkgadd -d /tmp/zfs-auto-snapshot-0.10/

The following packages are available:
1 src ZFS Automatic Snapshot Service
(all) 0.10

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: all
pkgadd: ERROR: unable to open package <src> pkgmap file </tmp/zfs-auto-snapshot- 0.10/src/pkgmap>: No such file or directory
pkgadd: ERROR: unable to install package <src>

1 package was not processed!

SECOND TEMPTATION

pkgadd -d zfs-auto-snapshot-0.10/ -s /var/spool/pkg/

The following packages are available:
1 src ZFS Automatic Snapshot Service
(all) 0.10

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: all
Transferring <src> package instance
pkgadd: ERROR: unable to complete package transfer
- no permission to overwrite existing path </var/spool/pkg//src>

Regards

Posted by Issa Kandji on September 26, 2008 at 10:33 AM IST #

Dear Tim
Don't pay attention to my last messages concerning the installation.
I finally found what i missed.
Just one question:
During installation,Is it normal to get such message :
## Executing postinstall script.
couldn't set locale correctly
couldn't set locale correctly

Regards

Posted by issa kandji on September 26, 2008 at 01:03 PM IST #

Yeah, don't worry about the locale warning messages. I think it's because I had set the locale in the pkginfo file, and not everybody has en_US.UTF-8 installed. Either way, it's harmless.

Posted by Tim Foster on September 26, 2008 at 08:07 PM IST #

Hi Tim.

I'm thinking that it might be smart to modify your method script to look for active scrubs/resilvers with an unprivileged user. This would avoid a bug in Solaris 10 05/08 where a resilver/scrub is restarted with the root user calls for zpool status.

I'm willing to contribute the fix if you are interested in it.

Blake

Posted by Blake Irvin on October 13, 2008 at 11:27 PM IST #

Hi Blake - the new 0.11 code should do just that - the SMF service in 0.11 runs as 'zfssnap', a user with the "ZFS File System Management" profile, so non-root by default. I'd be interested in hearing if that fixes the problem you're seeing?

Posted by Tim Foster on October 14, 2008 at 10:52 AM IST #

That sounds like the right direction to move. How can I track the release of 0.11?

Posted by Blake Irvin on October 14, 2008 at 05:24 PM IST #

0.11's out - in that it'll be in Nevada 100 - you can grab the sources via instructions at:

http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_11

Posted by Tim Foster on October 14, 2008 at 08:29 PM IST #

Hi all, anybody can tell me how to use the backup function of zfs-auto-snapshot tool ? I need to implement incremental snapshot for replication purpose.

Posted by Marc on November 17, 2008 at 12:20 PM GMT #

So we've continued to struggle with the snapshot services dropping into maintenance mode.

A new one... as of this weekend, with nothing logged to messages, all snapshots seem to have stopped being generated.

The services are still showing as online.

This, I would say, is a worse failure mode.

Posted by John Stewart on November 18, 2008 at 05:50 PM GMT #

No errors echoed either to syslog or /var/svc/log/* ? Can you manually run the method script from the command line and tell what's happening, eg.

# /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:frequent

or even,

# ksh -x /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:frequent

Are there entries in crontab for the services ok?

Posted by Tim Foster on November 18, 2008 at 05:56 PM GMT #

I have four snapshot services:

titan[root]:/vol0/.zfs/snapshot> svcs | grep snap
online Nov_03 svc:/system/filesystem/zfs/auto-snapshot:vol0,monthly
online Nov_12 svc:/system/filesystem/zfs/auto-snapshot:vol0,every15minutes
online Nov_13 svc:/system/filesystem/zfs/auto-snapshot:vol0,hourly
online Nov_16 svc:/system/filesystem/zfs/auto-snapshot:vol0,daily

"monthly" last updated on Nov 1, so it's unknown if that is affected.

"daily" last updated on Nov 11 at 15:00:

titan[root]:/> ls /vol0/.zfs/snapshot/snap-daily* | tail -1
/vol0/.zfs/snapshot/snap-daily-2008-11-15-00-00-00:

"hourly" last updated on Nov 18 at 13:00:

titan[root]:/> ls /vol0/.zfs/snapshot/snap-hourly* | tail -1
/vol0/.zfs/snapshot/snap-hourly-2008-11-18-13-00-00:

"every15" last updated on Nov 18 at 13:30:

titan[root]:/> ls /vol0/.zfs/snapshot/snap-every* | tail -1
/vol0/.zfs/snapshot/snap-every15minutes-2008-11-18-13-30-00:

And the logs:

titan[root]:/> ls -l /var/svc/log/*snap* | grep Nov
-rw-r--r-- 1 root root 6707 Nov 16 08:42 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,daily.log
-rw-r--r-- 1 root root 266711 Nov 12 15:34 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,every15minutes.log
-rw-r--r-- 1 root root 22074 Nov 13 11:26 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,hourly.log
-rw-r--r-- 1 root root 690 Nov 3 13:19 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,monthly.log

titan[root]:/> tail -5 !$
tail -5 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,daily.log
[ Nov 15 00:24:46 Stopping for maintenance due to administrative_request. ]
[ Nov 16 08:42:36 Leaving maintenance because clear requested. ]
[ Nov 16 08:42:36 Enabled. ]
[ Nov 16 08:42:36 Executing start method ("/lib/svc/method/zfs-auto-snapshot start") ]
[ Nov 16 08:42:36 Method "start" exited with status 0 ]

titan[root]:/> tail -5 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,every15minutes.log
[ Nov 12 14:29:46 Stopping for maintenance due to administrative_request. ]
[ Nov 12 15:34:42 Leaving maintenance because clear requested. ]
[ Nov 12 15:34:42 Enabled. ]
[ Nov 12 15:34:42 Executing start method ("/lib/svc/method/zfs-auto-snapshot start") ]
[ Nov 12 15:34:42 Method "start" exited with status 0 ]

titan[root]:/> tail -5 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,hourly.log
[ Nov 13 11:18:32 Stopping for maintenance due to administrative_request. ]
[ Nov 13 11:26:11 Leaving maintenance because clear requested. ]
[ Nov 13 11:26:11 Enabled. ]
[ Nov 13 11:26:11 Executing start method ("/lib/svc/method/zfs-auto-snapshot start") ]
[ Nov 13 11:26:11 Method "start" exited with status 0 ]

titan[root]:/> tail -5 /var/svc/log/system-filesystem-zfs-auto-snapshot:vol0,monthly.log
[ Nov 3 12:54:31 Method "stop" exited with status 0 ]
[ Nov 3 13:09:31 Disabled. ]
[ Nov 3 13:19:41 Executing start method ("/lib/svc/method/zfs-auto-snapshot start") ]
mkdir: Failed to make directory "/tmp/zfs-auto-snapshot-lock"; File exists
[ Nov 3 13:19:44 Method "start" exited with status 0 ]

So they all end with status 0.

Posted by John Stewart on November 18, 2008 at 08:47 PM GMT #

Crontab entries appear okay:

titan[root]:/> crontab -l | grep snap
0 0 1 1,2,3,4,5,6,7,8,9,10,11,12 * /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,monthly
0,15,30,45 * * * * /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,every15minutes
0 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23 * * * /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,hourly
0 0 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31 * * /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,daily

And I'm running /lib/svc/method/zfs-auto-snapshot svc:/system/filesystem/zfs/auto-snapshot:vol0,every15minutes
manually to see what happens (no return so far).

However, I just noticed that as of a few hours ago, the hourly/15minute snapshots started working again:

titan[root]:/vol0/.zfs/snapshot> ls /vol0/.zfs/snapshot/snap-hourly* | tail -1
/vol0/.zfs/snapshot/snap-hourly-2008-11-18-14-00-04:
titan[root]:/vol0/.zfs/snapshot> ls /vol0/.zfs/snapshot/snap-every* | tail -1
/vol0/.zfs/snapshot/snap-every15minutes-2008-11-18-14-00-02:

OMG weird!

Posted by John Stewart on November 18, 2008 at 08:53 PM GMT #

Anything being reported to syslog?

Posted by Tim Foster on November 18, 2008 at 10:29 PM GMT #

Thanks for the great program!

Just one question about backup-save-cmd. I try to use remote execution of zfs receive with ssh.
However it appears that zfssnap role under which the service is running can't login to the remote host through ssh.

Permission denied (gssapi-keyex,gssapi-with-mic,publickey,keyboard interactive).

The backup command is
ssh roman(at)172.0.2.10 zfs recv -d stor-serv2 from_serv1

Seems like ssh requesting password for user roman, because it can't find ssh key while starts under zfssnap role.

What should be changed in this case to allow remote login without a password? Add keys for zfssnap? Or maybe the backup command should be different?

Thanks!

Posted by Roman on March 14, 2009 at 01:59 AM GMT #