Tuesday September 28, 2004

SMF/ Predictive Self Healing: svcadm(1)

Today we'll take a look at SMF's main administrative tool, svcadm(1M).  With this tool, you will enable, disable, and maintain your services.

enable/disable

Let's start with a commonly used service, ssh.

[straylight] % svcs network/ssh
STATE          STIME    FMRI
online         Sep_24   svc:/network/ssh:default

From another machine, we can verify that everything is running fine:

[proxima-centauri] % ssh straylight
Password:

As you can see, ssh is enabled and running on my machine.  Now let's disable it.

[straylight] % svcadm disable network/ssh

[straylight] % svcs network/ssh
STATE          STIME    FMRI
disabled       0:56:48 svc:/network/ssh:default

Now the service reads as disabled, and we can see:

[proxima-centauri] % ssh straylight
ssh: connect to host straylight port 22: Connection refused

...that it actually is.  Turning the service on is just as easy.

[straylight] % svcadm enable network/ssh

[straylight] % svcs network/ssh
STATE          STIME    FMRI
online         0:58:07  svc:/network/ssh:default

Note that it's now online.  (The time changes each time we execute an administrative action)

Enable and disable have some extra options that are quite useful.  Say we enabled ssh, but that we noticed that it was offline:

[straylight] % svcs network/ssh
STATE          STIME    FMRI
offline        1:00:47  svc:/network/ssh:default 

We know from my previous post that offline means that the service is enabled, but something it depends on is missing (either disabled, or offline).  Let's see what is wrong:

[straylight] % svcs -d network/ssh:default
STATE          STIME    FMRI
disabled       1:00:24  svc:/system/cryptosvc:default
online         Sep_24   svc:/network/loopback:default
online         Sep_24   svc:/system/filesystem/usr:default 

Ah hah, cryptosvc isn't enabled.  You might have a service with lots of dependencies that are disabled, or you might have dependencies disabled many levels deep.

Do you want to walk through all those services, find out why they're not on, and enable every dependency by hand?  Of course you don't.  So svcadm has a "recursive enable" option that goes through and enables everything that your service depends on.

[straylight] % svcadm enable -r network/ssh


[straylight] % svcs network/ssh
STATE          STIME    FMRI
online         1:02:23  svc:/network/ssh:default


[straylight] % svcs -d network/ssh:default
STATE          STIME    FMRI
online         Sep_24   svc:/network/loopback:default
online         Sep_24   svc:/system/filesystem/usr:default
online         1:02:22  svc:/system/cryptosvc:default

As you can see, we recursively enabled not only ssh, but everything it depended on, allowing it to come online.

One last option of note for enable/disable is the "temporary" option.  Say that you want to enable/disable a service just for this session, but have it revert to its previous state on reboot, in case there are problems.  If ssh is disabled and you issue:

[straylight] % svcadm enable -t network/ssh 

The enable will only be temporary.  If you reboot the machine, the service will once again be disabled.

refresh

Refresh serves two purposes.  One is if you've changed any of the properties of your service, say that you've added a dependency or changed the timeout for starting, you refresh the service, and the properties become active.  The other purpose is that there's an optional method, in addition to "start" and "stop", called "refresh" that you can define.  If your daemon can be sent a HUP signal to re-read its configuration file, you put this in the refresh method, and when you refresh the service, this method is called.

A good example of this is DHCP.  If you change one of the parameters in dhcpsvc.conf, you issue:

[straylight] % svcadm refresh network/dhcp-server 

... and your changes become active.

restart

Restart is pretty self evident.  Restarting a service means that you stop it and start it again.  Where in the past you might have issued a /etc/init.d/sendmail stop followed by /etc/init.d/sendmail start, now you would use:

[straylight] % svcadm restart network/smtp:sendmail 

... which will restart sendmail.

mark (degraded | maintenance)

Mark is used to force a service into a certain state.  (The states are here if you've forgotten them)  An administrator might want to force a service into the maintenance state to let other administrators know that there's something wrong with it that needs to be addressed before it's started again.  You can force a service into either maintenance (which will shut the service down) or degraded (which will leave it running, but let others know that it's running in a degraded state).

Keeping with our earlier example of ssh:

[straylight] % svcadm mark maintenance network/ssh

[straylight] % svcs network/ssh
STATE          STIME    FMRI
maintenance    1:12:47  svc:/network/ssh:default

clear

Clear is used to "reset" the state of a service, and have it be re-evaluated.  For example, say that syslog is in maintenance:

[straylight] % svcs system/system-log 
STATE          STIME    FMRI 
maintenance    1:15:33  svc:/system/system-log:default 

You debug the problem, and realize that syslog failed to start because someone had accidentally deleted syslog.conf, which syslog needs to start.  It attempted to start, saw that the conf file was missing, and fell into maintenance.  You repair the file, and issue a clear:

[straylight] % svcadm clear system/system-log


[straylight] % svcs system/system-log 
STATE          STIME    FMRI 
online         1:25:07  svc:/system/system-log:default

Summary

So now you know how to perform basic maintenance on a Solaris 10 machine using SMF.  I hope it's clear that this system of administration is quite easy, and incredibly powerful.  No longer do you have to hunt around for daemons and init scripts, every service is given a unique FMRI, administered through a unified framework.  This, combined with explicit states and dependencies, gives administrators flexibility and power that is unavailable in other Unix distributions.

My next post will be about manifests, which are the XML files used to describe each service.  We'll examine a manifest in depth, and take a look at the properties and the dependencies that make it up.  As always, questions and suggestions are welcome.