Wednesday December 07, 2005

Resource Pools will be SMF services

Before I left Sun I had started work on enhancements to Resource Pools to integrate them with SMF. I haven't had much time to work on this since I re-joined Sun, mainly because I now work on the OpenSolaris project and Resource Management is not my main focus. However, I was keen to complete the work as I thought it would be a very worthwhile enhancement for Solaris.

I integrated the changes yesterday and when the code becomes available you'll be able to manage Resource Pools and Dynamic Resource Pools as separate SMF services.

e.g.

bash-3.00# uname -a
SunOS trolls-b8 5.11 onnv-gate:2005-12-07 i86pc i386 i86pc
bash-3.00# svcs 'system/pool*'
STATE          STIME    FMRI
online          1:58:20 svc:/system/pools:default
online          1:58:21 svc:/system/pools/dynamic:default
bash-3.00# svcadm disable pools
bash-3.00# svcs 'system/pool*'
STATE          STIME    FMRI
disabled        2:12:34 svc:/system/pools:default
offline         2:12:36 svc:/system/pools/dynamic:default
bash-3.00# svcadm enable pools
bash-3.00# svcs 'system/pool*'
STATE          STIME    FMRI
online          2:12:48 svc:/system/pools:default
online          2:12:48 svc:/system/pools/dynamic:default
bash-3.00#

I hope it's useful

(2005-12-07 03:16:03.0) Permalink Comments [11]

Thursday January 27, 2005

Zones + Resource Management

I was perusing the Solaris Zones forum the other day and found a useful link to this document which provides examples of using various Resource Management features with Zones.

I think it's a useful document and sheds light on some of the decisions you need to make when using Zones with Resource Management.

(2005-01-27 06:05:36.0) Permalink Comments [2]

Monday August 09, 2004

Using poolbind to execute jobs in different pools

You are lazy. Of course you are, it's a virtue when applied to computing. You don't want to type reams of instructions and check to see if commands succeeded or failed. No way, that's just too time consuming. Joe Salaryman can waste his time doing that, but you've got better things to do.

In that case, here's a minimal script which wraps poolbind(1M) to make it easier to run a process in a particular pool. For example, let's say that I wanted to run ls(1) in pool "listings".

[1]
bash-2.05b$ newpool
usage: newpool <pool name> <command> [parameters]
[2]
bash-2.05b$ newpool listings ls
poolbind: binding pid 100964 to pool 'listings': Not owner
Bind operation failed
[3]
bash-2.05b$ su -
Password:
Sun Microsystems Inc.   SunOS 5.10      s10_63  Jul. 03, 2004
SunOS Internal Development: gk 2004-07-03 [on10_63]
bfu'ed from /bfu/s10_63 on 2004-07-22
Sun Microsystems Inc.   SunOS 5.10      s10_37  May 2004
You have mail.
# bash
[4]
bash-2.05b#  newpool listings ls
backup         bin            Documents      mnt            tmp
bfu            boot           etc            net            TT_DB
bfu.ancestor   cdbuild        export         opt            usr
bfu.child      cdrom          home           platform       var

bfu.conflicts  Desktop        kernel         proc           vol
bfu.parent     dev            lib            rootb
bfu.realmode   devices        lost+found     sbin
bash-2.05b#

1. Run the script to display the usage message. It's fairly clear I think.
2. Now try to run the script as myself. Oops, I'm don't have the privileges to bind to a differnt pool.
3. I'll become root, I know I have enough privileges now ;-)
4. Run it again and this time it works.

#!/bin/sh

if [ $# -lt 2 ]
then
        echo "usage: newpool <pool name> <command> [parameters]"
        exit 2
fi
/usr/sbin/poolbind -p $1 $$
if [ $? != 0 ]
then
        echo "Bind operation failed"
        exit 1
fi
shift
exec $*

Monday August 02, 2004

The nuances of poolbind

poolbind(1M) allows an authorized user to change the resource pool binding of a zone, project, task or process. However, what does this actually mean? As you might imagine, it means different things for the different pool-bindable entities. Here's a quick explanatory note that should help eliminate any confusion.

The main point to note is this: any changes realized through poolbind(1M) are temporal in nature and will only affect the set of processes identified as belonging to the target process collective at that point in time. By this I mean that no permanent configuration is changed.

For processes and tasks, this is a moot point since there is no way (currently) to specify that either a task or a process is associated with a resource pool. I can't really think of any reason why you would ever want such fine-grained resource management that you would want to have a configuration that specified this relationship; so this is unlikely to ever change.

If you want to permanently change the binding of a project or zone to a resource pool then you must edit the configuration (favourite editor for /etc/project(4) or zonecfg(1M) for a zone). If you want the change to take place immediately, rather than the next time you boot your OS, then you must also use poolbind(1M) to execute the temporal change immediately.

So what actually happens when you use poolbind(1M) to modify the binding of a resource pool bind-able entity?
process The process stops consuming the resources associated with its existing resource pool and begins consuming the resources associated with the new resource pool. Any new processes which fork(2) from this process are also bound to the new resource pool.
task All processes which belong to the task stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool.
project All processes which belong to the project stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. However, new processes which join the project after this operation completes will still be associated with the configured resource pool.
zone All processes which belong to the zone stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. Re-booting the zone will restore the configured resource pool binding.

poolbind(1M) is a very powerful workload managment command, especially when applied to a process collective such as a zone or a project. I'll return to the subject of poolbind(1M) later and give some examples of how it can be used to manage workloads quickly and flexibly when problems occur.

(2004-08-02 04:41:30.0) Permalink Comments [6]

Thursday June 10, 2004

Transforming a Resource Pools Configuration XML Document using XSL

Isn't XML wonderful? No! Well maybe it isn't, but it certainly is useful.

Solaris Resource Pools utilities, pooladm(1M) poolcfg(1M), can be used to examine a Resource Pools configuration. Of course, like most text oriented utilities, the output of these commands is designed to be read by a human working with a shell. I thought it would be nice to be able to examine the Resource Pools configuration using a web browser.

In order to do this you need a few things:

• A tool to transform your XML document into HTML
• A web server that can perform XSLT transformations
• An XSL stylesheet which describes the transformation

In Solaris 10, the xsltproc(1) utility can be used to transform XML documents using XSL stylesheets. If you don't have Solaris 10, think about joining the Solaris Express program and getting early access to Solaris 10. Alternatively, you can download xsltproc as part of the libxslt package for Solaris 9 here.

I've written a cgi-bin script which can be used to invoke xsltproc to perform transformations on input XML documents. Of course, you don't have to use xsltproc, many web servers have extensions that can perform XSLT transformations based on the document type. The virtue of this approach is that it is a "lowest common denominator" which doesn't require any web server configuration or addition module installation. This script relies on a utility program called pooldump which is basically the program from my earlier post about dumping a pools configuration as an XML document. The script, xslt_proc, needs a little customization. Instructions are included at the top of the file.

Finally, you need a stylesheet, pool_html.xsl, which will perform the transformation. I don't claim to be an XSLT expert, so please be gentle with any suggestions for improvements that you might want to make. It's good enough for my needs and I hope that you might find it useful.I keep this in my cgi-bin directory so that I don't need to set an explicit path to it in the xslt_proc script, but you can put it anywhere you like as long as it can be accessed when your cgi-bin script runs.

You can download the cgi-bin script and stylesheet from here.

If you want to see what the HTML output looks like before spending any time on this, here is a sample page that I generated on my workstation.

(2004-06-10 02:16:26.0) Permalink

Generating an XML Document from a Resource Pools Configuration

XML is a convenient representation format for small quantities of data. There are many tools for manipulating XML documents and a lot of people are now familiar enough with the structure of XML to feel comfortable with this type of representation. Especially in preference to yet another problem domain specific data representation format (YAPDSDRF?).

libpool(3LIB) provides an API for manipulating Resource Pools configurations. In particular, the pool_conf_export(3POOL) function may be used to create a file containing the XML representation of a configuration.

pool_conf_export(3POOL) takes three arguments:

 conf - A valid pools configuration pointer
 location - The pathname you wish to write the configuration into
 format - This must be POX_NATIVE which is the native pools representation format 

It's not immediately obvious (and I say this with some shame since I defined this function) that POX_NATIVE == XML. Once you know that, it's fairly obvious how a program to dump a resource pools configuration in XML format can be written.

One further tip, you can pass "-" as the location and the output will be directed to stdout.

I'm sure that I don't need to say that the following program is completely unsupported and used at your peril, but I thought I would say it just to make it clear.

#include <pool.h>

/* ARGSUSED */
int
main(int argc, char **argv)
{
        pool_conf_t *conf;

        if ((conf = pool_conf_alloc()) == NULL) {
                return (2);
        }
        if (pool_conf_open(conf, pool_dynamic_location(), PO_RDONLY) !=
            PO_SUCCESS) {
                pool_conf_free(conf);
                return (2);
        }

        if (pool_conf_export(conf, "-", POX_NATIVE) != PO_SUCCESS) {
                (void) pool_conf_close(conf);
                pool_conf_free(conf);
                return (2);
        }
        (void) pool_conf_close(conf);
        pool_conf_free(conf);
        return (0);
}

Once you have your configuration as an XML document you can manipulate it easily using a wide range of tools. In a later post, I'll show how you can use an XSL stylesheet to transform the XML document into an HTML document using the xsltproc(1) utility. (2004-06-10 01:00:48.0) Permalink Comments [1]