Wednesday March 30, 2005

Wget wrapper script...
Interestingly the article that is most linked from this blog is the wget tip I gave a while ago, yep that surprised me too. Anyway I'm currently working remotely (very remotely), and one of the things that I need to do reguarly is grab files from machines back home to edit/view, as 18,000km makes any connection slow. Being a command line person I don't really like using ftp, to much interaction for me, and while I could use rsync or something similar, I felt that it was a bit too heavy weight for my needs, so I've been using wget to grab individual or sets of files when needed.

Using wget manually involved too much typing, and not wanting to store passwords in text format I wrote a small wrapper script around wget, with a couple of features that I wanted.

[fintanr@dhcp-ack03-200-118 fintanr] $ ipwget -h

Usage : ipwget [-p pword] [-u user] [-m mach] [-i infile] [-d] [-h] -f file
Where : -p password is your password, defaults to anonymous
           ftp otherwise
        -u user name
        -m machine to connect to
        -i infile a list of files to get
        -h files are in your home directory (requires -p)
        -f file to get (absolute path, or name if in homedir)
        -h show this help

And the code....

#!/bin/ksh
#
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
#


WGET=/usr/sfw/bin/wget
HOMEDIR=0
USER=
MACH=
PASSWORD=

usage() {

	gettext "\nUsage : ipwget [-p pword] [-u user] [-m mach] [-i infile] [-d] [-h] -f file\n"
	gettext "Where : -p password is your password, defaults to anonymous\n"
	gettext "           ftp otherwise\n"
	gettext "        -u user name\n"
	gettext "        -m machine to connect to\n"
	gettext "        -i infile a list of files to get\n"
	gettext "        -h files are in your home directory (requires -p)\n"
	gettext "        -f file to get (absolute path, or name if in homedir)\n"
	gettext "        -h show this help\n\n"
}

setUser() {

	if [ -z $USER ]
	then
		USER=`/usr/bin/who am i | awk '{print $1}'`
	fi
}

getFiles() {

	# if we are not getting a file from a homedir we
	# need to add an extra slash so that wget knows to cd

	case $HOMEDIR in
	0)	HOMEDIRSLASH="/"
		;;
	1)	HOMEDIRSLASH=""
		;;
	esac

	if [ ! -z $FILE ]
	then
		URL="${BASE_URL}${HOMEDIRSLASH}${FILE}"
		${WGET} ${URL}
	else
		for i in `cat $INFILE`
		do
			URL="${BASE_URL}${HOMEDIRSLASH}$i"
			${WGET} ${URL}
		done
	fi
}

while getopts 'f:i:m:p:u:dh' opt
do
        case $opt in
        h)      usage
                exit 1
                ;;
        p)      PASSWORD=$OPTARG
                ;;
        f)      FILE=$OPTARG
                ;;
	d)	HOMEDIR=1
		;;
	i)	INFILE=$OPTARG
		;;
	u) 	USER=$OPTARG
		;;
	m)	MACH=$OPTARG
		;;
        \?)     usage
                exit 1
        esac
done

if [ -z $MACH ]
then
	gettext "\nNo Machine Set, use -m or set MACH in $0\n"
	usage
	exit 1
fi

if [ ! -z $PASSWORD ]
then
	BASE_URL="ftp://$USER:$PASSWORD@$MACH"
else
	BASE_URL="ftp://$MACH"
fi

if [ -z $FILE -a -z $INFILE ]
then
	gettext "\nYou must supply a filename or an infile\n"
	usage
	exit 1
fi

if [ $HOMEDIR -eq 1 -a -z $PASSWORD ]
then
	gettext "\nYou must provide a password to access a home directory\n"
	usage 
	exit 1
fi

if [ ! -z $INFILE ]
then
	if [ ! -f $INFILE ]
	then
		gettext "\nInfile : $INFILE not found\n"
		usage
		exit 1
	fi
fi

setUser
getFiles

[fintanr@dhcp-ack03-200-118 fintanr] $ ipwget -p myPassword -f /tmp/foo
--23:53:42--  ftp://fintanr:*password*@myMachine//tmp/foo
           => `RUN'
Resolving myMachine... xxx.xxx.xxx.xxx
Connecting to myMachine[xxx.xxx.xxx.xxx]:21... connected.
Logging in as fintanr ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /tmp ... done.
==> PORT ... done.    ==> RETR foo ... done.
Length: 2,081 (unauthoritative)

100%[========================================>] 2,081         --.--K/s

23:53:51 (56.02 KB/s) - `foo' saved [2081]

Monday March 28, 2005

apropos and catman in one minute
I normally jumpstart my machine, and in doing such tend to have a bunch of finish scripts that setup extra things for me, and remove some minor irritations with a default install. One of these is being able to use apropos(1).

The current install on my laptop was done from cd's as I wanted to take a look at the common complaints people have about the cd based Solaris install (and the fact that the box I normally jumpstart off is on the other side of the globe at the moment), so on completing it those finish scripts I mentioned above had not been run. So currently if I run apropos I get the following

[fintanr@tiresias fintanr] $ apropos scf_myname
/usr/dt/man/windex: No such file or directory
/usr/man/windex: No such file or directory
/usr/openwin/share/man/windex: No such file or directory

# catman
....... some output....

[fintanr@tiresias fintanr] $ apropos scf_myname
scf_myname      scf_handle_create (3scf)    - Service Configuration Facility handle functions

# catman -M /usr/sfw/man

[update - March 30th ]
Stephen Hahn has posted an example of creating a transient service with catman as the example over on his blog.

Saturday March 12, 2005

Matching Funds for the Tsumani Appeal
I've avoided non technical posts for a while, but this one should be mentioned. Sun matched any donations that employees made to Tsunami Relief funds through the Sun Microsystems Foundation. I got the following mail in my inbox today (edited as the amount is my own business).

Date: March 11, 2005

Thank you for using the Sun Microsystems Foundation, Inc.
Matching Gifts Program. This is to advise you that we have 
received your request to match your $personal donation to 
American National Red Cross (for Tsunami Disaster Relief) 
dated January 4, 2005.

Matching gift payments are made on a quarterly basis. A 
matching gift payment of $personal will be sent to American
National Red Cross - Fairfax, VA, by the end of April 2005.

People have asked me why I work in Sun, theres several thousand examples of why on blogs.sun.com, as an engineer it was a place I want (and wanted in college) to work in, but its intangible things such as the reasons behind the mail above that just push it that little bit beyond other places I have worked in. (I can hear the naysayers going "oh, but everywhere does this now", well Sun's being doing it for quite a bit longer, and in many, many ways, and long before CSR became a buzzword in board rooms - with the consultants, consultancy fees and press releases that go with such buzzwords).

I mentioned this over on my personal blog as well.
(2005-03-11 18:25:54.0) Permalink Comments [0]

Wednesday March 09, 2005

The Least Privilege Model in Solaris 10
Theres a nice article on the bigadmin website giving an introduction to The Least Privilege Model in the Solaris 10 OS.

I did a post back in June last year on giving a user privileges to use DTrace which might interest you if your want to play around with both privileges and DTrace.

Sunday March 06, 2005

DTrace compared with other tools
So yet again a thread on OSNews regarding Solaris 10 has descended into "DTrace can be replaced by x,y and z", so rather than having to rehash the information each time, here are a few urls in rebuttal.

The DTrace Usenix paper
Bryan Cantrill, Adam Leventhal and Mike Shapiro.
- discusses DTrace v's dprobes, ltt etc.

Comparison of DTrace to tools in Linux
Daniel Berrange of Redhat

DTrace v's OProfile
John Levon, author of OProfile - slashdot comment regarding OProfile versus DTrace

Rebuttals from Bryan
DTrace v's DProbes/LTT
The DTrace Integration
- discussess LTT
DTrace on LKML

kprobes in linux v's dtrace
Alan Hargreaves gives an example (production safe) dtrace script versus a kernel hack show on IBM developerworks

I'm sure there are others, I'll add them as I get time. Now back to my reguarly scheduled work.

Wednesday March 02, 2005

Tarballs and CGI scripts
One of the things that a performance group tends to generate is a huge amount of data, and while we make all of this available online, sometimes people like to just grab the data and analyse it later. So in order to make it easier for them to do this (and the fact that it is definately needed to finish up the current project I'm working on) I wrote up a quick cgi script earlier on today that automagically generates tarballs from a cgi script. It may be of use to some folks.

First a quick explanation about using stdout for output, in tar you use the - option, while in compress you just don't give it any options. So to create a compressed tarball on standard out for the directory your in you would do the following.

tar -cf * - | compress

Our perl script does the following, first up it creates our cgi header, we need to set the mime type correctly (or semi correctly in this case, CGI.pm has, I belive, a limitation about what it will allow you to use if you wish to give a binary file a name and force its download, but I could be wrong on this). So in this case the mime type is set to being "application/octet-stream" while we use the -attachment attribute to give a file name.

Next its just a matter of outputting the actual tarfile, so we switch into binary mode for the output, and then read in the output we are getting from standard out, buffering it to make sure we don't run into any strangeness, and then print out each buffer. And thats it. The relevant snippets of code are below for your perusal.

#!/usr/bin/perl

use CGI qw(:standard);
use CGI::Carp qw(fatalsToBrowser);
use constant BUFFER_SIZE => 4_096;
use strict;

my $buffer = "";
my $tar = "/usr/bin/tar";
my $compress = "/usr/bin/compress";

my $cgi = new CGI;
my $dir = $cgi->param("dir");

print $cgi->header(-type => "application/octet-stream", 
		   -expires => "-1d", 
		   -attachment => "$dir.tar.Z" );

binmode STDOUT;
local *TARBALL;

chdir($dir);
open(TARBALL, "$tar -cf - * | $compress |");
while ( read( TARBALL, $buffer, BUFFER_SIZE ) ) {
    print $buffer;
}
close TARBALL;