Wednesday March 30, 2005

Wget wrapper script...
Interestingly the article that is most linked from this blog is the wget tip I gave a while ago, yep that surprised me too. Anyway I'm currently working remotely (very remotely), and one of the things that I need to do reguarly is grab files from machines back home to edit/view, as 18,000km makes any connection slow. Being a command line person I don't really like using ftp, to much interaction for me, and while I could use rsync or something similar, I felt that it was a bit too heavy weight for my needs, so I've been using wget to grab individual or sets of files when needed.

Using wget manually involved too much typing, and not wanting to store passwords in text format I wrote a small wrapper script around wget, with a couple of features that I wanted.

[fintanr@dhcp-ack03-200-118 fintanr] $ ipwget -h

Usage : ipwget [-p pword] [-u user] [-m mach] [-i infile] [-d] [-h] -f file
Where : -p password is your password, defaults to anonymous
           ftp otherwise
        -u user name
        -m machine to connect to
        -i infile a list of files to get
        -h files are in your home directory (requires -p)
        -f file to get (absolute path, or name if in homedir)
        -h show this help

And the code....

#!/bin/ksh
#
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
#


WGET=/usr/sfw/bin/wget
HOMEDIR=0
USER=
MACH=
PASSWORD=

usage() {

	gettext "\nUsage : ipwget [-p pword] [-u user] [-m mach] [-i infile] [-d] [-h] -f file\n"
	gettext "Where : -p password is your password, defaults to anonymous\n"
	gettext "           ftp otherwise\n"
	gettext "        -u user name\n"
	gettext "        -m machine to connect to\n"
	gettext "        -i infile a list of files to get\n"
	gettext "        -h files are in your home directory (requires -p)\n"
	gettext "        -f file to get (absolute path, or name if in homedir)\n"
	gettext "        -h show this help\n\n"
}

setUser() {

	if [ -z $USER ]
	then
		USER=`/usr/bin/who am i | awk '{print $1}'`
	fi
}

getFiles() {

	# if we are not getting a file from a homedir we
	# need to add an extra slash so that wget knows to cd

	case $HOMEDIR in
	0)	HOMEDIRSLASH="/"
		;;
	1)	HOMEDIRSLASH=""
		;;
	esac

	if [ ! -z $FILE ]
	then
		URL="${BASE_URL}${HOMEDIRSLASH}${FILE}"
		${WGET} ${URL}
	else
		for i in `cat $INFILE`
		do
			URL="${BASE_URL}${HOMEDIRSLASH}$i"
			${WGET} ${URL}
		done
	fi
}

while getopts 'f:i:m:p:u:dh' opt
do
        case $opt in
        h)      usage
                exit 1
                ;;
        p)      PASSWORD=$OPTARG
                ;;
        f)      FILE=$OPTARG
                ;;
	d)	HOMEDIR=1
		;;
	i)	INFILE=$OPTARG
		;;
	u) 	USER=$OPTARG
		;;
	m)	MACH=$OPTARG
		;;
        \?)     usage
                exit 1
        esac
done

if [ -z $MACH ]
then
	gettext "\nNo Machine Set, use -m or set MACH in $0\n"
	usage
	exit 1
fi

if [ ! -z $PASSWORD ]
then
	BASE_URL="ftp://$USER:$PASSWORD@$MACH"
else
	BASE_URL="ftp://$MACH"
fi

if [ -z $FILE -a -z $INFILE ]
then
	gettext "\nYou must supply a filename or an infile\n"
	usage
	exit 1
fi

if [ $HOMEDIR -eq 1 -a -z $PASSWORD ]
then
	gettext "\nYou must provide a password to access a home directory\n"
	usage 
	exit 1
fi

if [ ! -z $INFILE ]
then
	if [ ! -f $INFILE ]
	then
		gettext "\nInfile : $INFILE not found\n"
		usage
		exit 1
	fi
fi

setUser
getFiles

[fintanr@dhcp-ack03-200-118 fintanr] $ ipwget -p myPassword -f /tmp/foo
--23:53:42--  ftp://fintanr:*password*@myMachine//tmp/foo
           => `RUN'
Resolving myMachine... xxx.xxx.xxx.xxx
Connecting to myMachine[xxx.xxx.xxx.xxx]:21... connected.
Logging in as fintanr ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /tmp ... done.
==> PORT ... done.    ==> RETR foo ... done.
Length: 2,081 (unauthoritative)

100%[========================================>] 2,081         --.--K/s

23:53:51 (56.02 KB/s) - `foo' saved [2081]