All | 43 Folders | Accessibility | BoingBoing | Books | Computer Related | Family | Films | General | Hacking | Hobbies | Humor | Java | Links | Omni | OpenSolaris | Puzzles and Games

« Previous day (Feb 24, 2007) | Main | Next day (Feb 26, 2007) »
20070225 Sunday February 25, 2007

Problem Automatically Downloading Files With Wget

A question for you wget experts.

I'm trying to put together a script that will automatically download all the user supplied custom levels for the Professor Fizzwizzle game.

You can see them, if you go to their web page. Each one is of the form:

http://grubbygames.com/pf_levels/download.php?id=NNN
where "NNN" is a unique level.

Now if I do (say):

  % wget http://grubbygames.com/pf_levels/download.php?id=852

the download succeeds, but the file is saved with a name of "download.php?id=852".

If I do:

  % wget --spider --debug http://grubbygames.com/pf_levels/download.php?id=852"

I can see from the debug messages, that the actual filename I want it saved as is in the "Content-Disposition:" line:

DEBUG output created by Wget 1.10.2 on linux-gnu.

--11:55:57--  http://grubbygames.com/pf_levels/download.php?id=852
           => `download.php?id=852.1'
Resolving grubbygames.com... 69.20.54.231
Caching grubbygames.com => 69.20.54.231
Connecting to grubbygames.com|69.20.54.231|:80... connected.
Created socket 3.
Releasing 0x00000000005576a0 (new refcount 1).

---request begin---
HEAD /pf_levels/download.php?id=852 HTTP/1.0
User-Agent: Wget/1.10.2
Accept: */*
Host: grubbygames.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK
Date: Sun, 25 Feb 2007 20:00:08 GMT
Server: Apache/2.0.46 (Red Hat)
Accept-Ranges: bytes
X-Powered-By: PHP/4.3.2
Pragma: public
Cache-Control: must-revalidate, post-check=0, pre-check=0
Content-Disposition: attachment; filename="duncan1.lvl"
Connection: close
Content-Type: lvl

---response end---
200 OK
Length: unspecified [lvl]
Closed fd 3
200 OK

Are their any command options that I can give to wget, to get it to save into the filename given on the "Content-Disposition:" line?

I'm using wget 1.10.2.

Update:

Thanks to everybody who commented. As I wanted to write a Python script to grab all the user supplied custom levels within a given range, the solution by Stephen English was just perfect.

Here's my complete script using a slight variation of his code:

#!/usr/bin/env python
#
# Script to automatically download a range of Professor Fizzwizzle custom
# levels from the Grubbygames website.
#

import urllib2
import sys

startLevel = 1
endLevel = 853
baseUrl = "http://grubbygames.com/pf_levels/download.php?id="

def main():
    for i in range(startLevel, endLevel+1):
        url = baseUrl + str(i)
        print "url: `%s`" % url

        level = urllib2.urlopen(url)
        f = open(level.headers["Content-Disposition"].split("\"")[1], "w")
        f.write(level.read())
        f.close()

if __name__ == "__main__":
    main()

[]

[]

( Feb 25 2007, 12:42:12 PM PST ) [Listen] Permalink Comments [5]