Updated: Be Informed When Used Amazon Books Are Available At Your Price

Short Version: Regular readers may remember a post back in September where I showed how to find out if the books on your Amazon wish list were now available used, at a price you were willing to pay. This stopped working for me on the 7th February. There are now new versions of the make_book_list.py and cheap_books.py scripts that fix the problem found.

Long Version: I have a cron job that runs the cheap_books.py script nightly. The last time it worked was the 6th February. I finally got around to investigating the problem over the weekend. As I've got it setup to send the mail message with the results to my Gmail account, the first thing I did was look to see if gmail had thrown it in my Spam folder. It had done exactly this for the mail that was the results of a similar script, so it wasn't much of a stretch to think it had happened again.

That wasn't the case, so I then ran the script manually. It terminated abruptly with:

Traceback (most recent call last):
  File "./cheap_books.py", line 108, in 
    checkBookList(books.list_of_books)
  File "./cheap_books.py", line 95, in checkBookList
    books = amazon.searchByASIN(asin)
  File "/home/richb/DVD/python/CheapBooks/amazon.py", line 321, in searchByASIN
    return search("AsinSearch", ASIN, None, type, None, license_key, http_proxy, locale, associate)
  File "/home/richb/DVD/python/CheapBooks/amazon.py", line 293, in search
    xmldoc = minidom.parse(usock)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse
    return expatbuilder.parse(file)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py", line 931, in parse
    result = builder.parseFile(file)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py", line 208, in parseFile
    parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 24, column 19

There's a really useful error message. What was worst, was that by turning on debugging in the cheap_books.py script, I could see that it wasn't always happening for the same book. Sometimes the script would run for a longer time before generating the traceback.

It would have been nice if the amazon.py code caught that error. As there are no debug facilities in the pyAmazon source code, and there hasn't been a new release since August 2004 (and Michael Josephson is looking around for a new maintainer), I wasn't expecting too much help there. I ended up doing:

  $ sudo vi /usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py

and adding a:

                print ">>>buffer: ", buffer

statement just before line 208 and ran it again. When it crashed, I could see that the last part of buffer was:

<h2>503 - Service Unavailable</h2>
<p>
You are receiving this error because you sent more than one request per second to Amazon Web Services (AWS).  Per the AWS Licensing Agreement, AWS restricts each IP address to no more than one request per second.  Please see the full
 <a href="http://www.amazon.com/gp/browse.html?node=3440661">AWS Licensing Agreement</a>.

Well this is new (and presumably introduced about 6/7th February), but okay. I can live with that. So I adjusted my two scripts and put in a

         time.sleep(1)

in the book searching loop, just after doing the amazon.searchByASIN(asin) call.

When I ran it again, it crapped out in the same place. I then thought that maybe my browser (which had an Amazon book web page showing) was helping to cause the problem. Nowadays, those Amazon book web pages have a live region for that message that goes something like: "Want it delivered Tuesday, February 26? Order it in the next 7 hours and 1 minute,". It's updating every second. Darned if I know why it needs to be that active.

So I terminated my browser and ran it again. No change. Still Crapsville, U.S.A. I then changed it to:

 
         time.sleep(2)

and lo and behold, the script started working again.

So I now wonder whether Amazon have a rounding error in their code. Whatever. The script runs at night. If it takes twice as long as it used to to run (8 minutes instead of four), that's not a big deal.

[Technorati Tag: Amazon]

[Technorati Tag: Books]

[Technorati Tag: Hacking]

[Technorati Tag: Python]

( Feb 25 2008, 08:44:03 AM PST ) [Listen] Permalink Comments [3]