Saturday February 18, 2006 Picked this tip up off slashdot - some websites (e.g. the Washington Post) insist on making you register to read their content, even if the content is actually free - the assumption is that they only want the registration so they can harvest your email address and use it for marketing purposes, and as someone who gets huge volumes of spam already (despite 500+ a day caught by the corporate filters, 40-50 still get through) I'm very loathe to give out my email address when I don't have to. Although there are services such as BugMeNot which allow you to create a throw-away email address for registration and to look up already registered addresses, the webmasters of the sites that require registration cotton on to these pretty fast and block them.
However, these selfsame sites obviously want google to index them, so they allow the google search robot in without requiring registration. Enter the User Agent Switcher plugin for Firefox. I've used this in the past to get access sites that insist you use Internet Explorer to view them, but you can add an entry to make your browser look like it is the google bot. In the Description field put whatever you want to identify the entry, and in the User Agent field put the string googlebot/2.1 (+http://www.google.com/bot.html), you can leave the rest of the fields empty. Switch to the new User Agent string and hey presto, all that content you couldn't previously access is visible. Unlike the disposable email address arms race, I don't see how this can be circumvented without making the content invisible to Google as well.
Posted by alanbur
( Feb 18 2006, 05:58:18 PM GMT )
Permalink
Comments [6]