Monday February 11, 2008
An Interesting Problem with Python, Proxies and OS X
I just came back from long a vacation and I spent first half a day back at work dealing with a frustrating although interesting issue with Python and Mac OS. The problem and (even more so) the solution are poorly documented, so I am writing this post hoping that other people facing it will be able to google my blog and find the way out here.
While I was away the NetBeans project migrated from CVS to Mercurial so my first task was to check out the sources (or rather clone the repository). I did anticipate trouble and it came indeed: the 'hg clone' command failed with a message:
abort: error: No address associated with nodename
After running a series of sanity tests I realized the problem was located at low level, in the Python framework and affected every Python script trying to read data from the Internet. I had a hunch to check the proxy settings so I executed the the following code:
import urllib2
print urllib2.getproxies()
and discovered that Python was using a phantom proxy server (it was a one that looked familiar but I didn't use it for ages).
After some research I learned that the problem is caused by the fact that Python uses Internet Config to discover network settings on Mac.
Internet Config is a very old utility that stores all the network settings in one place and provides an API to query for them. Internet Config was a very welcomed innovation back in 1994 when it was started by a community members. in fact it was such a blast that Apple decided to include it in Mac OS 8.4 and further releases.
Later OS X provided new standard ways of storing the network configuration info but the Internet Config API remained to preserve compatibility. Many developers continued to use the Internet Config API without realizing it was not only deprecated but also out of sync with the system settings. Moreover there is no longer a graphical way to access those data in OS X.
Apparently I used some application that wrote the proxy info to the Internet Config database. The setting was ignored by most application, but Python was still relying on it.
Even once I knew that much it still wasn't easy to find out where this data was stored: the 'com.apple.internetconfig.plist' file in the Preferences directory. The file can be edited with the Property List Editor utility (it comes with the Developer Tools). Pay attention to these settings: UseHTTPProxy and HTTPProxyHost.
I hope that at least one of these two sneaky bugs (one on the Apple and the other on the Python side) gets fixed soon, in the meantime it is possible to work it around editing the 'com.apple.internetconfig.plist' file by hand.
Posted by tomslot
( Feb 11 2008, 01:51:43 PM CET )
Permalink
