Fetch a Wikipedia article with Python
You need to use the urllib2 that superseedes urllib in the python std library in order to change the user agent.
Straight from the examples
import urllib2opener = urllib2.build_opener()opener.addheaders = [('User-agent', 'Mozilla/5.0')]infile = opener.open('http://en.wikipedia.org/w/index.php?title=Albert_Einstein&printable=yes')page = infile.read()
It is not a solution to the specific problem. But it might be intersting for you to use the mwclient library (http://botwiki.sno.cc/wiki/Python:Mwclient) instead. That would be so much easier. Especially since you will directly get the article contents which removes the need for you to parse the html.
I have used it myself for two projects, and it works very well.