Python urllib2 Basic Auth Problem

The problem could be that the Python libraries, per HTTP-Standard, first send an unauthenticated request, and then only if it's answered with a 401 retry, are the correct credentials sent. If the Foursquare servers don't do "totally standard authentication" then the libraries won't work.

Try using headers to do authentication:

import urllib2, base64request = urllib2.Request("http://api.foursquare.com/v1/user")base64string = base64.b64encode('%s:%s' % (username, password))request.add_header("Authorization", "Basic %s" % base64string)   result = urllib2.urlopen(request)

Had the same problem as you and found the solution from this thread: http://forums.shopify.com/categories/9/posts/27662

(copy-paste/adapted from https://stackoverflow.com/a/24048772/1733117).

First you can subclass urllib2.BaseHandler or urllib2.HTTPBasicAuthHandler, and implement http_request so that each request has the appropriate Authorization header.

import urllib2import base64class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):    '''Preemptive basic auth.    Instead of waiting for a 403 to then retry with the credentials,    send the credentials if the url is handled by the password manager.    Note: please use realm=None when calling add_password.'''    def http_request(self, req):        url = req.get_full_url()        realm = None        # this is very similar to the code from retry_http_basic_auth()        # but returns a request object.        user, pw = self.passwd.find_user_password(realm, url)        if pw:            raw = "%s:%s" % (user, pw)            auth = 'Basic %s' % base64.b64encode(raw).strip()            req.add_unredirected_header(self.auth_header, auth)        return req    https_request = http_request

Then if you are lazy like me, install the handler globally

api_url = "http://api.foursquare.com/"api_username = "johndoe"api_password = "some-cryptic-value"auth_handler = PreemptiveBasicAuthHandler()auth_handler.add_password(    realm=None, # default realm.    uri=api_url,    user=api_username,    passwd=api_password)opener = urllib2.build_opener(auth_handler)urllib2.install_opener(opener)

Here's what I'm using to deal with a similar problem I encountered while trying to access MailChimp's API. This does the same thing, just formatted nicer.

import urllib2import base64chimpConfig = {    "headers" : {    "Content-Type": "application/json",    "Authorization": "Basic " + base64.encodestring("hayden:MYSECRETAPIKEY").replace('\n', '')    },    "url": 'https://us12.api.mailchimp.com/3.0/'}#perform authenticationdatas = Nonerequest = urllib2.Request(chimpConfig["url"], datas, chimpConfig["headers"])result = urllib2.urlopen(request)