Python Interpreter blocks Multithreaded DNS requests?

On some systems, getaddrinfo is not thread-safe. Python believes that some such systems are FreeBSD, OpenBSD, NetBSD, OSX, and VMS. On those systems, Python maintains a lock specifically for the netdb (i.e. getaddrinfo and friends).

So if you can't switch operating systems, you'll have to use a different (thread-safe) resolver library, such as twisted's.

Send DNS requests asynchronously using Twisted Names:

import sysfrom twisted.internet import reactorfrom twisted.internet import deferfrom twisted.names    import clientfrom twisted.python   import logdef process_names(names):    log.startLogging(sys.stderr, setStdout=False)    def print_results(results):        for name, (success, result) in zip(names, results):            if success:                print "%s -> %s" % (name, result)            else:                print >>sys.stderr, "error: %s failed. Reason: %s" % (                    name, result)    d = defer.DeferredList(map(client.getHostByName, names), consumeErrors=True)    d.addCallback(print_results)    d.addErrback(defer.logError)    d.addBoth(lambda _: reactor.stop())reactor.callWhenRunning(process_names, """    google.com    www.2eg11erdhrtj.com     www.e2ger2dh2rtj.com    www.eg2de3rh1rtj.com    www.ege2rh4rd1tj.com    www.ege52drhrtj1.com    """.split())reactor.run()

if it's suitable you could use the multiprocessing module to enable process-based parallelism

import multiprocessing, socketNUM_PROCESSES = 5def get_url(url):    try:        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)        s.setblocking(0)        s.connect((url, 80))    except socket.gaierror:        pass #not interested in it    return 'finished ' + urldef main(url_list):    pool = multiprocessing.Pool( NUM_PROCESSES )    for output in pool.imap_unordered(get_url, url_list):        print outputif __name__=="__main__":    main("""             www.2eg11erdhrtj.com              www.e2ger2dh2rtj.com             www.eg2de3rh1rtj.com             www.ege2rh4rd1tj.com             www.ege52drhrtj1.com          """.split())