Why does multithreading do not speed up parsing HTML with lxml?

The documentation gives a good lead there: "as long as you use either the default parser (which is replicated for each thread) or create a parser for each thread yourself."

You're definitely not creating a parser for each thread. You can see that, if you do not specify a parser yourself, the fromstring function uses a global one.

Now for the other condition, you can see at the bottom of the file that html_parser is a subclass of lxml.etree.HTMLParser. With no special behavior and most importantly no thread local storage. I cannot test here but I would believe you end up sharing a parser across your two threads, which does not qualify as "default parser".

Could you try instanciating the parsers yourself and feeding them to fromstring? Or I'll do it in an hour or so and update this post.

def func(number):    parser = HTMLParser()    for x in range(number):        fromstring(DATA, parser=parser)

That's because how threads work in python.And there are differences between python 2.7 and python 3.If you really want to speed up the parsing you should use multiprocessing and not multithreading.Read this:How do threads work in Python, and what are common Python-threading specific pitfalls?

And this is about multiprocessing :http://sebastianraschka.com/Articles/2014_multiprocessing_intro.html

As long as it's not an io operations, when you use threads you add overhead of the context switching because only one thread can run at a time.When are Python threads fast?

Good luck.