Is collections.defaultdict thread-safe? Is collections.defaultdict thread-safe? python python

Is collections.defaultdict thread-safe?


It is thread safe, in this specific case.

To know why it is important to understand when Python switches threads. CPython only allows switching between threads between Python bytecode steps. This is where the GIL comes in; every N byte code instructions the lock is released and a thread switch can take place.

The d['key'] code is handled by one bytecode (BINARY_SUBSCR) that triggers the .__getitem__() method to be called on the dictionary.

A defaultdict, configured with list as the default value factory, and using string values as keys, handles the dict.__getitem__() method entirely in C, and the GIL is never unlocked, making dict[key] lookups thread safe.

Note the qualification there; if you create a defaultdict instance with a different default-value factory, one that uses Python code (lambda: [1, 2, 3] for example), all bets are off as that means the C code calls back into Python code and the GIL can be released again while executing the bytecode for the lambda function. The same applies to the keys, when using an object that implements either __hash__ or __eq__ in Python code then a thread switch can take place there. Next, if the factory is written in C code that explicitly releases the GIL, a thread switch can take place and thread safety is out the window.