Is boto3 client thread-safe

If you take a look at the Multithreading/Processing documentation for boto3 you can see that they recommend one client per session as there is shared data between instance that can be mutated by individual threads.

It also looks like there's an open GitHub issue for this exact question. https://github.com/boto/botocore/issues/1246

I recently tried using the single boto client instance using concurrent.futures.ThreadPoolExecutor. I run into exceptions coming from boto. I assume the boto client is not thread safe in this case.

The exception I got

  File "xxx/python3.7/site-packages/boto3/session.py", line 263, in client    aws_session_token=aws_session_token, config=config)  File "xxx/python3.7/site-packages/botocore/session.py", line 827, in create_client    endpoint_resolver = self._get_internal_component('endpoint_resolver')  File "xxx/python3.7/site-packages/botocore/session.py", line 694, in _get_internal_component    return self._internal_components.get_component(name)  File "xxx/python3.7/site-packages/botocore/session.py", line 906, in get_component    del self._deferred[name]

From documentation:

Low-level clients are thread safe. When using a low-level client, it is recommended to instantiate your client then pass that client object to each of your threads.

Instantiation of the client is not thread safe while an instance is. To make things work in a multi-threaded environment, put instantiation in a global Lock like this:

boto3_client_lock = threading.Lock()def create_client():    with boto3_client_lock:        return boto3.client('s3', aws_access_key_id='your key id', aws_secret_access_key='your access key')