Python multithreading max_workers
To begin with, you seem to be quoting the wrong part of the documentation in your link, namely the one for processes, not threads. The one for concurrent.futures.ThreadPoolExecutor states:
Changed in version 3.5: If
max_workersisNoneor not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.
Since you're using threads, not processes, the assumption is that your application is IO bound, not CPU bound, and that you're using this for concurrency, not parallelism. The more threads you use, the higher concurrency you'll achieve (up to a point), but the less CPU cycles you'll get (as there will be context switches). You have to instrument your application under typical workloads to see what works best for you. There is no universally optimal solution for this.