Controlling scheduling priority of python threads? Controlling scheduling priority of python threads? python python

Controlling scheduling priority of python threads?


I believe that threading priority is not controllable in python due to how they are implemented using a global interpreter lock (GIL). Having said that, even if you could give one thread more CPU processing priority, the python implementation that hands around the GIL would not be aware of this as it handed around the GIL. If you were able to increase niceness in a single thread in your pool (say it is doing a more important job) you would need to use your own implementation of locks to give the higher priority thread access to the GIL more often.

A google search returns this article which I believe is similar to what you are asking

Explains why it doesnt workhttp://www.velocityreviews.com/forums/t329441-threading-priority.html

Explains the workaround I was suggestinghttp://bytes.com/topic/python/answers/645966-setting-thread-priorities


The python threading-docs mention explicitly that there is no support for setting thread-priorities:

The design of this module is loosely based on Java’s threading model. However, where Java makes locks and condition variables basic behavior of every object, they are separate objects in Python. Python’s Thread class supports a subset of the behavior of Java’s Thread class; currently, there are no priorities, no thread groups, and threads cannot be destroyed, stopped, suspended, resumed, or interrupted. The static methods of Java’s Thread class, when implemented, are mapped to module-level functions.


It doesn't work, but I tried:

  1. getting the parent pid and priority
  2. launching threads using concurrent.futures.ThreadPoolExecutor
  3. using ctypes to get the (linux) thread id from within the thread(works)
  4. using the tid with os.setpriority(os.PRIO_PROCESS,tid,parent_priority+1)
  5. calling pool.shutdown() from the parent.

Even with liberal sprinkling of os.sched_yield(), the child threads never actually run past the setpriority().

Reading man pages, it seems threads don't have the capability to change (even their) scheduling priority; you have to do something with "capabilities" to give the thread the "CAP_SYS_NICE" capability. Running the process with root permissions didn't help either; child threads still don't run.