Portable way of detecting number of *usable* CPUs in Python
I don't think you will get any truly portable answers, so I will give a correct one.
The correct* answer for Linux is len(os.sched_getaffinity(pid))
, where pid
may be 0
for the current process. This function is exposed in Python 3.3 and later; if you need it in earlier, you'll have to do some fancy cffi
coding.
Edit: you might try to see if you can use a function int omp_get_num_procs();
if it exists, it is the only meaningful answer I found on this question but I haven't tried it from Python.
Use psutil:
from the doc https://psutil.readthedocs.io/en/latest/:
>>> import psutil>>> psutil.cpu_count()4>>> psutil.cpu_count(logical=False) # Ignoring virtual cores2
This is portable
Here's an approach that gets the number of available CPU cores for the current process on systems that implement sched_getaffinity
, and Windows:
import ctypesimport ctypes.wintypesimport osfrom platform import systemdef num_available_cores() -> int: if hasattr(os, 'sched_getaffinity'): return len(os.sched_getaffinity(0)) elif system() == 'Windows': kernel32 = ctypes.WinDLL('kernel32') DWORD_PTR = ctypes.wintypes.WPARAM PDWORD_PTR = ctypes.POINTER(DWORD_PTR) GetCurrentProcess = kernel32.GetCurrentProcess GetCurrentProcess.restype = ctypes.wintypes.HANDLE GetProcessAffinityMask = kernel32.GetProcessAffinityMask GetProcessAffinityMask.argtypes = (ctypes.wintypes.HANDLE, PDWORD_PTR, PDWORD_PTR) mask = DWORD_PTR() if not GetProcessAffinityMask(GetCurrentProcess(), ctypes.byref(mask), ctypes.byref(DWORD_PTR())): raise Exception("Call to 'GetProcessAffinityMask' failed") return bin(mask.value).count('1') else: raise Exception('Cannot determine the number of available cores')
On Linux and any other systems that implement sched_getaffinity
, we use Python's built-in wrapper for it.
On Windows we use ctypes
to call GetProcessAffinityMask
.
As far as I know there are no user APIs or tools to get/set the CPU affinity on macOS. In most cases os.cpu_count()
will work fine, but if you truly need the number of available cores you may be out of luck.