What is the subprocess.Popen max length of the args parameter? What is the subprocess.Popen max length of the args parameter? shell shell

What is the subprocess.Popen max length of the args parameter?


If you're passing shell=False, then Cmd.exe does not come into play.

On windows, subprocess will use the CreateProcess function from Win32 API to create the new process. The documentation for this function states that the second argument (which is build by subprocess.list2cmdline) has a max length of 32,768 characters, including the Unicode terminating null character. If lpApplicationName is NULL, the module name portion of lpCommandLine is limited to MAX_PATH characters.

Given your example, I suggest providing a value for executable (args[0]) and using args for the first parameter. If my reading of the CreateProcess documentation and of the subprocess module source code is correct, this should solve your problem.

[edit: removed the args[1:] bit after getting my hands on a windows machine and testing]


For Unix-like platforms, the kernel constant ARG_MAX is defined by POSIX. It is required to be at least 4096 bytes, though on modern systems, it's probably a megabyte or more.

On many systems, getconf ARG_MAX will reveal its value at the shell prompt.

The shell utility xargs conveniently allows you to break up a long command line. For example, if

python myscript.py *

fails in a large directory because the list of files expands to a value whose length in bytes exceeds ARG_MAX, you can work around it with something like

printf '%s\0' * |xargs -0 python myscript.py

(The option -0 is a GNU extension, but really the only completely safe way to unambiguously pass a list of file names which could contain newlines, quoting characters, etc.) Maybe also explore

find . -maxdepth 1 -type f -exec python myscript.py {} +

Conversely, to pass a long list of arguments to subprocess.Popen() and friends, something like

p = subprocess.Popen(['xargs', '-0', 'command'],    stdin=subprocess.PIPE, stdout=subprocess.PIPE,    stderr=subprocess.PIPE)out, err = p.communicate('\0'.join(long_long_argument_list))

... where in most scenarios you should probably avoid raw Popen() and let a wrapper function like run() or check_call() do most of the work:

r = subprocess.run(['xargs', '-0', 'command'],    input='\0'.join(long_long_argument_list),    universal_newlines=True)out = r.stdout

subprocess.run() supports text=True in 3.7+ as the new name of universal_newlines=True. Older Python versions than 3.5 didn't have run, so you need to fall back to the older legacy functions check_output, check_call, or (rarely) call.