Python on Windows - how to wait for multiple child processes?

It might seem overkill, but, here it goes:

import Queue, thread, subprocessresults= Queue.Queue()def process_waiter(popen, description, que):    try: popen.wait()    finally: que.put( (description, popen.returncode) )process_count= 0proc1= subprocess.Popen( ['python', 'mytest.py'] )thread.start_new_thread(process_waiter,    (proc1, "1 finished", results))process_count+= 1proc2= subprocess.Popen( ['python', 'mytest.py'] )thread.start_new_thread(process_waiter,    (proc2, "2 finished", results))process_count+= 1# etcwhile process_count > 0:    description, rc= results.get()    print "job", description, "ended with rc =", rc    process_count-= 1

Twisted has an asynchronous process-spawning API which works on Windows. There are actually several different implementations, many of which are not so great, but you can switch between them without changing your code.

Building on zseil's answer, you can do this with a mix of subprocess and win32 API calls. I used straight ctypes, because my Python doesn't happen to have win32api installed. I'm just spawning sleep.exe from MSYS here as an example, but clearly you could spawn any process you like. I use OpenProcess() to get a HANDLE from the process' PID, and then WaitForMultipleObjects to wait for any process to finish.

import ctypes, subprocessfrom random import randintSYNCHRONIZE=0x00100000INFINITE = -1numprocs = 5handles = {}for i in xrange(numprocs):    sleeptime = randint(5,10)    p = subprocess.Popen([r"c:\msys\1.0\bin\sleep.exe", str(sleeptime)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)    h = ctypes.windll.kernel32.OpenProcess(SYNCHRONIZE, False, p.pid)    handles[h] = p.pid    print "Spawned Process %d" % p.pidwhile len(handles) > 0:    print "Waiting for %d children..." % len(handles)    arrtype = ctypes.c_long * len(handles)    handle_array = arrtype(*handles.keys())    ret = ctypes.windll.kernel32.WaitForMultipleObjects(len(handle_array), handle_array, False, INFINITE)    h = handle_array[ret]    ctypes.windll.kernel32.CloseHandle(h)    print "Process %d done" % handles[h]    del handles[h]print "All done!"