Making sure a Python script with subprocesses dies on SIGINT
The subprocess is by default part of the same process group, and only one can control and receive signals from the terminal, so there are a couple of different solutions.
Setting stdin as a PIPE (in contrast to inheriting from the parent process), this will prevent the child process from receiving signals associated to it.
subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)Detaching from the parent process group, the child will no longer receive signals
def preexec_function(): os.setpgrp()subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, preexec_fn=preexec_function)Explicitly ignoring signals in the child process
def preexec_function(): signal.signal(signal.SIGINT, signal.SIG_IGN)subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, preexec_fn=preexec_function)This might however be overwritten by the child process.
Fist thing; there is a send_signal() method on the Popen object. If you want to send a signal to one you've launched, use this method to send it.
Second thing; a deeper problem with the way you're setting up communication with your subprocess and then, um, not communicating with it. You cannot safely tell the subprocess to send its output to subprocess.PIPE and then not read from the pipes. UNIX pipes are buffered (typically a 4K buffer?), and if the subprocess fills up the buffer and the process on the other end of the pipe doesn't read the buffered data, the subprocess will pend (locking up, from an observer's perspective) on its next write to the pipe. So, the usual pattern when using subprocess.PIPE is to call communicate() on the Popen object.
It is not mandatory to use subprocess.PIPE if you want data back from the subprocess. A cool trick is to use the tempfile.TemporaryFile class to make an unnamed temp file (really it opens a file and immediately deletes the inode from the file system, so you have access to the file but no-one else can open one. You can do something like:
with tempfile.TemporaryFile() as iofile: p = Popen(cmd, stdout=iofile, stderr=iofile) while True: if p.poll() is not None: break else: time.sleep(0.1) # without some sleep, this polling is VERY busy...Then you can read the contents of your temporary file (seek to the beginning of it before you do, to be sure you're at the beginning) when you know the subprocess has exited, instead of using pipes. The pipe buffering problem won't be a problem if the subprocess's output is going to a file (temporary or not).
Here is a riff on your code sample that I think does what you want. The signal handler just repeats the signals being trapped by the parent process (in this example, SIGINT and SIGTERM) to all current subprocesses (there should only ever be one in this program) and sets a module-level flag saying to shutdown at the next opportunity. Since I'm using subprocess.PIPE I/O, I call communicate() on the Popen object.
#!/usr/bin/env pythonfrom subprocess import Popen, PIPEimport signalimport syscurrent_subprocs = set()shutdown = Falsedef handle_signal(signum, frame): # send signal recieved to subprocesses global shutdown shutdown = True for proc in current_subprocs: if proc.poll() is None: proc.send_signal(signum)signal.signal(signal.SIGINT, handle_signal)signal.signal(signal.SIGTERM, handle_signal)for _ in range(10): if shutdown: break cmd = ["sleep", "2"] p = Popen(cmd, stdout=PIPE, stderr=PIPE) current_subprocs.add(p) out, err = p.communicate() current_subprocs.remove(p) print "subproc returncode", p.returncodeAnd calling it (with a Ctrl-C in the third 2 second interval):
% python /tmp/proctest.py subproc returncode 0subproc returncode 0^Csubproc returncode -2