Kill Python Multiprocessing Pool

SIGQUIT (Ctrl + \) will kill all processes even under Python 2.x.

You can also update to Python 3.x, where this behavior (only child gets the signal) seems to have been fixed.

Unfortunately in Python 2.x there really isn't a good solution for this problem. The best workaround that I know of is to use pool.map_async(...).get(timeout=<large number>) instead of pool.map. The problem is that pool.map makes a call to threading.Condition.wait(), which for some reason can't be interrupted by Ctrl+C in Python 2.x (it works in Python 3). When you use map_async(), it calls threading.Condition.wait(timeout=<large number>), which ends up doing a busy wait loop, which can be interrupted by Ctrl+C.

Try it for yourself:

c = threading.Condition()try:    c.acquire()    c.wait()  # You won't be able to interrupt thisexcept KeyboardInterrupt:    print("Caught it")c = threading.Condition()try:    c.acquire()    c.wait(timeout=100)  # You CAN interrupt thisexcept KeyboardInterrupt:    print("Caught it")

So, to make your map call interruptable, do this:

if __name__ == "__main__":    p = multiprocessing.Pool()    try:        p.map_async(func, iterable).get(timeout=10000000)    except KeyboardInterrupt:        print("Caught it")        # Optionally try to gracefully shut down the worker processes here.        p.close()        # DON'T join the pool. You'll end up hanging.

Also note, as pointed out by phihag, this issue is fixed in Python 3.4 (and possibly earlier in 3.x).

There are a couple of ways. The first way is to mark the thread as a daemon using

in Threading,

myThread.setDaemon(true)

in multiprocessing,

myThread.daemon = True

All threads marked as a daemon will terminate with the main thread. This is not the proper way to do it as it doesn't allow the threads to clean up

The next way is to listen for KeyboardInterrupt with a try-catch, and then .join() the threads like such.

try:    myThread = MyThread()except KeyboardInterrupt:    myThread.join()

If your thread is in a loop, you could use a condition such as a boolean, set this to false, and when the condition is false, it performs cleanup.

class MyThread(Threading.thread):    def __init__(self):        self.alive=True    def run(self):        while self.alive:            #do stuff        #cleanup goes here, outside the looptry:    myThread = MyThread()except KeyboardInterrupt:    myThread.alive = False    myThread.join()