Catch Ctrl+C / SIGINT and exit multiprocesses gracefully in python [duplicate]

The previously accepted solution has race conditions and it does not work with map and async functions.

The correct way to handle Ctrl+C/SIGINT with multiprocessing.Pool is to:

1. Make the process ignore SIGINT before a process Pool is created. This way created child processes inherit SIGINT handler.
2. Restore the original SIGINT handler in the parent process after a Pool has been created.
3. Use map_async and apply_async instead of blocking map and apply.
4. Wait on the results with timeout because the default blocking waits to ignore all signals. This is Python bug https://bugs.python.org/issue8296.

Putting it together:

#!/bin/env pythonfrom __future__ import print_functionimport multiprocessingimport osimport signalimport timedef run_worker(delay):    print("In a worker process", os.getpid())    time.sleep(delay)def main():    print("Initializng 2 workers")    original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)    pool = multiprocessing.Pool(2)    signal.signal(signal.SIGINT, original_sigint_handler)    try:        print("Starting 2 jobs of 5 seconds each")        res = pool.map_async(run_worker, [5, 5])        print("Waiting for results")        res.get(60) # Without the timeout this blocking call ignores all signals.    except KeyboardInterrupt:        print("Caught KeyboardInterrupt, terminating workers")        pool.terminate()    else:        print("Normal termination")        pool.close()    pool.join()if __name__ == "__main__":    main()

As @YakovShklarov noted, there is a window of time between ignoring the signal and unignoring it in the parent process, during which the signal can be lost. Using pthread_sigmask instead to temporarily block the delivery of the signal in the parent process would prevent the signal from being lost, however, it is not available in Python-2.

The solution is based on this link and this link and it solved the problem, I had to moved to Pool though:

import multiprocessingimport timeimport signalimport sysdef init_worker():    signal.signal(signal.SIGINT, signal.SIG_IGN)def worker():    while(True):        time.sleep(1.1234)        print "Working..."if __name__ == "__main__":    pool = multiprocessing.Pool(50, init_worker)    try:        for i in range(50):            pool.apply_async(worker)        time.sleep(10)        pool.close()        pool.join()    except KeyboardInterrupt:        print "Caught KeyboardInterrupt, terminating workers"        pool.terminate()        pool.join()

Just handle KeyboardInterrupt-SystemExit exceptions in your worker process:

def worker():    while(True):        try:            msg = self.msg_queue.get()        except (KeyboardInterrupt, SystemExit):            print("Exiting...")            break