Mimicing glib.spawn_async with Popen… Mimicing glib.spawn_async with Popen… multithreading multithreading

Mimicing glib.spawn_async with Popen…


asyncio has subprocess_exec, there is no need to use the subprocess module at all:

import asyncioclass Handler(asyncio.SubprocessProtocol):    def pipe_data_received(self, fd, data):        # fd == 1 for stdout, and 2 for stderr        print("Data from /bin/ls on fd %d: %s" % (fd, data.decode()))    def pipe_connection_lost(self, fd, exc):        print("Connection lost to /bin/ls")    def process_exited(self):        print("/bin/ls is finished.")loop = asyncio.get_event_loop()coro = loop.subprocess_exec(Handler, "/bin/ls", "/")loop.run_until_complete(coro)loop.close()

With subprocess and threading, it's simple as well. You can just spawn a thread per pipe, and one to wait() for the process:

import subprocessimport threadingclass PopenWrapper(object):    def __init__(self, args):       self.process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.DEVNULL)       self.stdout_reader_thread = threading.Thread(target=self._reader, args=(self.process.stdout,))       self.stderr_reader_thread = threading.Thread(target=self._reader, args=(self.process.stderr,))       self.exit_watcher = threading.Thread(target=self._exit_watcher)       self.stdout_reader_thread.start()       self.stderr_reader_thread.start()       self.exit_watcher.start()    def _reader(self, fileobj):        for line in fileobj:            self.on_data(fileobj, line)    def _exit_watcher(self):        self.process.wait()        self.stdout_reader_thread.join()        self.stderr_reader_thread.join()        self.on_exit()    def on_data(self, fd, data):        return NotImplementedError    def on_exit(self):        return NotImplementedError    def join(self):        self.process.wait()class LsWrapper(PopenWrapper):    def on_data(self, fd, data):        print("Received on fd %r: %s" % (fd, data))    def on_exit(self):        print("Process exited.")LsWrapper(["/bin/ls", "/"]).join()

However, mind that glib does not use threads to asynchroneously execute your callbacks. It uses an event loop, just as asyncio does. The idea is that at the core of your program is a loop that waits until something happens, and then synchronously executes an associated callback. In your case, that's "data becomes available for reading on one of the pipes", and "the subprocess has exited". In general, its also stuff like "the X11-server reported mouse movement", "there's incoming network traffic", etc. You can emulate glib's behaviour by writing your own event loop. Use the select module on the two pipes. If select reports that the pipes are readable, but read returns no data, the process likely exited - call the poll() method on the subprocess object in this case to check whether it is completed, and call your exit callback if it has, or an error callback elsewise.