How can I run an external command asynchronously from Python?

subprocess.Popen does exactly what you want.

from subprocess import Popenp = Popen(['watch', 'ls']) # something long running# ... do other stuff while subprocess is runningp.terminate()

(Edit to complete the answer from comments)

The Popen instance can do various other things like you can poll() it to see if it is still running, and you can communicate() with it to send it data on stdin, and wait for it to terminate.

If you want to run many processes in parallel and then handle them when they yield results, you can use polling like in the following:

from subprocess import Popen, PIPEimport timerunning_procs = [    Popen(['/usr/bin/my_cmd', '-i %s' % path], stdout=PIPE, stderr=PIPE)    for path in '/tmp/file0 /tmp/file1 /tmp/file2'.split()]while running_procs:    for proc in running_procs:        retcode = proc.poll()        if retcode is not None: # Process finished.            running_procs.remove(proc)            break        else: # No process is done, wait a bit and check again.            time.sleep(.1)            continue    # Here, `proc` has finished with return code `retcode`    if retcode != 0:        """Error handling."""    handle_results(proc.stdout)

The control flow there is a little bit convoluted because I'm trying to make it small -- you can refactor to your taste. :-)

This has the advantage of servicing the early-finishing requests first. If you call communicate on the first running process and that turns out to run the longest, the other running processes will have been sitting there idle when you could have been handling their results.

This is covered by Python 3 Subprocess Examples under "Wait for command to terminate asynchronously":

import asyncioproc = await asyncio.create_subprocess_exec(    'ls','-lha',    stdout=asyncio.subprocess.PIPE,    stderr=asyncio.subprocess.PIPE)# do something else while ls is working# if proc takes very long to complete, the CPUs are free to use cycles for # other processesstdout, stderr = await proc.communicate()

The process will start running as soon as the await asyncio.create_subprocess_exec(...) has completed. If it hasn't finished by the time you call await proc.communicate(), it will wait there in order to give you your output status. If it has finished, proc.communicate() will return immediately.

The gist here is similar to Terrels answer but I think Terrels answer appears to overcomplicate things.

See asyncio.create_subprocess_exec for more information.