Python subprocess Popen.communicate() equivalent to Popen.stdout.read()? Python subprocess Popen.communicate() equivalent to Popen.stdout.read()? python python

Python subprocess Popen.communicate() equivalent to Popen.stdout.read()?


If you look at the source for subprocess.communicate(), it shows a perfect example of the difference:

def communicate(self, input=None):    ...    # Optimization: If we are only using one pipe, or no pipe at    # all, using select() or threads is unnecessary.    if [self.stdin, self.stdout, self.stderr].count(None) >= 2:        stdout = None        stderr = None        if self.stdin:            if input:                self.stdin.write(input)            self.stdin.close()        elif self.stdout:            stdout = self.stdout.read()            self.stdout.close()        elif self.stderr:            stderr = self.stderr.read()            self.stderr.close()        self.wait()        return (stdout, stderr)    return self._communicate(input)

You can see that communicate does make use of the read calls to stdout and stderr, and also calls wait(). It is just a matter of order of operations. In your case because you are using PIPE for both stdout and stderr, it goes into _communicate():

def _communicate(self, input):    stdout = None # Return    stderr = None # Return    if self.stdout:        stdout = []        stdout_thread = threading.Thread(target=self._readerthread,                                         args=(self.stdout, stdout))        stdout_thread.setDaemon(True)        stdout_thread.start()    if self.stderr:        stderr = []        stderr_thread = threading.Thread(target=self._readerthread,                                         args=(self.stderr, stderr))        stderr_thread.setDaemon(True)        stderr_thread.start()    if self.stdin:        if input is not None:            self.stdin.write(input)        self.stdin.close()    if self.stdout:        stdout_thread.join()    if self.stderr:        stderr_thread.join()    # All data exchanged.  Translate lists into strings.    if stdout is not None:        stdout = stdout[0]    if stderr is not None:        stderr = stderr[0]    # Translate newlines, if requested.  We cannot let the file    # object do the translation: It is based on stdio, which is    # impossible to combine with select (unless forcing no    # buffering).    if self.universal_newlines and hasattr(file, 'newlines'):        if stdout:            stdout = self._translate_newlines(stdout)        if stderr:            stderr = self._translate_newlines(stderr)    self.wait()    return (stdout, stderr)

This uses threads to read from multiple streams at once. Then it calls wait() at the end.

So to sum it up:

  1. This example reads from one stream at a time and does not wait for it to finish the process.
  2. This example reads from both streams at the same time via internal threads, and waits for it to finish the process.
  3. This example waits for the process to finish, and then reads one stream at a time. And as you mentioned has the potential to deadlock if there is too much written to the streams.

Also, you don't need these two import statements in your 2nd and 3rd examples:

from subprocess import communicatefrom subprocess import wait

They are both methods of the Popen object.