If you want to read continuously from a running subprocess, you have to make that process' output unbuffered. Your subprocess being a Python program, this can be done by passing -u to the interpreter:

python -u -m http.server

This is how it looks on a Windows box.

With this code, you can`t see the real-time output because of buffering:

for line in p.stdout:    print(line, end='')

But if you use p.stdout.readline() it should work:

while True:  line = p.stdout.readline()  if not line: break  print(line, end='')

See corresponding python bug discussion for details

UPD: here you can find almost the same problem with various solutions on stackoverflow.

I think the main problem is that http.server somehow is logging the output to stderr, here I have an example with asyncio, reading the data either from stdout or stderr.

My first attempt was to use asyncio, a nice API, which exists in since Python 3.4. Later I found a simpler solution, so you can choose, both of em should work.

asyncio as solution

In the background asyncio is using IOCP - a windows API to async stuff.

# inspired by https://pymotw.com/3/asyncio/subprocesses.htmlimport asyncioimport sysimport timeif sys.platform == 'win32':    loop = asyncio.ProactorEventLoop()    asyncio.set_event_loop(loop)async def run_webserver():    buffer = bytearray()    # start the webserver without buffering (-u) and stderr and stdin as the arguments    print('launching process')    proc = await asyncio.create_subprocess_exec(        sys.executable, '-u', '-mhttp.server',        stdout=asyncio.subprocess.PIPE,        stderr=asyncio.subprocess.PIPE    )    print('process started {}'.format(proc.pid))    while 1:        # wait either for stderr or stdout and loop over the results        for line in asyncio.as_completed([proc.stderr.readline(), proc.stdout.readline()]):            print('read {!r}'.format(await line))event_loop = asyncio.get_event_loop()try:    event_loop.run_until_complete(run_df())finally:    event_loop.close()

redirecting the from stdout

based on your example this is a really simple solution. It just redirects the stderr to stdout and only stdout is read.

from subprocess import Popen, PIPE, CalledProcessError, run, STDOUT import osdef execute(cmd):    with Popen(cmd, stdout=PIPE, stderr=STDOUT, bufsize=1) as p:        while 1:            print('waiting for a line')            print(p.stdout.readline())cmd2 = ["python", "-u", "-m", "http.server"]execute(cmd2)