Python subprocess with real-time input and multiple consoles Python subprocess with real-time input and multiple consoles python python

Python subprocess with real-time input and multiple consoles


The issue you're up against is the architecture of the console subsystem on Windows, the console window that you normally see is not hosted by cmd.exe but instead by conhost.exe, a child process of a conhost window can only connect to a single conhost instance meaning you're limited to a single window per process.

This then leads on to having an extra process for each console window you wish to have, then in order to look at displaying anything in that window you need to look at how stdin and stdout are normally handled, in that they are written and read from by the conhost instance, except if you turn stdin into a pipe (so you can write to the process) it no longer comes from conhost but instead from your parent process and as such conhost has no visibility of it. This means that anything written to stdin is only read by the child process so is not displayed by conhost.

As far as I know there isn't a way to share the pipe like that.

As a side effect if you make stdin a pipe then all keyboard input sent to the new console window goes nowhere, as stdin is not connected to that window.

For an output only function this means you can spawn a new process that communicates with the parent via a pipe to stdin and echos everything to stdout.

Heres an attempt:

#!python3import sys, subprocess, timeclass Console():    def __init__(self):        if '-r' not in sys.argv:            self.p = subprocess.Popen(                ['python.exe', __file__, '-r'],                stdin=subprocess.PIPE,                creationflags=subprocess.CREATE_NEW_CONSOLE                )        else:            while True:                data = sys.stdin.read(1)                if not data:                    break                sys.stdout.write(data)    def write(self, data):        self.p.stdin.write(data.encode('utf8'))        self.p.stdin.flush()if (__name__ == '__main__'):    p = Console()    if '-r' not in sys.argv:        for i in range(0, 100):            p.write('test %i\n' % i)            time.sleep(1)

So a nice simple pipe between two processes and echoing the input back to the output if its the subprocess, I used a -r to signify whether the instance is a process but there are other ways depending on how you implement it.

Several things to note:

  • the flush after writing to stdin is needed as python normally uses buffering.
  • the way this approach is written is aimed at being in its own module hence the use of __file__
  • due to the use of __file__ this approach may need modification if frozen using cx_Freeze or similar.

EDIT 1

for a version that can be frozen with cx_Freeze:

Console.py

import sys, subprocessclass Console():    def __init__(self, ischild=True):        if not ischild:            if hasattr(sys, 'frozen'):                args = ['Console.exe']            else:                args = [sys.executable, __file__]            self.p = subprocess.Popen(                args,                stdin=subprocess.PIPE,                creationflags=subprocess.CREATE_NEW_CONSOLE                )        else:            while True:                data = sys.stdin.read(1)                if not data:                    break                sys.stdout.write(data)    def write(self, data):        self.p.stdin.write(data.encode('utf8'))        self.p.stdin.flush()if (__name__ == '__main__'):    p = Console()

test.py

from Console import Consoleimport sys, timeif (__name__ == '__main__'):    p = Console(False)    for i in range(0, 100):        p.write('test %i\n' % i)        time.sleep(1)

setup.py

from cx_Freeze import setup, Executablesetup(    name = 'Console-test',    executables = [        Executable(            'Console.py',            base=None,            ),        Executable(            'test.py',            base=None,            )        ])

EDIT 2

New version that should work under dev tools like IDLE

Console.py

#!python3import ctypes, sys, subprocessKernel32 = ctypes.windll.Kernel32class Console():    def __init__(self, ischild=True):        if ischild:            # try allocate new console            result = Kernel32.AllocConsole()            if result > 0:                # if we succeed open handle to the console output                sys.stdout = open('CONOUT$', mode='w')        else:            # if frozen we assume its names Console.exe            # note that when frozen 'Win32GUI' must be used as a base            if hasattr(sys, 'frozen'):                args = ['Console.exe']            else:                # otherwise we use the console free version of python                args = ['pythonw.exe', __file__]            self.p = subprocess.Popen(                args,                stdin=subprocess.PIPE                )            return        while True:            data = sys.stdin.read(1)            if not data:                break            sys.stdout.write(data)    def write(self, data):        self.p.stdin.write(data.encode('utf8'))        self.p.stdin.flush()if (__name__ == '__main__'):    p = Console()

test.py

from Console import Consoleimport sys, timeif (__name__ == '__main__'):    p = Console(False)    for i in range(0, 100):        p.write('test %i\n' % i)        time.sleep(1)

setup.py

from cx_Freeze import setup, Executablesetup(    name = 'Console-test',    executables = [        Executable(            'Console.py',            base='Win32GUI',            ),        Executable(            'test.py',            base=None,            )        ])

This could be made more robust, i.e. always checking for an existing console and detaching it if found before creating a new console, and possibly better error handling.


Since you are on windows you can use win32console module to open a second console or multiple consoles for your thread or subprocess output. This is the most simple and easiest way that works if you are on windows.

Here is a sample code:

import win32consoleimport multiprocessingdef subprocess(queue):    win32console.FreeConsole() #Frees subprocess from using main console    win32console.AllocConsole() #Creates new console and all input and output of subprocess goes to this new console    while True:        print(queue.get())        #prints any output produced by main script passed to subprocess using queueif __name__ == "__main__":     queue = multiprocessing.Queue()    multiprocessing.Process(target=subprocess, args=[queue]).start()    while True:        print("Hello World in main console")        queue.put("Hello work in sub process console")        #sends above string to subprocess and it prints it into its console        #and whatever else you want to do in ur main process

You can also do this with threading. You have to use queue module if you want the queue functionality as threading module doesn't have queue

Here is the win32console module documentation