Run programs in background and redirect their outputs to file in real time

The -u switch and the equivalent PYTHONUNBUFFERED environment variable forces stdout to be unbuffered. Try this:

#!/bin/bashpython -u 1.py > 1.output &python -u 2.py > 2.output &python -u 3.py > 3.output &

or

#!/bin/bashexport PYTHONUNBUFFERED=yespython 1.py > 1.output &python 2.py > 2.output &python 3.py > 3.output &

Note that -u has side effects: read the doc to learn more.

Reference:

• https://docs.python.org/2/using/cmdline.html#cmdoption-u
• https://docs.python.org/2/using/cmdline.html#envvar-PYTHONUNBUFFERED

If you can find some sort of main loop in the program, that could be a good indicator of progress, it would be good to do write-to-file. If you say that the output does not fill until the output stream is closed by Python, then this is probably the simplest alternative.

You can try to override stderr/stdout in your scripts by doing:

# in the beginning of your scriptimport osimport sysdesired_output_file = open('/path/to/file', 'a')os.dup2(desired_output_file.fileno(), sys.stdout)os.dup2(desired_output_file.fileno(), sys.stderr)

Then all print calls will write to this special file, not to stdout. However, main problem could be with buffered IO. You have to somehow flush content of such file to the disk.