Answer by Elias for How to capture terminal output as execution proceeds?

You can do as in the following example.

Suppose we have a script called myscript.py that looks like this:

import time
for x in range(0, 33):
   print("hello", x, flush=True)
   time.sleep(1)

Then if we run it like this:

python3 myscript.py > mylog.txt

it will hang there until it completes, so we will not see the output while it is running.

To be able to see the output while it is running, we can do this instead:

python3 myscript.py > mylog.txt &

where the ampersand means that the terminal will not hang, we can give more commands while it is running. Then, we can do this to look at the contents of the log file while myscript.sh is writing to it:

tail -f mylog.txt

(Another possibility is to open a separate terminal window and do tail -f mylog.txt from there.)

Note that for this to work, the flush=True part of the print line is important, without flush=True the file contents will only be seen after the program completes.

In case the output is generated from some other program inside your python script, you can try adding import sys and then doing sys.stdout.flush() in the python code after calling the other program. See https://stackoverflow.com/a/230774/6708867