I have a command that takes a long time that I like to run in the background like this:
python3 script.py -f input.key -o output >> logs/script.log 2>&1 &
This works perfectly in the sense that the command is indeed in the background and I can check the output and potential errors later.
The main problem is the output is only appended after the command is completely finished, whereas I would like to have up-to-date log messages so I check the progress.
So currently the log would be empty and than suddenly at 08:30 two lines would appear:
[08:00] Script starting... [08:30] Script finished!
Instead, I would like to have output saved to file before the command is completely finished.
Advertisement
Answer
Since you are calling a Python script you would want to use the -u option, which forces the stdout and stderr streams to be unbuffered.
$ python3 -u script.py -f input.key -o output >> logs/script.log 2>&1 &
You can check the log periodically using cat or realtime in combination with watch:
$ watch cat logs/script.log
↳ https://docs.python.org/3.7/using/cmdline.html#cmdoption-u