I have a legacy app running that sends its output to log files. Rather than tail and manage those log files, I want to capture the output before it hits the disk and send it off elsewhere on the network (using syslog, or fluentd, or logstash, etc.).
Is there any way I can capture the output without changing the application code itself? I do have the option of changing the process startup, so I can “wrap” it.
I thought of a few options:
- Named pipe – it would work, but subject to pipe buffer and broken pipe limitations.
- Standard redirect – really only works for known file descriptors, i.e. stdin, stdout, stderr / 0,1,2
Is there any sane way to interject my program so that the app thinks it is still writing to /var/log/myapp.log but instead is being sent to my log controller which can then do as it wishes?
Advertisement
Answer
Use named pipes. The pipe buffer size isn’t something you should need to worry about and you shouldn’t get SIGPIPES if the reader end doesn’t die (or close the connection early).
Your reader could be as simple as
<named_pipe nc some_address some_port &
If you were to kill it while your writer was writing to it, the writer would get a SIGPIPE.
As for the buffer size concern, as long as ncing to some_address someport is faster or as fast as the rate at which data is being written to the named pipe, the writer process won’t get blocked. If the writer is faster, it will when the buffer fills up. (But then, writing to disk or the network will get you blocked too).
You don’t have to worry about the system-determined pipe buffer size; simply modify buffering at the reader end of the pipe (e.g., you can set the TCP output buffer size of nc, which will have the same effect as increasing the pipe buffer size) if the pipe buffer size is too small for you.
If you’re launching the programs from bash and your app takes the log file as an argument, you can simply give it >( nc some_address some_port), which will create an anonymous named pipe for you.