C Linux Pipe Reading Out-of-Order. Missing, and Duplicated Output

I am working on an application that needs to launch arbitrary child processes and read their output while they are running. It might be useful to think of it as a terminal emulator because it has to be able to run child processes many times throughout it’s lifetime and read back their output, and I want it to read back their output the same way the terminal would show it.

I also don’t have control over the code of the child processes, so I can’t just solve this problem by flushing more often in the child process code although flushing after every print does seem to help most (all?) of the issues.

I setup the following small demonstration that exhibits the problem and I can’t fix it in this small demonstration either:

The setup:

exerciser – A program that prints some stuff, and launches a child process that also prints some stuff. Their prints are setup so that both processes could have the “A” buffered unless stdout is flushed before the fork.

reader – Launches a child process and sets up a pipe to read it’s output. Prints immediately everything it reads from the child. The child runs the exerciser program.

—-terminal output—-

<...>/pipe-testing$ ./exerciser
A: 10
B: 42
C: 42
C: 10
<...>/pipe-testing$ ./reader
READER...
A: 10
B: 42
C: 42
A: 10
C: 10

—-exerciser.cpp—-

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

int main(){
    int n = 10, status = 0;
    fprintf(stdout, "A: %dn", n);
    if (fork() == 0){
        n = 42;
        fprintf(stdout, "B: %dn", n);
    }
    else{
        wait(&status);
    }
    fprintf(stdout, "C: %dn", n);
    return(0);
}

—-reader.cpp—-

#include <stdio.h>
#include <unistd.h>

int main(){
    fprintf(stdout, "READER...n");
    fflush(stdout);

    int pipe_fds[2];
    if (pipe(pipe_fds) == -1){
        fprintf(stdout, "failed to make pipen");
        return(0);
    }

    pid_t child_pid = fork();
    if (child_pid == -1){
        fprintf(stdout, "failed to forkn");
        return(0);
    }

#define PIPE_READ 0
#define PIPE_WRITE 1

    if (child_pid == 0){
        close(pipe_fds[PIPE_READ]);
        dup2(pipe_fds[PIPE_WRITE], STDOUT_FILENO);
        dup2(pipe_fds[PIPE_WRITE], STDERR_FILENO);

        char *argv[] = {
            "sh",
            "-c",
            "./exerciser",
            0
        };

        if (execv("/bin/sh", argv) == -1){
            fprintf(stdout, "failed to execvn");
            return(0);
        }
    }
    else{
        close(pipe_fds[PIPE_WRITE]);

        char buffer[1024];
        int capacity = sizeof(buffer);
        for (;;){
            ssize_t num = read(pipe_fds[PIPE_READ], buffer, capacity);
            if (num == -1){
                fprintf(stdout, "read errorn");
                return(0);
            }
            else if (num == 0){
                break;
            }
            else{
                fprintf(stdout, "%.*s", (int)num, buffer);
                fflush(stdout);
            }
        }
    }

    return(0);
}

Answer

I also don’t have control over the code of the child processes, so I can’t just solve this problem by flushing more often in the child process code although flushing after every print does seem to help most (all?) of the issues.

Then there’s nothing you can do, because the problem is in the child process. The child process calls fork, giving its child the same output stream it has. Then both processes write to that same output stream with no protection or synchronization whatsoever. There is nothing to stop the exerciser parent process and the child process from writing to their streams alternately or concurrently.

It’s no different from doing this in a shell script:

echo -n "test" &
echo -n "ing" &

The output might be “testing”, but it also could be “ingtest” or “tiensgt” or many other things.

Since it is the exerciser process that does this, it is the exerciser program that must be fixed if you expect coherent output. There’s no way to untangle the mess that comes out of the pipe other than not putting the mess into the pipe in the first place.