Pipes in Bash: One at a Time (Line by Line) or All at Once

$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

First, I create 5 files

$ touch a b c d e
$ ls
a  b  c  d  e

In the following example, xargs seems to get all the results from find via the pipe all at once:

$ find . -type f | xargs -t echo | wc -w
echo ./c ./b ./a ./d ./e
5

In the following example, xargs seems to get all the results from find via the pipe one at a time (line by line):

$ find . -type f | xargs -t -i mv {} {}.bak | wc  -w
mv ./c ./c.bak
mv ./b ./b.bak
mv ./a ./a.bak
mv ./d ./d.bak
mv ./e ./e.bak
0

In the following example, xargs seems to get all the results from ‘find’ via the pipe one at a time (line by line); and wc seems to get all the results from xargs via the pipe all at once.

Otherwise, if wc got the results line by line, you would see 1 appear 5 times instead of a 5 at the end.

$ find . -type f | xargs -t -i echo {} | wc -w
echo ./e.bak
echo ./b.bak
echo ./c.bak
echo ./a.bak
echo ./d.bak
5

So do pipes get results from previous command One at a Time (Line by Line) or All at Once?

Answer

The behavior you are observing comes from the xargs options you are using. The -i option specifically instructs xargs to create one new process for each input token, and replace the {} placeholder with that token.

Pipes by themselves do not stipulate any specific behavior; many kernels will pass one full I/O buffer at a time (hence the many questions about I/O buffering with pipes, along the lines of “this command doesn’t seem to do anything in a pipe” when the actual symptom is just that the output buffer isn’t full yet).