Using variables as the input to command

I’ve scoured various message boards to understand why I can’t use a variable as input to a command in certain scenarios. Is it a STDIN issue/limitation? Why does using echo and here strings fix the problem?

For example,

~$ myvar=$(ls -l)
~$ grep Jan "$myvar"
grep: total 9
-rwxr-xr-x 1 jvp jvp 561 Feb  2 23:59 directoryscript.sh
-rw-rw-rw- 1 jvp jvp   0 Jan 15 10:30 example1
drwxrwxrwx 2 jvp jvp   0 Jan 19 21:54 linuxtutorialwork
-rw-rw-rw- 1 jvp jvp   0 Jan 15 13:08 littlefile
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 man
drwxrwxrwx 2 jvp jvp   0 Feb  2 20:33 projectbackups
-rwxr-xr-x 1 jvp jvp 614 Feb  2 20:41 projectbackup.sh
drwxrwxrwx 2 jvp jvp   0 Feb  2 20:32 projects
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test1
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test2
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test3: File name too long

As you can see I get the error… ‘File name too long’

Now, I am able to get this to work by using either:

  echo "$myvar" | grep Jan  

  grep Jan <<< "$myvar"

However, I’m really after a better understanding of why this is the way it is. Perhaps I am missing something about basics of command substitution or what is an acceptable form of STDIN.

Answer

The grep utility can operate…

1. On files the names of which are provided on the command line, after the regular expression used for matching
2. On a stream supplied on its standard input.

You are doing this :

myvar=$(ls -l)
grep Jan "$myvar"

This provides the content of variable myvar as an argument to the grep command, and since it is not a file name, it does not work.

There are many ways to achieve your goal. Here are a few examples.

Use the content of the variable as a stream connected to the standard input of grep, with one of the following methods (all providing the same output) :

grep Jan <<<"$myvar" 
echo "$myvar" | grep Jan
grep Jan < <(echo "$myvar")

Avoid the variable to start with, and send the output of ls directly to grep :

ls -l | grep Jan
grep Jan < <(ls -l)

Provide grep with an expression that actually is a file name :

grep Jan <(ls -l)

The <(ls -l) expression is syntax that causes a FIFO (first-in first-out) special file to be created. The ls -l command sends its output to FIFO. The expression is converted by Bash to an actual file name that can be used for reading.

To clear any confusion, the two statements below (already shown above) look similar, but are fundamentally very different :

grep Jan <(ls -l)
grep Jan < <(ls -l)

In the first one, grep receives a file name as an argument and reads this files. In the second case, the additional < (whitespace between the two < is important) creates a redirection that reads the FIFO and feeds its output to the standard input of grep. There is a FIFO in both cases, but it is presented in a totally different way to the command.