Expanding shell variable in sed in remote host

The sed command, which is run locally looks like this (it works fine):

sed -i "|$IFM_MOUNT|d" /opt/genApp.cfg

However, if i try t run the same in a remote host doing an ssh, it gives me issue:

ssh root@$host  'sed -i "|$IFM_MOUNT|d" /opt/genApp.cfg'
Password:
sed: -e expression #1, char 0: no previous regular expression

What am i missing here?

Answer

Variable Expansion

The problem you’re encountering has to do with shell expansion in strings. There are two relevant categories of strings:

• singly-quoted (surrounded by '), in which variables are not expanded, and
• doubly-quoted (surrounded by "), in which they are, and in which a number of special characters (such as $) need to be escaped to retain their literal meaning.

Arthur “Two Shells” Jackson (scnr)

In your case, there are two shells involved that can expand variables: the local shell and the remote shell (that the remote sshd spawns for you). So let’s break it down: with the command

ssh root@$host 'sed -i "|$IFM_MOUNT|d" /opt/genApp.cfg'

the local shell sees the singly-quoted string 'sed -i "|$IFM_MOUNT|d" /opt/genApp.cfg'. Because it is a singly-quoted string, variable expansion does not happen, and this string is passed verbatim to the remote shell. The remote shell, accordingly, sees the command

sed -i "|$IFM_MOUNT|d" /opt/genApp.cfg

Here, "|$IFM_MOUNT|d" is a doubly-quoted string, and the shell accordingly expands $IFM_MOUNT. Because $IFM_MOUNT does not have a value in the remote shell, the replacement is the empty string, so the command ends up as

sed -i "||d" /opt/genApp.cfg

…whereupon sed sees the code ||d, which leads to the error you see (an empty regex clause in sed code reattempts the most recently attempted regex, which only works if a most recently attempted regex exists).

Solution (read the Caveats section below!)

Since it appears that you want to replace $IFM_MOUNT with the value it has in the local shell rather than the remote one, the most straightforward fix is to switch the quotation marks around, i.e.

ssh "root@$host" "sed -i '\|$IFM_MOUNT|d' /opt/genApp.cfg"

Now the local shell sees a doubly-quoted string "sed -i '\|$IFM_MOUNT|d' /opt/genApp.cfg" into which it dutifully substitutes $IFM_MOUNT, which means that the remote shell sees something like sed -i '|/mnt/ifm|d' /opt/genApp.cfg (if $IFM_MOUNT was /mnt/ifm).

Notes

The escaped backslash is not strictly necessary in this particular case, but since there are contexts in which backslashes need to be escaped in doubly-quoted strings (e.g., "\$foo") to retain their literal meaning, I consider it a good habit to do it all the time when I mean to retain the literal meaning of a backslash.

Also, expanding shell variables outside double quotes is rarely a good idea (because the shell will split the expanded string if it contains whitespace¹), so I took the liberty of putting "root@$host" between double quotes to prevent splitting. It is unlikely to work if $host contains whitespace, but this way it will fail in a more predictable manner if such broken input appears.

¹ More precisely: Anything in $IFS, which means space, tab, and newline unless you fiddled with IFS.

Caveats (read this!)

Caveats to consider: There are two substitutions of text into code here, both of which carry the risk of code injection.

First, since $IFM_MOUNT is substituted directly into shell code, it can be used to inject shell commands into the remote shell. Consider, for example, a case in which

IFM_MOUNT="/mnt/ifm|d' /opt/genApp.cfg; rm -Rf /; #"

The remote (root!) shell would see

sed -i '|/mnt/ifm|d' /opt/genApp.cfg; rm -Rf /; #|d' /opt/genApp.cfg

That is not a good thing for it to see.

Second, the sed code also carries a possibility of code injection, particularly if GNU sed is on the other side (it can execute shell code). It seems hardly consequential in this case, where shell commands can be injected directly, but consider for the purpose of completeness the case that

IFM_MOUNT='/mnt/ifm|!d; #'

Whereupon the remote shell would see

sed -i '|/mnt/ifm|!d; #|d' /opt/genApp.cfg

…which would throw out everything except the lines you wanted thrown out. It becomes worse with GNU sed and the likes of IFM_MOUNT='/mnt/ifm/|!d; e rm -Rf/; #, but again, there already is a way to inject shell commands, so it doesn’t really matter in this case.

What this means is that IFM_MOUNT should not come from an untrustworthy source, and you should be damn sure it doesn’t contain any characters you don’t expect.