Skip to content
Advertisement

Unexpected characters coming in bash in while loop

I am trying to get IP of a few 1000 domains… below is my program…

#!/bin/bash
yesterday=$(date --date "1 days ago" +%F)

while read -r line comment
do
  
    ip=$(host -4 $line | head -1 | awk -F" " '{print $5}')
  
    echo "$ip"

done <  /home/domainsa/public_html/data/domains/$yesterday-domains.txt

it shows me

sh -x domains-ip.sh

++ date --date '1 days ago' +%F
+ yesterday=2021-05-02
+ read -r line comment
++ host -4 $'0-96.comr'
++ head -1
++ awk '-F ' '{print $5}'
+ ip='3(NXDOMAIN)'
+ echo '3(NXDOMAIN)'
3(NXDOMAIN)
+ read -r line comment
++ host -4 $'0-roll.comr'
++ head -1
++ awk '-F ' '{print $5}'
+ ip='3(NXDOMAIN)'
+ echo '3(NXDOMAIN)'
3(NXDOMAIN)
+ read -r line comment
++ host -4 $'000850.xyzr'
++ head -1
++ awk '-F ' '{print $5}'
+ ip='3(NXDOMAIN)'
+ echo '3(NXDOMAIN)'
3(NXDOMAIN)
+ read -r line comment
++ host -4 $'00097971.comr'
++ head -1
++ awk '-F ' '{print $5}'
+ ip='3(NXDOMAIN)'
+ echo '3(NXDOMAIN)'
3(NXDOMAIN)

$ and ‘ and r is not something I was expecting…

Any idea where I am doing wrong?

I check While Loop in Bash Unexpected Character but the selected answer I am also using…

PS: is there any better and quick bash command line I am missing to get IP address of domains… This is all I know, so wrote this program.

[EDIT]

I am using awk -F" " '{print $4}

Advertisement

Answer

Your file ...domains.txt contains carriage returns (r). These non-printable characters are shown by bash by using C-style strings ($'someString') so that the non-printable carriage is visible.

You probably have windows line endings (rn) instead of linux line endings (n). You could convert the file by using dos2unix. But just to be sure, you can remove all r in the file with tr -d \r < ...dommains.txt.

#!/bin/bash
yesterday=$(date --date "1 days ago" +%F)
tr -d \r < "/home/domainsa/public_html/data/$yesterday-domains.txt" |
while read -r line comment
do
    host -4 "$line" | head -n1 | awk -F' ' '{print $5}'
done
User contributions licensed under: CC BY-SA
5 People found this is helpful
Advertisement