Skip to content

Is there any faster way to grep the pattern or to increase the speed of while loop?

codeforester
sam ali

this loop is taking time because lacks of entry in main_file. 

while read line;do  
         cat main_file | grep "$line";
done<file

cat file

pattern_1
pattern_2

cat main_file
  main  pattern_1
  main  pattern_2
  main pattern_2

Advertisement

Answer

Your current approach is very inefficient – the whole loop could be done in a single grep, with the -f option.

grep -Fxf file main_file
  • -F treats lines in file as strings, not patterns
  • -x looks for exact matching line (if that is what you want)
  • -f file reads the lines from file and looks for them in main_file

The above approach will work well as long as the files are small. For larger files, use awk:

awk 'FNR==NR {hash[$1]; next} $2 in hash' file main_file

For details, look at this post – it had other solutions as well: