Skip to content
Advertisement

Display duplicate lines in two different files

I have two files and I would like to display the duplicate line. I tried this but it doesn’t work :

cat id1.txt | while read id; do grep "$id" id2.txt; done

I am wondering if there are any other way to display the duplicate lines in the file. Both of my 2 files contain list of ids. Thank you.

Advertisement

Answer

Are the files sorted? Can they be sorted?

If sorted:

comm -12 id1.txt id2.txt

If not sorted but using bash 4.x:

comm -12 <(sort id1.txt) <(sort id2.txt)

There are solutions using temporary files if you don’t have bash 4.x and ‘process substitution’.

You could also use grep -F:

grep -F -f id1.txt id2.txt

This looks for the words in id1.txt that appear in id2.txt. The only problem here is ensuring that an ID 1 doesn’t match every ID containing a 1 somewhere. The -w or -x options available in some versions of grep will work here.

Advertisement