Skip to content
Advertisement

Tag: mapreduce

Count lines in large files

I commonly work with text files of ~20 Gb size and I find myself counting the number of lines in a given file very often. The way I do it now it’s just cat fname | wc -l, and it takes very long. Is there any solution that’d be much faster? I work in a high performance cluster with Hadoop

Advertisement