I have two set of folders that have files with the same filenames and structure. The folder structure is something like this:
outputfolder |---folder1 | |---file1.txt | |---file2.txt | |---folder2 |---file1.txt |---file2.txt
So what I need to do is to combine (append) all the files with the same name in these folders (file1.txt with file1.txt etc.) into another file inside the outputfolder. After getting these combined files I also need to create a tar.gz file from all of these combined files.
How can I accomplish this in a Linux based command line environment? The folder name (folder1 and folder2 etc) is variable so this needs to be given but the files need not and it should automatically combine all the files with the same name.
Also, these files have headers for column names, so I would need to remove that as well while appending.
Advertisement
Answer
Here’s some code to get you started
topdir=outputfolder dir1=folder1 dir2=folder2 for f in $topdir/$dir1/*.txt do outf=$topdir/`basename $f .txt`-concat.txt cp $f $outf sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf done tar czf foo.tar.gz $topdir/*-concat.txt
Edit: added the part removing the header of the 2nd file.