Skip to content
Advertisement

Extracting lines by pattern matching from multiple files and writing them to another file in linux

I have 20 files. I want to extract lines by matching pattern '<script src="{%.*%}>' from those 20 files and write those lines to new file.

I also want to delete those lines from the original files.

Is there a better way to do it other than what I have tried which is terrible?

This is my attempt:

import os
import sh

folder_path='/home/username/folder/'
match_phrase = '<script src="{%'
new_file = '/home/username/file.txt'

files = os.listdir(folder_path)
print files
for file in files:
    full_filename = folder_path + file
    lines=[]
    line_nos=[]

    with open(full_filename) as myFile:
        for num, line in enumerate(myFile, 1):
            if match_phrase in line:
                line_nos.append(num)
                lines.append(line)
        print lines
        print line_nos

        with open(new_file,'a') as newfile:
            for line in lines:
                newfile.write(line)

    for del_line in line_nos:
        print "deleting line %s from file=%s"%(del_line,full_filename)
        del_line=str(del_line)+'d'
        sh.sed('-i',del_line,full_filename)

Advertisement

Answer

Using sed:

sed -i -e "/$pattern/w $newfile" -e "/$pattern/d" $files

Sed‘s w command writes match to some other file. d deletes it.


example:

$ pattern='<script src="{%'
$ files=/home/username/folder/*
$ newfile=/home/username/file.txt
$ 
$ sed -i -e "/$pattern/w $newfile" -e "/$pattern/d" $files
User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement