I would like to format the log of my server to be readable easily when I do a “tail -f”.
My log look like :
{"item1":"ab","item2":"cdef","item3":"ghi"} {"item1":"abc","item2":"defgh","item3":"i"}
I would like to obtain :
var1=ab__ | var2=cdef_ | var3=ghi var1=abc_ | var2=defgh | var3=i__
Where ‘_’ is a white space.
For now I use :
sed -E 's/.*item1":"(.*)","item2":"(.*)","item3":"(.*)".*/var1=1 | var2=2 | var3=3/'
And I get :
var1=ab | var2=cdef | var3=ghi var1=abc | var2=defgh | var3=i
It is possible to set the length of captured group 1 2 3 ? Fill them with space or truncate them?
I would like to fix the length of each field ( fill or truncate ) and to stream my log in real time.
Advertisement
Answer
Comments are already recommending jq
.
Here’s one way to do it:
$ jq -r '[(.item1, .item2, .item3)] | @tsv' log.json ab cdef ghi abc defgh i
Note that the @tsv
filter was introduced with jq
version 1.5, so if you’re on 1.4, perhaps it’s time to upgrade. 🙂
If you want the vertical bars, you can just add them within the array:
$ jq -r '[(.item1, "|", .item2, "|", .item3)] | @tsv' log.json ab | cdef | ghi abc | defgh | i
Note that @tsv
adds a tab between every field, so this may not be exactly what you want. It can, however, could easily be parsed by bash:
$ while IFS=$'t' read one two three; do printf 'var1=%-4s | var2=%-5s | var3=%-4sn' "$one" "$two" "$three"; done < <(jq -r '[(.item1, .item2, .item3)] | @tsv' log.json) var1=ab | var2=cdef | var3=ghi var1=abc | var2=defgh | var3=i
Or if the specific format isn’t important and you just want things line up, perhaps this:
$ jq -r '[(.item1, "|", .item2, "|", .item3)] | @tsv' log.json | column -t ab | cdef | ghi abc | defgh | i
Of course, this doesn’t handle tail -f
, it just covers formatting. If you want to be able to process a stream, you will likely need to do that in a loop. For example:
tail -F "$logfile" | while read -r line; do jq -r '[(.item1, .item2, .item3)] | @tsv' <<< "$line" | while IFS=$'t' read one two three; do printf 'var1=%-4s | var2=%-5s | var3=%-4sn' "$one" "$two" "$three" done done
Note that the formatting option selected here is the printf
one, because all the other solutions require knowledge of the maximum length of input data. With printf
, we’ll assume that you already know the maximum length, and have accounted for it in your format string.