set length of captured group

I would like to format the log of my server to be readable easily when I do a “tail -f”.

My log look like :

{"item1":"ab","item2":"cdef","item3":"ghi"}
{"item1":"abc","item2":"defgh","item3":"i"}

I would like to obtain :

var1=ab__  | var2=cdef_ | var3=ghi
var1=abc_  | var2=defgh | var3=i__

Where ‘_’ is a white space.

For now I use :

 sed -E 's/.*item1":"(.*)","item2":"(.*)","item3":"(.*)".*/var1=1 | var2=2 | var3=3/'

And I get :

var1=ab | var2=cdef | var3=ghi
var1=abc | var2=defgh | var3=i

It is possible to set the length of captured group 1 2 3 ? Fill them with space or truncate them?

I would like to fix the length of each field ( fill or truncate ) and to stream my log in real time.

Answer

Comments are already recommending jq.

Here’s one way to do it:

$ jq -r '[(.item1, .item2, .item3)] | @tsv' log.json
ab      cdef    ghi
abc     defgh   i

^{Note that the @tsv filter was introduced with jq version 1.5, so if you’re on 1.4, perhaps it’s time to upgrade. 🙂}

If you want the vertical bars, you can just add them within the array:

$ jq -r '[(.item1, "|", .item2, "|", .item3)] | @tsv' log.json
ab      |       cdef    |       ghi
abc     |       defgh   |       i

Note that @tsv adds a tab between every field, so this may not be exactly what you want. It can, however, could easily be parsed by bash:

$ while IFS=$'t' read one two three; do 
  printf 'var1=%-4s  | var2=%-5s | var3=%-4sn' "$one" "$two" "$three"; 
  done < <(jq -r '[(.item1, .item2, .item3)] | @tsv' log.json)
var1=ab    | var2=cdef  | var3=ghi
var1=abc   | var2=defgh | var3=i

Or if the specific format isn’t important and you just want things line up, perhaps this:

$ jq -r '[(.item1, "|", .item2, "|", .item3)] | @tsv' log.json | column -t
ab   |  cdef   |  ghi
abc  |  defgh  |  i

Of course, this doesn’t handle tail -f, it just covers formatting. If you want to be able to process a stream, you will likely need to do that in a loop. For example:

tail -F "$logfile" | while read -r line; do
  jq -r '[(.item1, .item2, .item3)] | @tsv' <<< "$line" |
  while IFS=$'t' read one two three; do
    printf 'var1=%-4s  | var2=%-5s | var3=%-4sn' "$one" "$two" "$three"
  done
done

Note that the formatting option selected here is the printf one, because all the other solutions require knowledge of the maximum length of input data. With printf, we’ll assume that you already know the maximum length, and have accounted for it in your format string.