Skip to content
Advertisement

jq to remove one of the duplicated objects

I have a json file like this:

{"caller_id":"123321","cust_name":"abc"}
{"caller_id":"123443","cust_name":"def"}
{"caller_id":"123321","cust_name":"abc"}
{"caller_id":"234432","cust_name":"ghi"}
{"caller_id":"123321","cust_name":"abc"}
....

I tried:

jq -s 'unique_by(.field1)' 

but this will remove all the duplicated items, I,m looking to keep just one of the duplicated items, to get the file like this:

{"caller_id":"123321","cust_name":"abc"}
{"caller_id":"123443","cust_name":"def"}
{"caller_id":"234432","cust_name":"ghi"}
....

Advertisement

Answer

With field1, I doubt you are getting anything in the output, since there is no key/field with the given name. If you simply change your command to jq -s 'unique_by(.caller_id)' it will give you desired result containing unique & sorted objects based on caller_id key. It will ensure in result you have atleast & atmost one object for each caller_id.

NOTE: Same as what @Jeff Mercado has explained in the comments.

User contributions licensed under: CC BY-SA
5 People found this is helpful
Advertisement