Technitribe

For some reason it took me an unreasonable amount of time to figure out how to filter an array (or list) of objects from a JSON stream. Every single example I found was a little too weird for me, or resulted in printing each object, but not in a final array format. Here’s what I came up with:

Say for example you are parsing the AWS IP ranges JSON stream, you will receive an object like this:

{
  "syncToken": "1567728788",
  "createDate": "2019-09-06-00-13-08",
  "prefixes": [
    {
      "ip_prefix": "18.208.0.0/13",
      "region": "us-east-1",
      "service": "AMAZON"
    },
    ... more objects here ...

I was attempting to filter this down to ONLY objects where the service attribute was AMAZON. Using this jql I would get objects printed one after the other which is not what I wanted:

$ jq -c '.prefixes[] | select(.service=="AMAZON")' < ip-ranges.json | head
{"ip_prefix":"18.208.0.0/13","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"52.95.245.0/24","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"99.77.142.0/24","region":"ap-east-1","service":"AMAZON"}

The correct syntax was ultimately very similar. 

$ jq '.prefixes | map(. | select(.service=="AMAZON"))' < ip-ranges.json  | head
[
  {
    "ip_prefix": "18.208.0.0/13",
    "region": "us-east-1",
    "service": "AMAZON"
  },

Now we are getting each object returned as a member of an array. The difference is that we’re putting the .prefixes array objects into the map function and telling it to iterate every object through the select function. The map takes all of those matching objects and returns them as an array, whereas, previously we were only selecting objects that matched our select criteria. To get the objects back in a list we required the map.