Technitribe

interesting problems (and a few solutions, too)

Technitribe
  • About the Authors
  • Log In
  • Log Out
  • Lost Password
  • Register
  • Reset Password
    • 9 Sep 2019

      Using jq to filter an array of objects from JSON

      Written by Tim Bielawa

      For some reason it took me an unreasonable amount of time to figure out how to filter an array (or list) of objects from a JSON stream. Every single example I found was a little too weird for me, or resulted in printing each object, but not in a final array format. Here’s what I came up with:

      Say for example you are parsing the AWS IP ranges JSON stream, you will receive an object like this:

      {
        "syncToken": "1567728788",
        "createDate": "2019-09-06-00-13-08",
        "prefixes": [
          {
            "ip_prefix": "18.208.0.0/13",
            "region": "us-east-1",
            "service": "AMAZON"
          },
          ... more objects here ...

      I was attempting to filter this down to ONLY objects where the service attribute was AMAZON. Using this jql I would get objects printed one after the other which is not what I wanted:

      $ jq -c '.prefixes[] | select(.service=="AMAZON")' < ip-ranges.json | head
      {"ip_prefix":"18.208.0.0/13","region":"us-east-1","service":"AMAZON"}
      {"ip_prefix":"52.95.245.0/24","region":"us-east-1","service":"AMAZON"}
      {"ip_prefix":"99.77.142.0/24","region":"ap-east-1","service":"AMAZON"}

      The correct syntax was ultimately very similar. 

      $ jq '.prefixes | map(. | select(.service=="AMAZON"))' < ip-ranges.json  | head
      [
      {
      "ip_prefix": "18.208.0.0/13",
      "region": "us-east-1",
      "service": "AMAZON"
      },

      Now we are getting each object returned as a member of an array. The difference is that we’re putting the .prefixes array objects into the map function and telling it to iterate every object through the select function. The map takes all of those matching objects and returns them as an array, whereas, previously we were only selecting objects that matched our select criteria. To get the objects back in a list we required the map. 

      • Tags »
      • amazon array filter jq json

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    • The Authors
    • Virtual Disk Guide

      Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.


      Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.

    • bitmath

      bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.

    • latest posts

      • Querying block device sizes in Python on Linux and Mac OS X February 4, 2023
      • Using jq to filter an array of objects from JSON September 9, 2019
      • Two Year Break — And we’re back! November 16, 2018
    • tags

      bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP
    • h4ck teh world

      tbielawatbielawa
      • Issue Comment
        bitmath
        February 6, 2023 - 12:55 am UTC
      • Issue Comment
        bitmath
        February 6, 2023 - 12:54 am UTC
      • Push
        bitmath
        February 6, 2023 - 12:51 am UTC
      • Issue Comment
        bitmath
        February 6, 2023 - 12:36 am UTC
      • Push
        bitmath
        February 6, 2023 - 12:30 am UTC

Creative Commons License
Technitribe by Tim Bielawa is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.