Technitribe

interesting problems (and a few solutions, too)

Technitribe
  • About the Authors
  • Log In
  • Log Out
  • Lost Password
  • Register
  • Reset Password
    • 9 Sep 2019

      Using jq to filter an array of objects from JSON

      Written by Tim Bielawa

      For some reason it took me an unreasonable amount of time to figure out how to filter an array (or list) of objects from a JSON stream. Every single example I found was a little too weird for me, or resulted in printing each object, but not in a final array format. Here’s what I came up with:

      Say for example you are parsing the AWS IP ranges JSON stream, you will receive an object like this:

      {
  "syncToken": "1567728788",
  "createDate": "2019-09-06-00-13-08",
  "prefixes": [
    {
      "ip_prefix": "18.208.0.0/13",
      "region": "us-east-1",
      "service": "AMAZON"
    },
    ... more objects here ...

      I was attempting to filter this down to ONLY objects where the service attribute was AMAZON. Using this jql I would get objects printed one after the other which is not what I wanted:

      $ jq -c '.prefixes[] | select(.service=="AMAZON")' < ip-ranges.json | head
      {"ip_prefix":"18.208.0.0/13","region":"us-east-1","service":"AMAZON"}
      {"ip_prefix":"52.95.245.0/24","region":"us-east-1","service":"AMAZON"}
      {"ip_prefix":"99.77.142.0/24","region":"ap-east-1","service":"AMAZON"}

      The correct syntax was ultimately very similar. 

      $ jq '.prefixes | map(. | select(.service=="AMAZON"))' < ip-ranges.json  | head
      [
        {
          "ip_prefix": "18.208.0.0/13",
          "region": "us-east-1",
          "service": "AMAZON"
        },

      Now we are getting each object returned as a member of an array. The difference is that we’re putting the .prefixes array objects into the map function and telling it to iterate every object through the select function. The map takes all of those matching objects and returns them as an array, whereas, previously we were only selecting objects that matched our select criteria. To get the objects back in a list we required the map. 

      0 Comments
    • 16 Nov 2018

      Two Year Break — And we’re back!

      Written by Tim Bielawa

      Where has Technitribe been for the last 2 years? The short answer: in domain limbo.

      Christmas Islands

      .cx is a Christmas Islands top-level domain (TLD). I did not have the domain set to auto-renew because I was trying to get away from my old registrar. At the time it was expiring I didn’t have the cash on hand to renew it. I will admit, I did not read all the Term & Conditions for the .cx TLD before registering. I guess I really screwed the pooch there.

      It turns out that if you have a .cxdomain, and it expires, then you have a problem. They will effectively hold it hostage for a period of 4 months. During this time you have the “option” to restore the domain, but at the cost of $400.00 USD.  Like I said, I was not very cash flush at the time.

      Waiting Period

      Shortly after the domain expired I moved across the country. While getting set up in my new location the domain left the restoration period and was open to general registration again. As is typical, a reseller had squatted the lnx.cx domain and bought it as soon as it became available. I resigned myself to my fate, I would have to let it go for a while.

      Return of lnx.cx

      The blog is now back! A few weeks ago I finally had the cash necessary to register it from a new registrar who isn’t evil (Netim). We have a new SSL Certificate issued by Let’s Encrypt, no more untrusted RapidSSL security alerts in your browser. And, as an aside, the setup for installing this new certificate was incredibly easy. I am genuinely surprised at how smooth it was using the certbot tool.

      What’s Next?

      I need to start writing more blog posts! I might shift the focus of the blog from being so Tech focused and open it up to capture some of my other interests, too. Such as woodworking. I really enjoy making and restoring furniture. I should write some blog posts about that.

      0 Comments
    • 20 Jan 2017

      [Updated] GitHub + Gmail — Filtering for Review Requests and Mentions

      Written by Tim Bielawa

      Update – 2017-01-27

      Just 3 days after publishing this blog post GitHub made a new blog post:

      Pull request reviews are a great way to share the weight of building software, and with review requests you can get the exact feedback you need.

      To make it easier to find the pull requests that need your attention, you can now filter by review status from your repository pull request index.

      Source: Filter pull request reviews and review requests

      I have tried this out and it’s great! Like most everything else on GitHub it’s very intuitive and simple to use. I won’t steal their thunder and describe it all here. So go check out the blog post for yourself and read up on the details (screenshots included!).

      Continue reading if you’re still interested in incorporating this kind of filtering and labeling into your Gmail account.

      The Problem

      I’ve been looking for a way to filter my GitHub Pull Request lists under the condition that a review is requested of me. The online docs didn’t show any filter options for this, so I checked out the @GitHubHelp twitter account. The answer was there on the front page — they don’t support filtering PRs by review-requested-by:me yet:

      @zaghnaboot Adding a filter for reviewers is definitely on our radar, though I don’t have a specific timeline to share. –SJ

      — GitHub Support (@GitHubHelp) January 19, 2017

      So what is one to do? I’m using Gmail so I began considering what filter options were available to me there. My objectives were to clearly label and highlight:

      •  PRs where review has been requested
      • Comments where I am @mention‘d

      Interested in knowing more? Read on after the break for all the setup details.

      (more…)

      2 Comments
    • 19 Jan 2017

      References in a sub-select

      Written by Alex Wood

      Have you ever had a sub-select where you really needed to reference a value in the outer query? I know I have! The naive way would be to run the outer query and then loop over the results running the inner query on each one. Luckily, there’s a better way. The Correlated subquery. Check it out! The example given is 

      SELECT employee_number, name
  FROM employees AS Bob
  WHERE salary > (
    SELECT AVG(salary)
      FROM employees
      WHERE department = Bob.department);

      See how the sub-select references the outer query? It’s SQL magic.

      0 Comments
    • 24 Aug 2016

      bitmath-1.3.1 released

      Written by Tim Bielawa

      bitmath is a Python module I wrote which simplifies many facets of interacting with file sizes in various units as python objects. A few weeks ago version 1.3.1 was released with a few small updates.

      Updates

      • New function: bitmath.parse_string_unsafe(), a less strict version of bitmath.parse_string()

      This new function accepts inputs using non-standard prefix units such as single-letter, or mis-capitalized units. For example, parse_string will not accept a short unit like ‘100k‘, whereas parse_string_unsafe will gladly accept it:

      • Documentation Refresh: The project documentation has been thoroughly reviewed and refreshed.

      Several broken, moved, or redirecting links have been fixed. Wording and examples are more consistent. The documentation also lands correctly when installed via package.

      Getting bitmath-1.3.1

      bitmath-1.3.1 is available through several installation channels:

      • Fedora 23 and newer repositories
      • EPEL 6 and 7 repositories
      • PyPi

      Ubuntu builds have not been prepared yet due to issues I’ve been having with Launchpad and new package versions.

      0 Comments
    • →
    Page 1 of 14
    • The Authors

    • Virtual Disk Guide

      Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.

      Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.

    • bitmath

      bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.

    • latest posts

      • Using jq to filter an array of objects from JSON September 9, 2019
      • Two Year Break — And we’re back! November 16, 2018
      • [Updated] GitHub + Gmail — Filtering for Review Requests and Mentions January 20, 2017

    • tags

      bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP

    • h4ck teh world

      tbielawatbielawa

    Creative Commons License
    Technitribe by Tim Bielawa is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.