GNU/Linux | Technitribe

- 9 Sep 2019
  
  Using jq to filter an array of objects from JSON
  
  Written by Tim Bielawa
  For some reason it took me an unreasonable amount of time to figure out how to filter an array (or list) of objects from a JSON stream. Every single example I found was a little too weird for me, or resulted in printing each object, but not in a final array format. Here’s what I came up with:
  
  Say for example you are parsing the AWS IP ranges JSON stream, you will receive an object like this:
```
{
  "syncToken": "1567728788",
  "createDate": "2019-09-06-00-13-08",
  "prefixes": [
    {
      "ip_prefix": "18.208.0.0/13",
      "region": "us-east-1",
      "service": "AMAZON"
    },
    ... more objects here ...
```
  I was attempting to filter this down to ONLY objects where the service attribute was AMAZON. Using this jql I would get objects printed one after the other which is not what I wanted:
```
$ jq -c '.prefixes[] | select(.service=="AMAZON")' < ip-ranges.json | head
{"ip_prefix":"18.208.0.0/13","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"52.95.245.0/24","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"99.77.142.0/24","region":"ap-east-1","service":"AMAZON"}
```
  The correct syntax was ultimately very similar.
```
$ jq '.prefixes | map(. | select(.service=="AMAZON"))' < ip-ranges.json  | head
[
  {
    "ip_prefix": "18.208.0.0/13",
    "region": "us-east-1",
    "service": "AMAZON"
  },
```
  Now we are getting each object returned as a member of an array. The difference is that we’re putting the .prefixes array objects into the map function and telling it to iterate every object through the select function. The map takes all of those matching objects and returns them as an array, whereas, previously we were only selecting objects that matched our select criteria. To get the objects back in a list we required the map.
  0 Comments
- 3 Jul 2016
  
  bitmath – Now available in Ubuntu PPAs
  
  Written by Tim Bielawa
  bitmath is a Python module I wrote for working with file size units (ex: 12GiB, 64kB) as objects. You can use them just like you would use regular numbers in python. It’s full of other functionality as well. Objects have native ‘convert to $unit‘ methods, support native arithmetic, are sortable, and include a ‘best human readable prefix’ method.
  
  Since March 2014, bitmath had only been available via PyPi and Fedora/EPEL repositories. Now, as of July 2^nd 2016, bitmath is natively available to Ubuntu users by means of a new Personal Package Archive (PPA) hosting bitmath builds for Xenial, Wily, Vivid, Trusty, and Precise.
  
  Ubuntu users can install bitmath in the following way:
  
  Ubuntu support wouldn’t have happened if GitHub user hkraal hadn’t submitted an issue. Thanks Henk for getting the fire lit!
  - Check out bitmath on GitHub
  0 Comments
- 11 Mar 2016
  
  I just published my first book, The Virtual Disk Guide
  
  Written by Tim Bielawa
  I’m very excited (and proud) to announce that on March 3^rd, 2016 I reached a long-term goal I set for myself 3½ years ago, by self-publishing my first book, The Linux Sysadmin’s Guide to Virtual Disks. The book is published under my new brand, Scribe’s Guides.
  
  The first edition of The Virtual Disk Guide has been a long time coming. Nearly 7 years of on-and-off writing have gone into it. I’m relieved to have made it this far.
  
  Front cover
  
  Back cover
  
  About the Virtual Disk Guide
  
  I view the book as the definitive reference guide for virtual disk related activities — clear, concise, accurate, and approachable to readers of all skill levels— but that’s just my opinion. You can decide that for yourself.
  
  The book is quite thoroughly cited and annotated with nearly 100 individual footnotes and references to additional learning resources. The book weighs in at around 80 pages, 7 chapters, and two technical appendices. Here’s the byline from the scribesguides.com website:
  
  The Linux Sysadmin’s Guide to Virtual Disks demonstrates the core concepts of virtual disk management. Real-world problems are covered in the book’s “Cookbook” section. Other topics include: helper utilities, disk formats, troubleshooting tips, performance considerations, and comprehensive appendices.
  
  ~~Get~~ Buy one (please?)
  - Buy a hard copy of the first edition for $10.00
  - Read the latest build^[1] of the PDF or HTML versions online for free
  Or do both! Say “thanks!” by purchasing a copy, and then enjoy the latest builds online forever, for free!
  
  ^[1] – The original first edition text is also available for free in PDF and HTML formats and is identical to the print copy
  
  Open Source
  
  The official publishing of The Virtual Disk Guide does not change anything about it’s openness or your freedom to remix it however you wish. The book is still freely licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).
  
  All of the source material used to build the book’s body material and cover images are still free and open source, covered under the same license. All digital media displayed in the book, such as figures and the cover art, was created using free/open source software. Each media item was created and saved in digital formats unencumbered by patents.
  - GitHub: Book Source
  As ever, if you identify errors in the book or have thought of a way to improve it, please open a ticket on the GitHub issue tracker. If you’ve read a copy of the book already and would like to contribute a review or statement, feel free to reach out to me. Find my email in a github commit, or look at my other contact methods under the author highlight panel on scribesguides.com.
  
  More Blog Posts Coming Soon
  
  The experience of writing and publishing this book has taught me much, and it’s time to spread that information. Check back soon for a follow-up post I’m writing which covers more of the technical side of self-publishing. Specifically, self-publishing a DocBook 5 document at the on-demand printing website lulu.com.
  
  Let me be explicitly clear, this is not a promotion for lulu.com.
  
  Rather, the post will review some of the technical challenges I encountered (old examples: #1, #2, #3) during the publishing process, including challenges specific to Lulu. Such as, how I customized the PDF output from dblatex to look more personal and less generically academic, why I had to order three proof copies of the book before the cover matter printed in decent quality, and how to adjust your inner and outer page margins so there’s a reasonable amount of whitespace between the spine/binding and the body text.
  
  I have a feeling that by the time I’m done with the blog posts I’m going to have written another book of documentation about how I wrote a book of documentation
  1 Comment
- 3 Feb 2016
  
  bitmath-1.3.0 released
  
  Written by Tim Bielawa
  It’s been quite a while since I’ve posted any bitmath updates (bitmath is a Python module I wrote which simplifies many facets of interacting with file sizes in various units as python objects) . In fact, it seems that the last time I wrote about bitmath here was back in 2014 when 1.0.8 was released! So here is an update covering everything post 1.0.8 up to 1.3.0.
  
  New Features
  - A command line tool, bitmath, you can use to do simple conversions right in your shell [docs]!
  - New utility function bitmath.parse_string for parsing a human-readable string into a bitmath object
  - New utility: argparse integration: bitmath.BitmathType. Allows you to specify arguments as bitmath types
  - New utility: progressbar integration: bitmath.integrations.BitmathFileTransferSpeed. A more functional file transfer speed widget
  - New bitmath module function: bitmath.query_device_capacity(). Create bitmath.Byte instances representing the capacity of a block device
    
    This my favorite enhancement
    
    In an upcoming blog post I’ll talk about just how cool I thought it was learning how to code this feature
    
    Conceptual and practical implementation topics included
  - The bitmath.parse_string() function now can parse ‘octet’ based units
    
    Enhancement requested in #53 parse french unit names by walidsa3d.
  - New utility function: bitmath.best_prefix()
    
    Return an equivalent instance which uses the best human-readable prefix-unit to represent it
    
    This is way cooler than it may sound at the surface, I promise you
  Bug Fixes
  - #49 – Fix handling unicode input in the bitmath.parse_string function. Thanks drewbrew!
  - #50 – Update the setup.py script to be python3.x compat. Thanks ssut!
  - #55 “best_prefix for negative values”. Now bitmath.best_prefix() returns correct prefix units for negative values. Thanks mbdm!
  Misc
  
  To help with the Fedora Python3 Porting project, bitmath now comes in two variants in Fedora/EPEL repositories (BZ1282560). The Fedora and EPEL updates are now in the repos. TIP: python2-bitmath will obsolete the python-bitmath package. Do a dnf/yum ‘update‘ operation just to make sure you catch it.
  
  The PyPi release has already been pushed to stable.
  
  Back in bitmath-1.0.8 we had 150 unit tests. The latest release has almost 200! Go testing! :confetti:
  1 Comment
- 18 Sep 2015
  
  Eclipse Darkness
  
  Written by Alex Wood
  I’ve made several posts previously about the difficulties I’ve had with Eclipse and Gnome’s Adwaita theme: menu elements that have too little contrast to read, poor color choices, etc. I even took a stab at creating my own GTK3 theme to deal with the problem.
  
  I’m happy to report that my efforts are now obsolete. Eclipse Mars (now available in Fedora 22) has made significant improvements to the Dark theme (set under Preferences -> General -> Appearance). However, if you’re using Adwaita, the top menu bar is gray text on grey background. The simple fix is to change to the Adwaita Dark theme just for Eclipse. Here’s how:
  1. Open /usr/share/applications/eclipse.desktop in your text editor of choice.
  2. Modify the Exec line to read
    Exec=env GTK_THEME=Adwaita:dark eclipse
  3. Done!
  The one gotcha is that when you update the eclipse-platform package, it will destroy the changes you’ve made in the desktop file so you’ll have to redo them. But that’s a small price to pay in my opinion.
  
  Eclipse Mars with the Adwaita Dark GTK theme.
  0 Comments
- →
Page 1 of 7
- The Authors
- Virtual Disk Guide
  
  Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.
  
  Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.
- bitmath
  
  bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.
- latest posts
  - Using jq to filter an array of objects from JSON September 9, 2019
  - Two Year Break — And we’re back! November 16, 2018
  - [Updated] GitHub + Gmail — Filtering for Review Requests and Mentions January 20, 2017
- tags
  bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP
- h4ck teh world
  tbielawa
  - Push
    openshift/elliott
    September 17, 2020 - 4:58 pm UTC
  - Push
    openshift/doozer
    September 17, 2020 - 4:58 pm UTC
  - Delete
    aos-cd-jobs
    September 17, 2020 - 4:42 pm UTC
  - Push
    openshift/aos-cd-jobs
    September 17, 2020 - 4:41 pm UTC
  - Pull Request
    openshift/aos-cd-jobs
    September 17, 2020 - 4:41 pm UTC