Technitribe

- 4 Feb 2023
  
  Querying block device sizes in Python on Linux and Mac OS X
  
  Written by Tim Bielawa
  
  I drafted this blog post in 2016 (at least), but held off publishing it until I could have it fact checked. Well, 6 years have passed… I am 99% sure the information in this blog post is correct. But if you find an error with my explanation of the userspace-kernel-device dataflow then please send me an email so I can understand it better and update this post. Thank you!
  
  The Problem
  
  I’ve been experimenting with creating functionality within bitmath for reading the size of storage devices. This would provide a function similar to Python’s os.path.getsize, but for storage device capacity instead of file sizes.
  
  Unfortunately, it turns out that there is no out of the box (and cross-platform) solution in Python for reading the capacity of system storage devices. This meant some research was going to be required. Luckily, possible solutions for how to do this are abundant across the internet. Well, for Linux anyway. Figuring out how to make this work on Mac OS X was more challenging.
  
  And that’s where the story gets interesting.
  
  In the rest of this blog post we’ll learn the basics of how programs can interact with storage devices via the ioctl() system call. Then we’ll discuss the things we have to do and information we’ll need to have in order to implement an ioctl() request in Python. Next we’ll see how to gather all the necessary information (request codes and expected result sizes). Finally we’ll put all of this together into a runnable Python program.
  
  If you’re not familiar with that acronym, “ioctl” stands for “input/output control”.
  
  (more…)
  
  0 Comments
- 9 Sep 2019
  
  Using jq to filter an array of objects from JSON
  
  Written by Tim Bielawa
  For some reason it took me an unreasonable amount of time to figure out how to filter an array (or list) of objects from a JSON stream. Every single example I found was a little too weird for me, or resulted in printing each object, but not in a final array format. Here’s what I came up with:
  
  Say for example you are parsing the AWS IP ranges JSON stream, you will receive an object like this:
```
{
  "syncToken": "1567728788",
  "createDate": "2019-09-06-00-13-08",
  "prefixes": [
    {
      "ip_prefix": "18.208.0.0/13",
      "region": "us-east-1",
      "service": "AMAZON"
    },
    ... more objects here ...
```
  I was attempting to filter this down to ONLY objects where the service attribute was AMAZON. Using this jql I would get objects printed one after the other which is not what I wanted:
```
$ jq -c '.prefixes[] | select(.service=="AMAZON")' < ip-ranges.json | head
{"ip_prefix":"18.208.0.0/13","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"52.95.245.0/24","region":"us-east-1","service":"AMAZON"}
{"ip_prefix":"99.77.142.0/24","region":"ap-east-1","service":"AMAZON"}
```
  The correct syntax was ultimately very similar.
```
$ jq '.prefixes | map(. | select(.service=="AMAZON"))' < ip-ranges.json  | head
[
  {
    "ip_prefix": "18.208.0.0/13",
    "region": "us-east-1",
    "service": "AMAZON"
  },
```
  Now we are getting each object returned as a member of an array. The difference is that we’re putting the .prefixes array objects into the map function and telling it to iterate every object through the select function. The map takes all of those matching objects and returns them as an array, whereas, previously we were only selecting objects that matched our select criteria. To get the objects back in a list we required the map.
  1 Comment
- 16 Nov 2018
  
  Two Year Break — And we’re back!
  
  Written by Tim Bielawa
  
  Where has Technitribe been for the last 2 years? The short answer: in domain limbo.
  
  Christmas Islands
  
  .cx is a Christmas Islands top-level domain (TLD). I did not have the domain set to auto-renew because I was trying to get away from my old registrar. At the time it was expiring I didn’t have the cash on hand to renew it. I will admit, I did not read all the Term & Conditions for the .cx TLD before registering. I guess I really screwed the pooch there.
  
  It turns out that if you have a .cxdomain, and it expires, then you have a problem. They will effectively hold it hostage for a period of 4 months. During this time you have the “option” to restore the domain, but at the cost of $400.00 USD. Like I said, I was not very cash flush at the time.
  
  Waiting Period
  
  Shortly after the domain expired I moved across the country. While getting set up in my new location the domain left the restoration period and was open to general registration again. As is typical, a reseller had squatted the lnx.cx domain and bought it as soon as it became available. I resigned myself to my fate, I would have to let it go for a while.
  
  Return of lnx.cx
  
  The blog is now back! A few weeks ago I finally had the cash necessary to register it from a new registrar who isn’t evil (Netim). We have a new SSL Certificate issued by Let’s Encrypt, no more untrusted RapidSSL security alerts in your browser. And, as an aside, the setup for installing this new certificate was incredibly easy. I am genuinely surprised at how smooth it was using the certbot tool.
  
  What’s Next?
  
  I need to start writing more blog posts! I might shift the focus of the blog from being so Tech focused and open it up to capture some of my other interests, too. Such as woodworking. I really enjoy making and restoring furniture. I should write some blog posts about that.
  
  0 Comments
- 20 Jan 2017
  
  [Updated] GitHub + Gmail — Filtering for Review Requests and Mentions
  
  Written by Tim Bielawa
  Update – 2017-01-27
  
  Just 3 days after publishing this blog post GitHub made a new blog post:
  
  Pull request reviews are a great way to share the weight of building software, and with review requests you can get the exact feedback you need.
  
  To make it easier to find the pull requests that need your attention, you can now filter by review status from your repository pull request index.
  
  Source: Filter pull request reviews and review requests
  
  I have tried this out and it’s great! Like most everything else on GitHub it’s very intuitive and simple to use. I won’t steal their thunder and describe it all here. So go check out the blog post for yourself and read up on the details (screenshots included!).
  
  Continue reading if you’re still interested in incorporating this kind of filtering and labeling into your Gmail account.
  
  The Problem
  
  I’ve been looking for a way to filter my GitHub Pull Request lists under the condition that a review is requested of me. The online docs didn’t show any filter options for this, so I checked out the @GitHubHelp twitter account. The answer was there on the front page — they don’t support filtering PRs by review-requested-by:me yet:
  
  @zaghnaboot Adding a filter for reviewers is definitely on our radar, though I don’t have a specific timeline to share. –SJ
  
  — GitHub Support (@GitHubHelp) January 19, 2017
  
  So what is one to do? I’m using Gmail so I began considering what filter options were available to me there. My objectives were to clearly label and highlight:
  - PRs where review has been requested
  - Comments where I am @mention‘d
  Interested in knowing more? Read on after the break for all the setup details.
  
  (more…)
  2 Comments
- 19 Jan 2017
  
  References in a sub-select
  
  Written by Alex Wood
  Have you ever had a sub-select where you really needed to reference a value in the outer query? I know I have! The naive way would be to run the outer query and then loop over the results running the inner query on each one. Luckily, there’s a better way. The Correlated subquery. Check it out! The example given is
  
  SELECT employee_number, name FROM employees AS Bob WHERE salary > ( SELECT AVG(salary) FROM employees WHERE department = Bob.department);
  
  See how the sub-select references the outer query? It’s SQL magic.
  0 Comments
- →
Page 1 of 14
- The Authors
- Virtual Disk Guide
  
  Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.
  
  Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.
- bitmath
  
  bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.
- latest posts
  - Querying block device sizes in Python on Linux and Mac OS X February 4, 2023
  - Using jq to filter an array of objects from JSON September 9, 2019
  - Two Year Break — And we’re back! November 16, 2018
- tags
  bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP
- h4ck teh world
  tbielawa
  - Watch
    spotDL/spotify-downloader
    March 27, 2023 - 2:52 pm UTC
  - Watch
    exaloop/codon
    March 19, 2023 - 3:05 pm UTC
  - Issue Comment
    hashicorp/vault
    March 2, 2023 - 2:40 pm UTC
  - Issue Comment
    bitmath
    February 15, 2023 - 10:26 pm UTC
  - Issue Comment
    bitmath
    February 15, 2023 - 10:25 pm UTC

Technitribe

4 Feb 2023

Querying block device sizes in Python on Linux and Mac OS X

Written by Tim Bielawa

The Problem

9 Sep 2019

Using jq to filter an array of objects from JSON

Written by Tim Bielawa

16 Nov 2018

Two Year Break — And we’re back!

Written by Tim Bielawa

Christmas Islands

Waiting Period

Return of lnx.cx

What’s Next?

20 Jan 2017

[Updated] GitHub + Gmail — Filtering for Review Requests and Mentions

Written by Tim Bielawa

Update – 2017-01-27

The Problem

19 Jan 2017

References in a sub-select

Written by Alex Wood

Virtual Disk Guide

bitmath

latest posts

tags

h4ck teh world