Technitribe

interesting problems (and a few solutions, too)

Technitribe
  • About the Authors
  • Log In
  • Log Out
  • Lost Password
  • Register
  • Reset Password
    • 4 Feb 2023

      Querying block device sizes in Python on Linux and Mac OS X

      Written by Tim Bielawa

      I drafted this blog post in 2016 (at least), but held off publishing it until I could have it fact checked. Well, 6 years have passed… I am 99% sure the information in this blog post is correct. But if you find an error with my explanation of the userspace-kernel-device dataflow then please send me an email so I can understand it better and update this post. Thank you!

      The Problem

      I’ve been experimenting with creating functionality within bitmath for reading the size of storage devices. This would provide a function similar to Python’s os.path.getsize, but for storage device capacity instead of file sizes.

      Unfortunately, it turns out that there is no out of the box (and cross-platform) solution in Python for reading the capacity of system storage devices. This meant some research was going to be required. Luckily, possible solutions for how to do this are abundant across the internet. Well, for Linux anyway. Figuring out how to make this work on Mac OS X was more challenging.

      And that’s where the story gets interesting.

      In the rest of this blog post we’ll learn the basics of how programs can interact with storage devices via the ioctl() system call. Then we’ll discuss the things we have to do and information we’ll need to have in order to implement an ioctl() request in Python. Next we’ll see how to gather all the necessary information (request codes and expected result sizes). Finally we’ll put all of this together into a runnable Python program.

      If you’re not familiar with that acronym, “ioctl” stands for “input/output control”.

      (more…)

      0 Comments
    • 24 Sep 2013

      Mac OS X, Sed, and strange document encoding

      Written by Tim Bielawa

      The Problem

      You’re on Mac OS X (somewhere around 10.7.5) and you’re using the sed command to replace characters from the latin1 or Windows-1252 character encoding with their utf8 equivalents. Unfortunately you get an error like the following:

      sed: 1: "s/#/’/g
      ": RE error: illegal byte sequence

      Luckily you’re not alone!

      • vim_dev
      • homebre-deps
      • HamDecks
      • stackoverflow

      This happened to me while working on HamDecks, a small project that creates Mnemosyne decks to help you study for the Amateur Radio Operator exams using questions from the official ARRL Question pools. The source question pool files (Technician, General, Extra) though have some problems… There’s a lot of characters with strange/exotic encoding in the ARRL pool files that could not be imported into Mnemosyne. That’s how I got myself into this whole mess in the first place.

      Options

      The stackoverflow link above makes two suggestions:

      1. Use the iconv utility
      2. Use a PERL one-liner

      Your Mileage May Vary, but neither of those suggestions worked for me. So what did work then?

      Potential Solution

      Once again, we will visit our system locale settings.

      Here’s what worked for the HamDecks project:

      Instead of just prefixing the sed command with LANG=C, we prefix it with LANG=C LANG_ALL=C. I’m not saying this is a silver bullet, just that it worked for me and might work for you too.

      0 Comments
    • 15 Oct 2011

      Mac OS X 10.7 – Getting songs off an iPod

      Written by Tim Bielawa

      A while back I wanted to backup the songs I’ve saved to my iPod. I was running Mac OS X Leopard (10.5). When I searched around I found a tool called iPodDisk. It worked great! Then Snow Leopard (10.6) came out.

      The way the launcher for iPodDisk was written meant it only would launch on 10.4 or 10.5. When you launched iPodDisk you would receive the message:

      “Sorry iPodDisk requires OS X 10.4 or later”

      Fixing this was pretty simple. After searching their google code issue tracker I quickly came across issue #34, “Sorry iPodDisk requires OS X 10.4 or later“. The fix was simple, reply #3 in the thread said to just edit one line in the launcher so that the version check accepted a higher version.

      Time goes on and eventually I did a full OS reinstall (for some reason or another) and just recently upgraded to 10.7 (Lion). I’d bought a new network storage unit (Western Digital My Book Live) and wanted to back up my iPod again. Of course, upon downloading and launching iPodDisk again it produced the same error it did previously. It was clear to me that iPodDisk was clearly not being maintained.

      While searching for the fix again I decided to read some newer posts on the issue tracker. Reply #51 referenced MacFuse. The project site for that hasn’t made a new release since 2008 though. Reply #60 had the information I was finally looking for.

      So the final solution that allowed me to backup my iPod on OS X 10.7, Lion, was to install these two pieces of software:

      • iTunesFS
      • OSX Fuse
      The result was perfect. The same functionality I had when using iPodDisk previously. When you launch iTunesFS it opens a new Finder window showing all mounted volumes. Navigate to the iTunesFS volume and you’re set.
      5 Comments
    • The Authors
    • Virtual Disk Guide

      Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.


      Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.

    • bitmath

      bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.

    • latest posts

      • Querying block device sizes in Python on Linux and Mac OS X February 4, 2023
      • Using jq to filter an array of objects from JSON September 9, 2019
      • Two Year Break — And we’re back! November 16, 2018
    • tags

      bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP
    • h4ck teh world

      tbielawatbielawa
      • Watch
        cognitive-engineering-lab/aquascope
        July 17, 2023 - 9:47 pm UTC

Creative Commons License
Technitribe by Tim Bielawa is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.