Technitribe

interesting problems (and a few solutions, too)

Technitribe
  • About the Authors
  • Log In
  • Log Out
  • Lost Password
  • Register
  • Reset Password
    • 30 Oct 2015

      Streaming the serial numbers from an X509 certificate revocation list

      Written by Alex Wood

      The project I work on uses X509 certificates with custom extensions to manage content access on the Red Hat CDN. The basic idea is that Candlepin issues X509 certificates with an extension saying what content the certificate is good for. Client systems then use that certificate for TLS client authentication when connecting to the CDN. If the content they are requesting (deduced from the request URL) matches the content available to them in the certificate, then access is granted.

      This system works well in practice except for one problem: every time content for a particular product changes, the content data in the X509 extension becomes obsolete. We have to revoke the obsolete certificates and issue new ones. The result is an extremely large certificate revocation list (CRL).

      For our cryptography needs, Candlepin uses the venerable Legion of the Bouncy Castle Java library. This library anticipates normal CRL usage so when building a CRL object from an existing file, the entire structure is read into memory at once. This approach doesn’t scale well with the numbers of revoked certificates we are dealing with, so we needed to devise a way to stream the CRL. Moreover, the only thing we really care about for our purposes is the revoked certificate’s serial number.

      Streaming the CRL means we need to dissect the ASN1 that describes the CRL one piece at a time. RFC 5280 to the rescue! Looking at the description of the ASN1 for a CRL reveals that before the sequence containing the revocation entries, there will be a thisUpdate and optionally nextUpdate field of either type UTCTime or GeneralizedTime. We need to descend in the ASN1 until we get to the thisUpdate field, look for and discard the optional nextUpdate field and then walk through the revokedCertificates sequence reading the serial numbers.

      That procedure is not exactly a walk in the park, so in the hope that someone else may find it useful, here is the solution I came up with. Keep in mind that the code does not check the signature on the CRL so this code should not be used for any CRL that you do not trust implicitly.

      The end results are pretty dramatic. The benchmarking toolkit I’m using shows an improvement in execution time by an order of magnitude (from around 7 seconds to .7 seconds) and memory usage drops by about 30%. You can see the GC statistics in the graph below.
      Visualization of X509CRLStream's benchmarks

      and the benchmarking results are

      Benchmark             Mode Cnt    Score     Error  Units
      CRLBenchmark.inMemory avgt  20  7493.602 ± 941.592  ms/op
      CRLBenchmark.stream   avgt  20   669.084 ±  91.382  ms/op
      

      In writing this, A Layman’s Guide to a Subset of ASN.1, BER, and DER was of invaluable assistance to me as was the Wikipedia page on X.690. I recommend reading them both.

      • Tags »
      • asn1 java pki x509

    2 Comment on “Streaming the serial numbers from an X509 certificate revocation list”

    • Jeffrey Walton

      04 Jan 2016 09:01 pm

      “…This approach doesn’t scale well with the numbers of revoked certificates we are dealing with, so we needed to devise a way to stream the CRL….” – Yeah, that’s a well known problem with PKI. Its the reason Google (and others) issue certificates for 30 days. Its keeps the CRL lists small for mobile clients. Early experiments showed PKIs run by large organizations had CRLs that were 30 MB to hundreds of MB. Also see Peter Gutmann’s Engineering Security, http://www.cs.auckland.ac.nz/~pgut001/pubs/book.pdf. Its a great read if you are doing anything related to security.

    • Alex Wood

      08 Jan 2016 04:01 pm

      Jeffrey,

      Thanks for the link. Interesting and terrifying. I think my favorite part so far is the P25 radio system where a “0” label meant encrypted and “O” meant unencrypted and unsurprisingly people didn’t notice when the setting was incorrect.

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    • The Authors
    • Virtual Disk Guide

      Interested in virtualization? Do QCOWs rule your filesystem? Are you a libvirt or KVM+QEMU wizard? I wrote a book about virtual disk management. Check out the The Linux Sysadmin's Guide to Virtual Disks online for free at ScribesGuides.com.


      Consider supporting the author by purchasing a hard copy of the first edition for just $10.00 on Lulu.com.

    • bitmath

      bitmath is a Python library for dealing with file size units (GiB's, kB's, etc) in a sane way. bitmath supports arithmetic, rich comparison, conversion, automatic best human-readable representation, and many other utility functions. Read some examples on the docs site or check out the source on GitHub.

    • latest posts

      • Querying block device sizes in Python on Linux and Mac OS X February 4, 2023
      • Using jq to filter an array of objects from JSON September 9, 2019
      • Two Year Break — And we’re back! November 16, 2018
    • tags

      bitmath blog conference css dblatex DNS DocBook eclipse Emacs Erlang Fedora fedora 22 filter GNU Screen Haiku Introduction java jboss LCSEE Linux locale locales fix slicehost ubuntu Macports module nist nXML-Mode opengl open source OS X package packaging pki prefix units presentation project pypi Python scholarship si summit Tutorial ubuntu xcode XML XMPP
    • h4ck teh world

      tbielawatbielawa
      • Issue Comment
        bitmath
        February 6, 2023 - 12:55 am UTC
      • Issue Comment
        bitmath
        February 6, 2023 - 12:54 am UTC
      • Push
        bitmath
        February 6, 2023 - 12:51 am UTC
      • Issue Comment
        bitmath
        February 6, 2023 - 12:36 am UTC
      • Push
        bitmath
        February 6, 2023 - 12:30 am UTC

Creative Commons License
Technitribe by Tim Bielawa is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.