The project I work on uses X509 certificates with custom extensions to manage content access on the Red Hat CDN. The basic idea is that Candlepin issues X509 certificates with an extension saying what content the certificate is good for. Client systems then use that certificate for TLS client authentication when connecting to the CDN. If the content they are requesting (deduced from the request URL) matches the content available to them in the certificate, then access is granted.
This system works well in practice except for one problem: every time content for a particular product changes, the content data in the X509 extension becomes obsolete. We have to revoke the obsolete certificates and issue new ones. The result is an extremely large certificate revocation list (CRL).
For our cryptography needs, Candlepin uses the venerable Legion of the Bouncy Castle Java library. This library anticipates normal CRL usage so when building a CRL object from an existing file, the entire structure is read into memory at once. This approach doesn’t scale well with the numbers of revoked certificates we are dealing with, so we needed to devise a way to stream the CRL. Moreover, the only thing we really care about for our purposes is the revoked certificate’s serial number.
Streaming the CRL means we need to dissect the ASN1 that describes the CRL one piece at a time. RFC 5280 to the rescue! Looking at the description of the ASN1 for a CRL reveals that before the sequence containing the revocation entries, there will be a
thisUpdate and optionally
nextUpdate field of either type UTCTime or GeneralizedTime. We need to descend in the ASN1 until we get to the
thisUpdate field, look for and discard the optional
nextUpdate field and then walk through the
revokedCertificates sequence reading the serial numbers.
That procedure is not exactly a walk in the park, so in the hope that someone else may find it useful, here is the solution I came up with. Keep in mind that the code does not check the signature on the CRL so this code should not be used for any CRL that you do not trust implicitly.
The end results are pretty dramatic. The benchmarking toolkit I’m using shows an improvement in execution time by an order of magnitude (from around 7 seconds to .7 seconds) and memory usage drops by about 30%. You can see the GC statistics in the graph below.
and the benchmarking results are
Benchmark Mode Cnt Score Error Units CRLBenchmark.inMemory avgt 20 7493.602 ± 941.592 ms/op CRLBenchmark.stream avgt 20 669.084 ± 91.382 ms/op
In writing this, A Layman’s Guide to a Subset of ASN.1, BER, and DER was of invaluable assistance to me as was the Wikipedia page on X.690. I recommend reading them both.
04 Jan 2016 09:01 pm
“…This approach doesn’t scale well with the numbers of revoked certificates we are dealing with, so we needed to devise a way to stream the CRL….” – Yeah, that’s a well known problem with PKI. Its the reason Google (and others) issue certificates for 30 days. Its keeps the CRL lists small for mobile clients. Early experiments showed PKIs run by large organizations had CRLs that were 30 MB to hundreds of MB. Also see Peter Gutmann’s Engineering Security, http://www.cs.auckland.ac.nz/~pgut001/pubs/book.pdf. Its a great read if you are doing anything related to security.
08 Jan 2016 04:01 pm
Thanks for the link. Interesting and terrifying. I think my favorite part so far is the P25 radio system where a “0” label meant encrypted and “O” meant unencrypted and unsurprisingly people didn’t notice when the setting was incorrect.