One of the many pieces of witchcraft that Namecoin’s TLS interoperability requires is the ability to splice a signature into an X.509 certificate without otherwise modifying the certificate. Unfortunately, while the Go standard crypto library is generally quite pleasant to use (and is certainly better-designed than most other crypto libraries such as OpenSSL), the functions relevant to splicing a signature are not exported. This is understandable, since it’s not functionality that most users have any need for. However, since Namecoin does need that functionality, my only option back when this code was being written was to fork the crypto/x509 package from Go’s standard library.

I didn’t want to be responsible for maintaining or distributing a fork of Go’s library, though. So, the best option I could come up with was to use go generate, which is a fun feature in Go’s build system. In Namecoin’s case, the ncdns build process uses go generate to copy the source code from the official crypto/x509 package into the github.com/namecoin/ncdns/x509 package, which (when combined with the single existing source code file, x509_splice.go), produces a package that’s identical to upstream crypto/x509 but with an extra exported function (CreateCertificateWithSplicedSignature) that calls out to the unexported upstream functions in order to do what we want. The benefit here is that whenever those unexported upstream functions are updated in a new Go release, Namecoin’s forked package will automatically inherit those changes without the Namecoin developers needing to do anything. In fact, in theory the same forked package will work unmodified with arbitrary versions of the Go standard library, without us needing to do anything special.

Alas, theory does not always equal practice. The most obvious scenario in which this theory will fall apart is if the unexported functions called by CreateCertificateWithSplicedSignature change their API. Upstream is well within their rights to do this, of course: they’re not exported functions, so there’s no reason to expect a stable API. However, in practice, this API changes quite rarely. The more common issue I’ve been encountering, surprisingly, is that the go generate script has trouble finding and copying the upstream library, because upstream seems to keep changing the details of where this library and its dependencies (some of which are internal-only libraries) can be found. The result is that I’ve had to rebase our fork against upstream every couple of major Go releases to keep things working well.

The problem here is that this has the effect of coupling a specific commit hash of the ncdns repository to a specific range of Go compiler versions. If Namecoin rebases against a new Go standard library version, and then issues an ncdns bug fix that’s unrelated to our x509 fork, everyone who’s downstream of us needs to update their Go compiler in order to get the bug fix. This is generally problematic since lots of downstream distributors have other considerations for when they update their compiler. It was specifically causing me problems for getting ncdns to build in Tor’s rbm-based build system, because Tor is not likely to update their Go compiler exactly when Namecoin does.

So, what to do? Well, note that there’s no fundamentally important reason for the x509 package to be a subpackage of ncdns. It ended up there by default because it was initially only used by ncdns and there wasn’t an obvious reason to create a new Git repo, but now we have a good reason to move it to its own repo: if the Git commit hash of ncdns and x509 are independent, then downstream distributors can pick the version of ncdns with whatever bug fixes they want, and independently choose the version of x509 that supports whatever Go compiler version they want. I’ve now done this. x509 now lives at github.com/namecoin/x509-signature-splice/x509 , and branches are available for every version of the Go compiler that we’ve ever supported, ranging from Go 1.5.x all the way through Go 1.13.x. (Note that the Go 1.12.x and Go 1.13.x support is new, as both of those Go releases required rebases in order for go generate to run without errors. So if you use one of those Go versions, you’ll find this work especially useful.)

The two final steps here are:

  1. Removing x509 from the ncdns repo and switching over to the new repo; this has now been merged.
  2. Updating ncdns-repro to use the new ncdns version; this will happen soon.

This work was funded by NLnet Foundation’s Internet Hardening Fund and Cyphrs.