News

Reverse-Engineering CryptoAPI’s Certificate Registry Blobs

Every so often, I’m doing Namecoin-related development research (in this case, making TLS work properly) and I run across some really interesting information that no one else seems to have documented. While this post isn’t solely Namecoin-related (it’s probably useful to anyone curious about tinkering with TLS), I hope you find it interesting regardless.

A note on the focus here: while this research was done for the purpose of engineering specific things, I’m writing it from more of a “basic research” point of view. My dad’s career was in basic research, and I firmly believe that learning cool stuff for the sake of learning it is a worthwhile endeavor, regardless of what the practical applications are (and indeed, usually when basic research turns out to have applications, which is commonplace, the initial researchers didn’t know what those applications would be). Since I’m an engineer, there will be a bit of application-related commentary here, but don’t read this expecting it to be a summary of the next Namecoin software release’s feature set or use cases.

In Windows-based OS’s, most applications handle certificates via the CryptoAPI. CryptoAPI serves a somewhat similar role in Windows certificate verification as OpenSSL does on GNU/Linux-based systems. Notably, Mozilla-based applications like Firefox and Thunderbird don’t use CryptoAPI (nor OpenSSL); they use the Mozilla library NSS (on both Windows and GNU/Linux). However, except for Mozilla applications, and a few applications ported from GNU/Linux (e.g. Python) which use OpenSSL, just about everything on Windows uses CryptoAPI for its certificate needs. CryptoAPI is a quite old Microsoft technology; it dates back at least to Windows NT 4. (It might be even older, but I’ve never touched nor read about any of the earlier incarnations of Windows NT, so I wouldn’t know.) Like any other codebase that’s been around for over 2 decades, its design is somewhat convoluted, and my guess is that if it were being designed from scratch today, it would look very different.

CryptoAPI maintains a bunch of different stores for certificates. These stores are designated according to the certificate’s intended usage (e.g. a website cert, an intermediate CA, a root CA, a personal cert, and a bunch of other use cases that I don’t understand because I’ve never managed any kind of certificate infrastructure for an enterprise), the method by which the certificate was loaded (e.g. by a web browser cache, by group policy, and a bunch of other methods that, again, I don’t understand because I don’t do enterprise infrastructure), which users have permission to use the certs (roaming profiles have special handling), and even which applications are expected to consider them valid (Cortana, Edge, and Windows Store all have their own certificate stores, for reasons that I don’t understand in the slightest, although I do wonder whether adding an intercepting proxy to Cortana’s cert store would be useful in an attempt to wiretap Cortana and see what data Microsoft collects on its users). You can see a subset of the certificate stores’ contents via certmgr.msc, and there’s a command-line tool included with Windows called certutil which can edit or dump this data as well. Neither of these tools actually shows all of the stores, e.g. Cortana, Edge, and Windows Store are secret and invisible. Also, don’t confuse the CryptoAPI certutil with the Mozilla command-line tool also called certutil, which is similar but is for NSS stores and has an entirely different syntax.

Incidentally, CryptoAPI has some interesting behavior when it comes to root CA’s. If you add a self-signed website cert to a root CA store, that self-signed website cert becomes fully trusted (HSTS and HPKP even work, which implies that it doesn’t get reported as an override). Of course, this is usually a dangerous idea, since that self-signed website could then sign other websites’ certs – you did tell Windows to treat it as a root CA, after all. But Windows actually does respect the CA and CertUsage flags in this case: if you construct a cert that is not valid as a CA, Windows will happily let you add it to a root CA store, will accept it as a website cert, but will refuse to trust any other cert signed by that cert. Namecoin lead security engineer Ryan Castellucci told me on IRC that he’s not sure if this behavior is even defined in a spec, but in my testing, NSS seems to exhibit identical behavior (no idea about OpenSSL). Regardless of specs, Microsoft has a fanatical obsession with not changing behavior of any public-facing API that might impact backwards compatibility (to Microsoft, the original implementation is the spec), so I think it’s probably pretty safe to rely on this behavior, even when someone as thoroughly knowledgeable as Ryan has never encountered anything in the wild that does this. Of course, that’s just my assessment – I take no responsibility if this burns you. As they say on Brainiac: Science Abuse, “we do these experiments so you don’t have to – do not try this at home – no really, don’t.”

Now, unfortunately, CryptoAPI has a problem. It expects a user to have administrator privileges in order to add a cert to most of the stores. This is probably well-meaning, because you definitely don’t want some random piece of malware that abused a Javascript zero-day to be able to add a root CA, or anything like that. (Fun fact: any such malware can, however, add a root CA to Firefox, because the NSS cert stores are simply a file in your profile directory, and are therefore user-writeable. That’s even true for Firefox on many GNU/Linux systems, even though the OpenSSL store is protected.) Of course, the security benefits of requiring privileged access for this are dubious, given that malware running as the primary user can do all sorts of other mischief, such as replacing the shortcut to your browser with a patched version that MITM’s you. However, regardless of the alleged security benefit of this policy, there’s a fairly obvious problem here: this implies that if you want to run software that programatically adds root CA’s, perhaps for the use case in the previous paragraph, you need to give that software Administrator privileges. As a (minimally) sane person, running anything, much less a daemon that interacts with a bunch of untrusted network hosts (e.g. Namecoin peers), as an administrator is an absolute dealbreaker. Yes, I did code it that way as a proof of concept for the hackathon by the College Cryptocurrency Network that I got 3rd place in, but no way in hell am I going to ship software to end users that does such irresponsible things. And if you’re the kind of person who would be tempted to do that, please, for the sake of your users, exit the software development field before you get some dissident or whistleblower murdered. This stuff actually is important to those of us with ethics.

You might wonder: why the heck isn’t there a permission system for this? Coming from a culture that loves the concept (if not implementation) of things like AppArmor and SELinux, that was certainly my thought. But alas, I was unable to find any Microsoft documentation that suggested a way to delegate access to a specific cert store to some other user. (Of course, Microsoft’s documentation is a train wreck, so maybe they did address this use case and I just couldn’t find any mention of it.) However, I did learn something interesting by Googling. While OpenSSL cert stores are just a filesystem folder, and NSS cert stores are a database file (whose database backend is either BerkeleyDB or SQLite), CryptoAPI mostly uses… the Windows Registry. Remember, this is Microsoft, they dump their garbage in the registry with as little hesitation as petroleum companies dump their garbage in Latin-American rainforests. (Personal certificates that are part of roaming profiles are instead placed in a user’s profile folder, apparently ever since Windows 2000 came out. But almost everything else is in the registry.) Since the registry does have a permission system, this looks like the perfect solution.

It was relatively easy to figure out where these certificates are located in the dense, uncharted jungle that is the registry. Indeed, you can search your registry for keys titled Root and you’ll find all the root CA stores (the other types of stores are in sibling keys). Each certificate is located in its own subkey (the subkey is named based on the certificate’s SHA-1 fingerprint). Actually, let me digress for a moment. Why the hell is Microsoft using SHA-1 hashes as the names of registry keys, even in Windows 10? Yes, I know SHA-1 was not known to be weak when Microsoft designed CryptoAPI, but tying the name of something to a specific hash algorithm seems like a massively stupid idea in terms of design and safety. (And no, it’s not a good idea to drive drunk just because your crazy git uncle Linus does it every New Year’s Eve and hasn’t died yet.) Anyway, inside that subkey is a single value, called Blob, which contains binary data encoding the certificate. Not too complicated, right?

Oh, wait. We’re talking about Microsoft. Everything is complicated, usually for no discernable reason whatsoever. Also, the most complicated things usually have the least documentation. I know people who have long-ago adopted a policy of getting their Windows documentation from the ReactOS source code instead of the Microsoft website, because a small, minimally funded project that’s reverse-engineering everything writes more accurate documentation than the wildly successful company who actually engineered the system and wrote the original source code. Anyway, I looked at the contents of the binary blob in the registry, and noticed that it didn’t look right. More specifically, it wasn’t a DER-encoded x.509 structure, nor was it even PEM-encoded. Actually, there was a substring that did correspond to the DER-encoded x.509 structure, but there was a crapload of extra data too. For reference, it looked like this (in .reg format):

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates\trust\Certificates\BBC2FAE0B710372FC293E092904F7E628D3D4546]
"Blob"=hex:04,00,00,00,01,00,00,00,10,00,00,00,97,30,35,47,95,5b,3b,e4,05,71,\
  d4,5c,6c,cd,7e,21,0f,00,00,00,01,00,00,00,20,00,00,00,95,6e,94,6c,93,46,fd,\
  c8,b5,02,66,9b,c9,1b,be,5c,19,df,97,f0,b4,8d,fa,f2,57,28,77,a1,7a,37,bc,bc,\
  14,00,00,00,01,00,00,00,14,00,00,00,33,c6,3b,84,aa,7a,15,b1,23,a5,4c,7e,38,\
  23,25,bc,e8,7f,cb,eb,19,00,00,00,01,00,00,00,10,00,00,00,73,1a,dd,da,db,51,\
  b3,34,87,0f,15,1e,03,c0,b0,11,5c,00,00,00,01,00,00,00,04,00,00,00,00,01,00,\
  00,03,00,00,00,01,00,00,00,14,00,00,00,bb,c2,fa,e0,b7,10,37,2f,c2,93,e0,92,\
  90,4f,7e,62,8d,3d,45,46,20,00,00,00,01,00,00,00,d3,01,00,00,30,82,01,cf,30,\
  82,01,76,a0,03,02,01,02,02,14,00,f5,9d,9e,8e,09,5d,3f,54,a9,02,2a,89,09,62,\
  41,df,f1,fa,e1,30,0a,06,08,2a,86,48,ce,3d,04,03,02,30,3e,31,19,30,17,06,03,\
  55,04,03,13,10,74,65,73,74,2e,76,65,63,6c,61,62,73,2e,62,69,74,31,21,30,1f,\
  06,03,55,04,05,13,18,4e,61,6d,65,63,6f,69,6e,20,54,4c,53,20,43,65,72,74,69,\
  66,69,63,61,74,65,30,1e,17,0d,31,37,30,31,30,31,30,30,30,30,30,30,5a,17,0d,\
  31,38,30,31,30,31,30,30,30,30,30,30,5a,30,3e,31,19,30,17,06,03,55,04,03,13,\
  10,74,65,73,74,2e,76,65,63,6c,61,62,73,2e,62,69,74,31,21,30,1f,06,03,55,04,\
  05,13,18,4e,61,6d,65,63,6f,69,6e,20,54,4c,53,20,43,65,72,74,69,66,69,63,61,\
  74,65,30,59,30,13,06,07,2a,86,48,ce,3d,02,01,06,08,2a,86,48,ce,3d,03,01,07,\
  03,42,00,04,fe,1c,b5,b7,88,c1,d7,8a,e8,9f,1e,a7,d6,6f,15,42,1d,36,8d,b9,51,\
  7e,ed,9c,57,7f,cb,73,2a,26,7d,59,63,ca,95,10,30,68,e6,bb,15,8d,6c,f2,34,6b,\
  77,05,ea,68,8d,3a,28,d2,0a,eb,6a,4d,97,5b,ed,32,ef,8a,a3,52,30,50,30,0e,06,\
  03,55,1d,0f,01,01,ff,04,04,03,02,07,80,30,13,06,03,55,1d,25,04,0c,30,0a,06,\
  08,2b,06,01,05,05,07,03,01,30,0c,06,03,55,1d,13,01,01,ff,04,02,30,00,30,1b,\
  06,03,55,1d,11,04,14,30,12,82,10,74,65,73,74,2e,76,65,63,6c,61,62,73,2e,62,\
  69,74,30,0a,06,08,2a,86,48,ce,3d,04,03,02,03,47,00,30,44,02,20,65,d7,93,a8,\
  18,c7,de,6f,42,89,27,47,08,90,e1,ed,bb,23,e0,d7,51,69,04,f0,be,9d,98,bc,00,\
  88,69,dc,02,20,5b,b4,45,f5,9e,76,48,37,1d,58,1b,34,f9,17,1f,12,c6,98,cb,c0,\
  d0,d3,50,19,2a,db,63,69,6b,31,cb,20

Just to see what would happen, I took a raw DER-encoded x.509 certificate and shoved it into the registry to see if CryptoAPI would accept it, but of course it didn’t, so I needed to figure out what that extra data was. Now, as a middle school student, as a high school student, and as an undergraduate student (though sadly not as a graduate student), I did a lot of reverse-engineering of various binary formats (making a Mega Drive Zero Wing ROM include custom dialogue between “Cracker” and “Admin” about how “All your 802.11b are belong to us” and that “You are on the way to DDoS microsoft.com” was a few days of work in 7th grade). So I was quite ready to go that route. However, I learned long ago that it’s always better to spend a few hours on Google to see if someone else has already done your dirty work for you, because usually someone has. So I did that.

The first result I found was a Microsoft mailing list thread from 2002 where Mitch Gallant and Rebecca Bartlett both inquired about this format. Microsoft’s response was to refuse to provide any documentation, and generally be rude and unhelpful, on the grounds that this use case was unsupported. Now, hang on, because I need to point something out. I’ve asked a lot of software vendors for obscure technical information about their products, nearly all of which were for unsupported use cases. By far my favorite vendor to work with in this area was the robotics hardware company Charmed Labs (they are incredibly nice and helpful, even happily giving me proprietary source code that was protected by patents, for me to do whatever I wanted with, as long as I didn’t distribute or use commercially). But generally speaking, the “good” companies’ responses to such requests follow this kind of formulation:

  1. What you’re trying to do is not something that we officially support.
  2. We think what you’re doing is a bad idea for these reasons.
  3. There may be other reasons that it’s a bad idea, which we haven’t thought of.
  4. We think the “right” way is something else, and we’re happy to help you with that method if you like.
  5. Regardless, here are all the answers to your questions.
  6. If you have any other questions about this unsupported use case, we’ll try to answer, as long as it doesn’t become a time sink for us. Don’t expect super-quick replies, because we’re looking up answers in our spare time.
  7. If any of the info we’re providing turns out to be incomplete or wrong, or you end up getting burned in any other way for doing this, we’re not responsible.

Whenever I’ve gotten a reply like this (and it’s happened reasonably often), I was a happy camper, because I was able to make an informed decision. Sometimes I decided to abandon my quest; other times I disregarded the warning and pressed on anyway. In the latter case, I knew full well what I was getting into, because the vendor had given me sufficient information and context for me to make up my mind. Usually, I was satisfied with my decision at the end of the day. In fact, I cannot remember a case where I did something I seriously regretted after being warned against it like that. I’m sure it could have happened, had the quantum noise been different, but if that had come to pass, I’m confident that I wouldn’t have blamed the vendor for giving me information coupled with advice that I ignored. There were several times where my disregard for the warning resulted in some lost development time or temporary confusion, but seriously, who could possibly be angry for the chance to gain practical experience in an unfamiliar area, particularly given that when I did decide to change course, I now had both the vendor’s expert recommendations and my new practical experience to inform my decision. What more could anyone want? My point is, the good software vendors treat their users like real, sentient people when they ask for information, while the bad software vendors (Microsoft included) treat their users the way that the owners of Number 4 Privet Drive treated their nephew up until mid-1991: Don’t ask questions!

Technically, Microsoft did provide parts (1) through (4) of the above form, but they don’t even qualify for partial credit here, because the reason they gave for (2) is atrocious on its face: once in 2 decades, they moved a store from the registry to the filesystem to handle roaming profiles, and anyone using the registry directly would have had to update their software (with a really minor change) once Windows 2000 came out. Frankly, if you’re unwilling to update your software once in 2 decades, I don’t know why you’re in this field, and you should go get a Ph.D. in Latin Literature so that you don’t need to ever learn anything new in order to stay at the top of your field for your whole life. Anyway, reading that thread was entirely unhelpful, except for the fact that it told me I had made a great choice never interviewing at Microsoft, because if I ever become the kind of tech support robot who shows up in that thread, someone please kill me.

So, I continued to look through Google for a while. And I did find two people who had actually tried to reverse-engineer the blob format. Tim Jacobs had posted some information in 2008, and Willem Jan Hengeveld had posted some other info in 2003. Interestingly, both Tim and Willem had been looking into dealing with bugs in mobile versions of Windows that made it hard to import a certificate any other way. (See, basic research has diverse, and often non-obvious, use cases.) Tim’s documentation wasn’t particularly helpful for me, because while it explained how to solve a specific problem (which wasn’t the problem I had), it didn’t really explain why that solution worked, nor how he figured it out (actually, I’m very skeptical of why his solution even works, and based on my research below, I suspect that he simply got lucky and that his method will spectacularly break in many real-world scenarios). However, Willem’s documentation was very helpful. According to Willem, a cert blob is a sequence of records, each of which consists of a 4-byte propid (which I gather means “property ID”), a 4-byte unknown value (which I assume is reserved by Microsoft for future expansion, since everything I encountered used exactly the same value), a 4-byte size, and then the raw data for that property (whose size in bytes was specified by the size field). Willem also listed the common property ID’s that show up.

There was just one problem: the blob I was looking at had a bunch of property ID’s that weren’t in Willem’s list. So, of course, I Googled for one of the property ID’s that was in Willem’s list, and I ended up finding this page and this page on the Microsoft website, which had a (mostly) complete list. Except… those references had descriptions (which, admittedly, is nice) but not any info on what numerical values the property ID’s were. However, they did include a reference to Wincrypt.h. Ah ha, I thought, I’ve done this before! So I went and looked up that header file in the MinGW source code, and was treated to a complete list of the numerical values of all the property ID’s.

From there, I started gathering a list of which property ID’s Windows seemed to be using, so that I could generate the appropriate information while inserting a cert, given only its DER x.509 encoding. Unfortunately, quite a lot of the property ID’s were for things that looked quite annoying to calculate. After trying to figure out a way to calculate a “signature hash” of an x.509 cert in Golang, and not having any fun whatsoever (mind you, I did find ways to do it, I just knew I would despise the process of coding it, and of ever looking at the horrible code that was bound to result), a thought crossed my mind: What does CryptoAPI do if some of the properties are missing from the blob, as long as the record format is correct? So, I took the blob that Windows had generated, and I wiped everything except for the x.509 cert itself and the 12-byte header for that property. I inserted it into the registry, and visited the corresponding website in Chrome. The website loaded just fine! Then I went back to the registry editor, and refreshed, and was quite surprised to see that the moment that CryptoAPI had validated the cert, it had re-calculated all of the other missing fields, and inserted them into the registry.

So, basically, all of those other properties are, as best I can tell, just an elaborate caching mechanism, completely superfluous for proper operation. Microsoft made CryptoAPI substantially more complex, added at least 4 public-facing API functions (those are just the ones I accidentally ran across), and invented a custom, undocumented binary blob format, all so that they could avoid doing a couple of extra hash operations when verifying a chain that included a previously seen certificate. (Remember, hash operations are fast, while RSA and ECDSA, which aren’t cached here and are still needed to verify cert chains, are slow.)

Typical Microsoft. slow clap

Thanks goes to ncdns developer Hugo Landau and Monero developer Riccardo Spagni for keeping me company on IRC while I figured all of the above out. What does this have to do with Namecoin? You’ll find out in my next post.

Progress on ncdns-nsis

Development nears completion on the NSIS-based Namecoin and ncdns bundle installer for Windows.

The ncdns-nsis repository provides source code for an NSIS-based installer which can automatically install and configure Namecoin Core, ncdns and Unbound and configure name resolution of .bit domains via Unbound.

The installer can install Namecoin Core and Unbound automatically, but also allows users to opt out of the installation of these components if they wish to provide their own.

Completion of the ncdns-nsis installer project will enable the Namecoin project to distribute a Windows binary installer providing a turnkey, configuration-free solution for .bit domain resolution. The installer is also intended to support reproducible builds and can be built from a POSIX system.

At this point, extensive testing is the primary work remaining on the completion of the ncdns-nsis installer.

Namecoin Receives Funding from NLnet Foundation’s Internet Hardening Fund

We’re happy to announce that Namecoin is receiving 29,895 EUR in funding from NLnet Foundation’s Internet Hardening Fund. If you’re unfamiliar with NLnet, you might want to read about NLnet Foundation, or just take a look at the projects they’ve funded over the years (you might see some familiar names). The Internet Hardening Fund is managed by NLnet and funded by the Netherlands Ministry of Economic Affairs. The funding will be used to fund 4 Namecoin developers (Jeremy Rand, Hugo Landau, Brandon Roberts, and Joseph Bisch) to produce a usable decentralized TLS public key infrastructure.

Specifically, the following areas of development will be funded:

  • Integration with DNS functionality of major operating systems. We intend to support GNU/Linux and Windows, including DNS integration for Tor. Other operating system support may be developed if things go well.
  • Integration with TLS certificate validation functionality of major web browsers. We intend to support Chromium, Firefox, and Tor Browser on GNU/Linux and Windows. Other browser support may be developed if things go well.
  • Improvements to the lightweight SPV name lookup client.
  • A lightweight SPV wallet with name support. We intend to use Electrum.
  • Wallet GUI improvements, including Coin Control for name transactions and a name update GUI that doesn’t require knowing JSON.
  • Improved installation automation. We intend to provide a Windows installer that includes a Namecoin client, DNS integration, and TLS integration. Other OS support may be developed if things go well.

We’d like to thank the awesome people at NLnet Foundation for selecting us for this opportunity, as well as the Netherlands Ministry of Economic Affairs for recognizing that a hardened Internet is worth receiving government financial support.

We’ll be posting updates regularly as development proceeds. (Spoiler alert: a few components are already nearly ready for beta releases.)

ICANN58 Summary

As was announced, I represented Namecoin at ICANN58 in Copenhagen. Below is a brief summary of how it went.

  • I presented in the Emerging Identifier Technology Panel.
  • I presented in the Technical Experts Group / Board Joint Meeting.
  • A significant number of people in the ICANN community are interested in Namecoin.
  • While I have not attended previous ICANN events and therefore cannot evaluate this myself, my understanding is that the EIT panel session had an unusually large audience.
  • There is skepticism in the ICANN community of Namecoin’s ability to completely replace the DNS.
    • By far the most common reason for this skepticism is the concern that Namecoin may not be able to scale to DNS’s usage levels.
      • I fully agree that this is a good reason to be skeptical and that work needs to be done in this area.
    • Another concern raised was Namecoin’s lack of privacy in its current form (specifically the risk of transaction graph analysis).
      • The people who raised this concern appear to be satisfied that the Namecoin developers understand that this is a problem and that we intend to fix it. If we fail to fix it adequately, this concern is likely to become more of a big deal.
  • The ICANN community appears to be reasonably accepting of Namecoin’s role as an alternative to DNS; Namecoin makes different tradeoffs from DNS, is therefore likely to be optimal for a different userbase, and can co-exist with DNS in its current state.
  • Several people I met are interested in assisting Namecoin; we are following up with those people.
  • I ran out of business cards in my wallet 3 times in 3 days. Luckily, I carry a large stash of business cards with my travel laptop, so everyone who requested my business card received it.
  • My wallet is currently sufficiently full of business cards from ICANN58 attendees that I’m having trouble easily fitting my credit card into my wallet.
  • The joint meeting of ICANN’s Security and Stability Advisory Committee (SSAC) and the ICANN board included a segment on Special-Use Names and name collisions. For those who are unaware, this is of interest to Namecoin because it would be problematic for Namecoin if ICANN were to allow someone to purchase .bit as a standard DNS TLD.
    • Free-software-friendly video recording is hosted by Namecoin.org.
    • The above recording is converted from ICANN’s official Adobe Connect video recording. Copyright ICANN; used with permission.
    • The discussion of collisions between non-DNS names (such as Namecoin, though Namecoin wasn’t explicitly mentioned) and DNS names (such as if ICANN were to issue the .bit TLD to someone) begins at timestamp 42:25. I highly recommend watching the full segment, but some highlights include:
      • The SAC090 document “SSAC Advisory on the Stability of the Domain Namespace” was cited; most important are 3 Recommendations from SSAC (summarized by Jeremy, apologies for any errors):
        • Recognize that name collisions will always be with us, and they’re not going to go away. There’s no way to control how people use names.
        • It’s important to control the things that you can control: make sure that the parts of the namespace that ICANN controls are predictable (harmonize with private-use names). We need to allow private-use names to exist, in the spirit of innovation.
        • Since we recognize that we are not the only ones who have names that will look like TLD names, and the community is going to use that kind of stuff in an interesting way, we need to have procedures for dealing with other bodies who are going to be creating special-use names for their own purposes. It is important to establish regular communication, how we each recognize each other, how we’re going to work together, and set ourselves up for potentially others (besides IETF) who may want to create lists of names. Be prepared to deal with other groups who are going to have their own lists of names.
      • Steve Crocker (chair of the ICANN board) said the following:

        The IETF has a special names list, a reserved names list, but my understanding is that that’s not a definitive list in the following sense. It takes a while before a name gets onto that list. So, it tends to be on the conservative side. There are other names that are in use but have not gone through an IETF process. From where we’re sitting over at ICANN, if we want to be conservative, we would take into account not only the names on the reserve list from the IETF but also other names where it’s evident there is usage but nobody has come along and said we’re going to – you should reserve this and reserve this and so forth. So, I would think that our obligation is to have a somewhat wider field of view, including not only the official list but also what’s actually happening in the real world. And I can anticipate arguments that say well, there’s no official reason to reject this name [for ICANN issuance, e.g. someone buying the .bit TLD for non-Namecoin use], therefore you must accept it. I would say just the opposite, that we have an obligation to be careful, and if we see reasons why a name should not be allocated, then we have that authority, we have that obligation to do that and to err on the side of caution there.

    • I consider this an extremely good sign.
  • In response to a question in the Public Forum 2 about whether ICANN was looking into adopting Namecoin, Steve Crocker (chair of the ICANN board) commented “These things take time.” The full question and answer are in the ICANN transcript, pages 25-28. Steve’s comment is, in my opinion, a completely reasonable response.
  • We plan to continue engaging with the ICANN community.
  • We plan to continue engaging with IETF on Special-Use Name registration.
  • At this time, I have no reason to expect any hostile action by ICANN toward Namecoin.

As with other conferences, I won’t be releasing details of private conversations, because I want people to be able to talk to me at conferences without being worried that off-the-cuff comments will be publicly published. That said, all of the private conversations I engaged in were highly encouraging.

Huge thanks to David Conrad (ICANN CTO) for inviting me to attend ICANN58, and to Adiel Akplogan (ICANN VP of Technical Engagement) for inviting me to the EIT Panel. Also thanks to ICANN for covering my travel expenses. I hope we can do this again sometime.

QCon London 2017 Summary

As was announced, I represented Namecoin at QCon London 2017. Below is a brief summary of how it went.

The theme of the blockchain track was “Beyond the Hype”. As such, the presentations in the track primarily focused on all the things that can go wrong when using a blockchain. Riccardo Spagni (AKA fluffypony of Monero) is definitely an ideal person to host this track. My talk was on alternate blockchains, with a focus on Namecoin (and some Monero). In the spirit of going “Beyond the Hype”, my talk was almost entirely about things that can go wrong when using a non-Bitcoin blockchain.

I think this is a very important theme for a blockchain track, because the hype attached to the blockchain field is seriously problematic for our field’s credibility. None of us were there to sell our technology, attract investors, or grab media attention – we were there to provide a reality check for an audience who, in large part, had minimal exposure to blockchain technology and wanted to learn more about what use cases it’s good or bad for.

Lots of attendees talked with me over dinner and in the hallway, and I’ll be following up with them ASAP. After QCon, I attended a talk Riccardo gave at Imperial College; several people there were interested in Namecoin. I met up with Riccardo the next day to discuss lots of cool stuff involving Namecoin and Monero.

As with other conferences, I won’t be releasing details of private conversations, because I want people to be able to talk to me at conferences without being worried that off-the-cuff comments will be publicly published.

Huge thanks to Riccardo for inviting me, and to all the QCon conference organizers for an awesome conference (and for covering my travel expenses). It’d be awesome if we can do this again.

A video of my talk is scheduled for release on June 26, 2017.

Namecoin Core 0.13.99-name-tab-beta1 Ready for General Use

Namecoin Core 0.13.99-name-tab-beta1, which has been listed on our Beta Downloads page for a few months, has demonstrated itself to be stable enough that it is now listed on the main Downloads page. Huge thanks to our Lead C++ GUI Engineer Brandon Roberts for his work on this.

Namecoin’s Jeremy Rand will be a speaker at ICANN58

ICANN has invited a Namecoin developer to speak at the ICANN58 meeting on March 11-16 2017 in Copenhagen, and I’m happy to say that I’ve accepted their invitation. Since I’m well aware that this may be surprising to some readers, I think it’s beneficial to everyone to announce it here, and give some details about why I’ll be attending.

The rest of this post will be in the excellent Q&A-style format.

Why did ICANN invite you?

To my understanding, I was invited because of a perception that there was a lack of understanding and dialogue between ICANN and Namecoin about specifically what the goals of each group were. The hope is that by encouraging discussion between ICANN and Namecoin, the groups will have a more accurate idea of what the other is doing and what common interests we might have.

It’s not news to me that ICANN has an interest in Namecoin; an ICANN panel report favorably mentioned us. However, I admit that I was (pleasantly) surprised to receive this invitation.

What do you think of ICANN and DNS?

To be totally honest, I’m not really very knowledgeable about how ICANN operates. I hope to gain some knowledge on this subject at the meeting. That said, I’ve heard that ICANN has some political issues. (Indeed, if James Seng’s comments in the aforementioned report are any indication, this is a recognized issue by ICANN participants, not just from the outside.) This is really not surprising, and as far as I know, it’s not due to any kind of nefarious motivation by ICANN or any people within ICANN. My take is that ICANN’s political issues are likely to be simply because ICANN is very large, and large centralized entities are inevitably going to have political issues. If, in an alternate reality, OpenNIC were wildly successful and ended up as large as ICANN is today, I predict that OpenNIC would end up with political issues too.

Isn’t that just ICANN’s own fault for being centralized?

That’s not ICANN’s fault, it’s the reality of the laws of math. When DNS and ICANN were created, everyone believed that decentralized global consensus was impossible (and this belief was well-supported by a proof by Lamport dating back to the 1970’s). It wasn’t until Satoshi Nakamoto invented Bitcoin that anyone had any credible reason to believe that decentralized global consensus was solvable, and it wasn’t until Appamatto and Aaron Swartz proposed BitDNS and Nakanames 2 years later that anyone really seriously considered applying a Nakamoto blockchain to a DNS-like system.

But Namecoin exists now; doesn’t that make DNS obsolete?

Not really. Namecoin makes a number of design tradeoffs in order to achieve decentralization. Compared to DNS, Namecoin has significantly worse security against run-of-the-mill malware, significantly worse privacy against your nosy friends/neighbors/employer, and significantly worse resistance to squatting and trademark infringement, to list just a few. These are open research problems for Namecoin-like systems, whereas DNS has long ago solved them. I work on Namecoin because Namecoin also has some advantages over DNS, and I think there is a significant user base who want those advantages enough that they are willing to cope with the downsides. But that doesn’t mean that DNS is obsolete, or that I expect Namecoin to replace DNS anytime soon. If, in the future, Namecoin eventually solves those open research problems, and as a result replaces DNS, that’d be cool as heck from my point of view, but if that ever happens, I think it will be far enough in the future that it’s not worth worrying about right now.

Namecoin has almost no funding; if you had the budget of the DNS industry, wouldn’t those open research problems have been solved by now?

That would be inconsistent with the definition of “open research problem”. Funding would certainly help us spend more time tackling those problems, but there’s no guarantee that the problems are even solvable. Also, since no one is offering to give us such a budget, there’s not really much point in speculating here.

Are you being paid to attend?

ICANN is covering my travel expenses. (Naturally, I wasn’t going to ask NMDF to pay for me to travel. We don’t have anywhere near enough funding for that.) Other than that, I’m not being paid to attend.

Has ICANN asked for any control or influence on Namecoin?

Of course not. (And if they did, I would decline – as I assume would the other devs.) It’s entirely standard to talk to people working on related projects; it doesn’t imply any desire to influence or control those projects.

Are you concerned that this will be spun by market manipulators as some kind of sell-out?

I’m reasonably confident that market manipulators will try to profit by spinning this in some way, but that’s not anything new. We’ve already seen market manipulators try to make money by alleging a sell-out, based on everything from our application to Google Summer of Code in 2014 and 2015, to me getting a college scholarship from Google in 2013, to our collaboration with GNUnet, I2P, and Tor to try to register the .bit TLD as a special-use name via IETF. Those same market manipulators will, I assume, use this the same way, probably with the same minimal level of success that they had previously.

If I had any interest in spending my time worrying about market manipulators, I’d be in a different line of work, making way more money than I’m making right now. The best I can do is be transparent about this, so that it’s obvious to anyone who does an ounce of research that nothing nefarious occurred. Transparency FTW.

Will you publicly post your presentation slides?

Sure, why not?

Namecoin’s Jeremy Rand will be a speaker at QCon London 2017

As a result of an invitation from Riccardo Spagni (AKA fluffypony of Monero), I will be speaking at the “Practical Cryptography & Blockchains: Beyond the Hype” track at QCon London 2017 (March 6-8 2017). My talk is entitled “Case Study: Alternate Blockchains”. I will also be on a panel discussion alongside Paul Sztorc, David Vorick, Elaine Ou, Peter Todd, and Riccardo Spagni.

My understanding is that a video of my talk will be published by QCon. Assuming that that’s correct, I will post a link here when it’s available.

Huge thanks to Riccardo for inviting me, and to the QCon organizers for putting on the conference and covering my travel expenses. Looking forward to it!

Lightweight SPV Lookups: Initial Beta

If you watched my lightning talk at Decentralized Web Summit 2016 (and if you didn’t, shame on you – go watch it right now along with the other talks!), you’ll remember that I announced SPV name lookups were working. I’m happy to announce that that code is now published in preliminary form on GitHub, and binaries are available for testing.

You can download it at the Beta Downloads page. Once installed, it’s basically a drop-in replacement for Namecoin Core for any application that does name lookups (such as ncdns). Test reports are greatly appreciated so that we can do a proper release sooner.

Initial syncup using a residential clearnet cable modem connection takes between 5 minutes and 10 minutes, depending on the settings. (It is probably feasible to improve this.) Lookup latency for name_show varies from 2 seconds to 4 milliseconds, depending on the settings. (It is also probably feasible to improve this.)

This work wouldn’t have been possible without the work of some very awesome people whom I need to thank.

First, I need to thank Ross Nicoll from Dogecoin (warning: non-TLS link) for creating libdohj, an altcoin abstraction library that has prevented Namecoin from needing to maintain a fork of BitcoinJ. We’re using the same AuxPoW implementation from libdohj that Dogecoin is using – a fitting repayment, since Dogecoin Core uses the same AuxPoW implementation that Daniel Kraft wrote for Namecoin Core. We look forward to continuing to work with Ross and the other excellent people at Dogecoin on areas of shared interest.

Second, I need to thank Sean Gilligan for his work on bitcoinj-addons, a collection of tools that includes a JSON-RPC server implemented using BitcoinJ, which can substitute for Bitcoin Core. Sean is also a big Namecoin enthusiast. (I also finally got to meet Sean in person at DWS.)

Last but not least, I need to thank Marius Hanne, operator of the webbtc.com block explorer. The SPV lookup client currently is capable of using webbtc.com for extra efficiency (either for checking the height of blocks to download over P2P, or for downloading merkle proofs). Marius has been incredibly helpful at customizing the webbtc.com API for this purpose. webbtc.com is under a free software license (AGPLv3), so you can run your own instance if you like.

Remember: this is a beta, for testing purposes only. Don’t use this for situations where incorrect name responses could lead to results that you aren’t willing to accept.

In addition, some notes about security. SPV protects you from being served expired name data, and protects you from being served unexpired name data that isn’t part of the longest chain. However, the SPV modes other than leveldbtxcache (see the documentation) don’t protect you from being served outdated name data that hasn’t yet expired, nor does it protect you from being served false nonexistence responses, nor does it protect you from someone logging which names you look up. We made an intentional design decision to trust webbtc.com here, rather than the Namecoin P2P network, because the P2P network is unauthenticated, trivially easy to wiretap, and trivially easy to Sybil. leveldbtxcache mode avoids these isues, although it takes about twice as long to synchronize. We have plans to add further improvements in these areas as well. SPV also doesn’t protect you from attackers with a large amount of hashpower. As with Bitcoin, a major reason that miners can’t easily attack end users is because there are enough full nodes on the network to keep the miners honest. If you have the ability to run Namecoin Core (syncup time of a few hours, and a few GB of storage), you should do so – you’ll have better security for yourself, and you’ll be improving the security of other users who can’t run a full node.

Have fun testing!

Decentralized Web Summit Recap

As was mentioned on the forum and /r/Namecoin, I represented Namecoin at the Decentralized Web Summit at the Internet Archive in San Francisco, June 6 - June 10. Lots of awesomeness occurred.

I participated in a panel on naming and identity systems on Wednesday. Other panelists were Christopher Allen (Blockstream), Muneeb Ali (Blockstack), and Joachim Lohkamp (Jolocom); Chelsea Barabas (MIT Center for Civic Media) moderated. The panel had a diverse set of perspectives, and I think the discussion was informative.

On Thursday, I did a lightning talk. The talk briefly introduced Namecoin, and then went on to new developments, specifically new announcements about HTTPS and SPV. The lightning talk concluded with an invitation to talk to us about collaboration, and a plug for my workshop (which immediately followed).

The workshop was basically an intro to actually using Namecoin. I walked the attendees through registering domain names and identities, viewing domain names with ncdns, and logging into websites with NameID. We had some minor technical issues during the workshop (which is to be expected), but nothing too bad. At the end of the workshop, I showed a demo of the TLS code working. (Major thanks go out to fellow Namecoin developers Brandon Roberts, Jonas Östman, Joseph Bisch, and Cassini for helping me put together the workshop.)

But of course, I didn’t fly to San Francisco just to do a panel, lightning talk, and workshop. A major goal was to talk to as many other projects as possible to see where we could collaborate. (No single project is going to decentralize the entire Web, but working together, we might have a shot.) I won’t list all the conversations I had on this post, because I want people to be able to talk freely to me at conferences without being worried that the conversation will be posted for the world to see, but the number of orgs I talked to stands at at least 23. Hopefully we’ll be able to announce some results of these conversations in the near future.

And of course, it wouldn’t be an event by the Internet Archive without archived videos, so here are some of the highlights that Namecoiners will find particularly interesting:

Lightning Talk: Jeremy Rand of Namecoin

Builder’s Day Interview: Tamas Kocsis of ZeroNet (uses Namecoin)

Lightning Talk: Tamas Kocsis of ZeroNet (uses Namecoin)

Naming and User Identities Panel

Overall, it was an excellent event. I highly recommend watching all the other non-Namecoin content as well: full archives of all the talks are here.

I also want to thank Brewster Kahle and Wendy Hanamura for organizing the summit, and Kyle Drake of Neocities, Greg Slepak of okTurtles, and John Light of Bitseed for inviting me to attend. Also thanks to all the other organizers, speakers, and attendees: you’re all awesome. I really hope that Internet Archive makes this a regular event.

NMControl 0.8.1

UPNP Vulnerability in Bitcoin Core affects Namecoin

A vulnerability was found in Bitcoin Core. It allows an attack from malicious peers in the local network via UPNP. Namecoin is affected, too, so everybody should turn off UPNP until further notice.

Fix for OpenSSL Consensus Vulnerability has been deployed on 100% of mining hashpower

Fix for OpenSSL Consensus Vulnerability has been deployed on 100% of mining hashpower. Users of NamecoinQ (i.e. namecoind/Namecoin-Qt 0.3.x) are on semi-SPV security, and should wait for at least 6 confirmations for incoming transactions. Users of Namecoin Core (in beta) are on full-node security. Thanks to the miners for their quick action and everyone else who assisted in the response.

OpenSSL Consensus Vulnerability affects Namecoin

Namecoin Bounty Cornucopia

Namecoin’s 4th Birthday

The Namecoin blockchain is now four years old. Happy birthday!

N-O-D-E Interviews Daniel Kraft

Interview with Namecoin lead developer Daniel Kraft.

Softfork Upcoming

Softfork upcoming with block 212500 (around New Year’s Eve). Pool operators and solo miners please update to v0.3.80 asap. Regular users are recommended to update, too. See the Release notes.

Happy Halving Day!

As of block 210000 the Namecoin block reward halved to 25NMC. Happy halving day!

New Website

Thanks to Shobute for designing and Indolering for pushing the new website.