Coronavirus could cause “high risk of widespread outages”, ICANN says

There’s a “high risk of widespread outages” in the DNS if ICANN can’t get enough people in the same room for its next root DNSSEC ceremony because of the coronavirus pandemic.

That’s according to ICANN’s own board of directors, which yesterday published a contingency plan that — in the worst case scenario — could see parts of the internet come to a screeching halt in July.

The problem is with the elaborate “ceremonies” that ICANN and its IANA/PTI unit uses to make sure the internet can support DNSSEC — the secure version of the DNS protocol — all the way from the root servers down.

Every quarter, ICANN, Verisign and a select few “Trusted Community Representatives” from all over the world meet in person at one of two secure US-based facilities to generate the public Zone Signing Keys for the root.

In addition to the complex cryptographic stuff happening in the computers, there’s a shedload of physical security, such as retinal scans, PIN-based locks, and reinforced walls.

And the “secret key-holders”, memorably fictionalized in a US spy drama a few years ago, actually have physical keys that they must bring to these ceremonies.

The events are broadcast live and archived on YouTube, where they typically get anything from a few hundred to a few thousand views.

Obviously, with the key-holders dotted all over the globe and most under some form of coronavirus-related lockdown, getting a quorum into the same facility at the same time — originally, Culpeper, Virginia on April 23 — isn’t going to be possible.

So IANA has made the decision to instead move the ceremony to the facility in El Segundo, California, within easy driving distance of ICANN’s headquarters, and have it carried out almost entirely by ICANN staff, wrapped in personal protective equipment and keeping their distance from each other.

The TCRs for El Segundo live in Mauritius, Spain, Russia, Tanzania, Uruguay and on the east-coast of the US, according to ICANN.

Four of these key-holders have mailed their keys to different IANA staff “wrapped in opaque material” and sealed in “tamper-evident bags”. These IANA employees will stand in for the TCRs, who will be watching remotely to verify that nothing fishy is going on.

Verisign and the independent auditors will also be watching remotely.

That’s the current plan, anyway, and I’ve no reason to believe it won’t go ahead, but ICANN’s new contingency plans do provide four alternatives.

It’s already discarded the first two options, so if the current, third, plan for the ceremony can’t go ahead before June 19 for some reason, all that would be left is the nuclear option.

Option D: Suspend signing of the DNS root zone

This is the final option if there is no conceivable way to activate the KSK and perform signing operations. There would need to be a massive education campaign at short notice to have resolver operators disable DNSSEC validation. There is a high risk of widespread outages as it is not possible to ensure global implementation, and high risk this will fatally compromise trust in DNSSEC in general as a technology.

This is considered highly unlikely, but nonetheless the final option. Without exercising the option, in the absence of a successful key signing ceremony, DNSSEC validation would be unsuccessful starting in July 2020.

The reason for this scenario is that DNSSEC keys have a finite time-to-live and after that period expires they stop functioning, which means anyone validating DNSSEC on their network may well stop resolving the signed zones.

ICANN typically generates the keys one quarter in advance, so the current key expires at the start of July.

However, the planned April 23 ceremony will generate three quarters worth of keys in advance, so the root should be good until the end of March 2021, assuming everything goes according to plan.

Clearly, the idea that half the planet might be on the verge of lockdown wasn’t taken into consideration on February 12, the last ceremony, when ICANN’s biggest problem was that it couldn’t get into one of its safes.

If you’re interested in more about the ceremony and the coronavirus-related changes, info can be found here.

Hacker hostage crisis at ICANN secret key ceremony! (on TV)

One of ICANN’s Seven Secret Key-Holders To The Internet got taken out as part of an elaborate heist or something on American TV this week.
In tense scenes, a couple of secret agents or something with guns were forced to break into one of ICANN’s quarterly root zone key signing ceremonies to prevent a hacker or terrorist or something from something something, something something.
The stand-off came after the secret agents or whatever discovered that a hacker called Mayhew had poisoned a guy named Adler, causing a heart attack, in order to secure his position as a replacement ICANN key-holder and hijack the ceremony.
This all happened on a TV show called Blacklist: Redemption that aired in the US March 16.
I’d be lying if I said I fully understood what was supposed to be going on in the episode, not being a regular viewer of the series, but here’s the exposition from the beginning of the second act.

Botox Boss Lady: Seven keys control the internet? That can’t be possible.
Neck Beard Exposition Guy: They don’t control what’s on it, just how to secure it. All domain names have an assigned number. But who assigns the numbers?
Soap Opera Secret Agent: Key holders?
Neck Beard Exposition Guy: Seven security experts randomly selected by ICANN, the Internet Corporation for Assigned Names and Numbers.
Bored Secret Agent: Max Adler’s wife mentioned a key ceremony.
Neck Beard Exposition Guy: Yeah, four times a year the key holders meet to generate a master key and to assign new numbers, to make life difficult for hackers who want to direct folks to malicious sites or steal their credit card information.
Botox Boss Lady: But by being at the ceremony, Mayhew gets around those precautions?
Neck Beard Exposition Guy: Oh, he does more than that. He can route any domain name to him.

That’s the genuine dialogue. ICANN, jarringly, isn’t fictionalized in the way one might usually expect from US TV drama.
The scene carries on to explain the elaborate security precautions ICANN has put in place around its key-signing ceremonies, including biometrics, smart cards and the like.
The fast-moving show then cuts to the aforementioned heist situation, in which our villain of the week takes an ICANN staffer hostage before using the root’s DNSSEC keys to somehow compromise a government data drop and download a McGuffin.
Earlier this week I begged Matt Larson, ICANN’s VP of research and a regular participant in the ceremonies (which are real) to watch the show and explain to me what bits reflect reality and what was plainly bogus.
“There are some points about it that are quite close to how the how the root KSK administration works,” he said, describing the depiction as “kind of surreal”.
“But then they take it not one but two steps further. The way the ceremony happens is not accurate, the consequences of what happens at the ceremony are not accurate,” he added.
“They talk about how at the ceremony we generate a key, well that’s not true. It’s used for signing a new key. And then they talk about how as a result of the ceremony anyone can intercept any domain name anywhere and of course that’s not true.”
The ceremonies are used to sign the keys that make end-to-end DNSSEC possible. By signing the root, DNSSEC resolvers have a “chain of trust” that goes all the way to the top of the DNS hierarchy.
The root keys just secure the bit between the root at the TLDs. Compromising them would not enable a hacker to immediately start downloading data from the site of his choosing, as depicted in the show. He’d then have to go on to compromise the rest of the chain.
“You’d have to create an entire path of spoofed zones to who you wanted to impersonate,” Larson said. “Your fake root zone would have to delegate to a fake TLD zone to a fake SLD zone and so on so you could finally convince someone they were going to the address that you wanted.”
“If you could somehow compromise the processes at the root, that alone doesn’t give you anything,” he said.
But the show did present a somewhat realistic description of how the ceremony rooms (located in Virginia and California, not Manhattan as seen on TV) are secured.
Among other precautions, the facilities are secured with smart cards and PINs, retina scans for ICANN staff, and have reinforced walls to prevent somebody coming in with a sledgehammer, Larson said.
Blacklist: Redemption airs on Thursday nights on NBC in the US, but I wouldn’t bother if I were you.