Latest news of the domain name industry

Recent Posts

Root servers whacked after crypto change

Kevin Murphy, March 27, 2019, Domain Tech

The DNS root servers came under accidental attack from name servers across the internet following ICANN’s recent changes to their cryptographic master keys, according to Verisign.

The company, which runs the A and J root servers, said it saw requests for DNSSEC data at the root increase from 15 million a day in October to 1.15 billion a day a week ago.

The cause was the October 11 root Key Signing Key rollover, the first change ICANN had made to the “trust anchor” of DNSSEC since it came online at the root in 2010.

The KSK rollover saw ICANN change the cryptographic keys that rest at the very top of the DNSSEC hierarchy.

The move was controversial. ICANN delayed it for a year after learning about possible disruption at internet endpoints. Its Security and Stability Advisory Committee and even its own board were not unanimous that the roll should go ahead.

But the warnings were largely about the impact on internet users, rather than on the root servers themselves, and the impact was minimal.

Verisign is now saying that requests to its roots for DNSSEC key data increased from 15 million per day to 75 million per day, a five-fold increase, almost overnight.

It was not until January, when the old KSK was marked as “revoked”, did the seriously mahooosive traffic growth begin, however. Verisign’s distinguished engineer Duane Wessels wrote:

Everyone involved expected this to be a non-event. However, we instead saw an even bigger increase in DNSKEY queries coming from a population of root server clients. As of March 21, 2019, Verisign’s root name servers receive about 1.15 billion DNSKEY queries per day, which is 75 times higher than pre-rollover levels and nearly 7 percent of our total steady state query traffic.

Worryingly, the traffic only seemed to be increasing, until March 22, when the revoked key was removed from the root entirely.

Wessels wrote that while the root operators are still investigating, “it would seem that the presence of the revoked key in the zone triggered some unexpected behavior in a population of validating resolvers.”

The root operators hope to have answers in the coming weeks, he wrote.

The next KSK rollover is not expected for years, and the root traffic is now returning to normal levels, so there’s no urgency.

Nazis rejoice! A TLD for you could be coming soon

Kevin Murphy, January 21, 2019, Domain Registries

The domain name system could soon get its first new standard country-code domain for eight years.

This weekend, ICANN’s board of directors is set to vote on whether to allow the delegation of a ccTLD for the relatively new nation of South Sudan.

The string would be .ss.

It would be the first Latin-script ccTLD added to the root since 2010, when .cw and .sx were delegated for Curaçao and Sint Maarten, two of the countries formed by the breakup of the Netherlands Antilles.

Dozens of internationalized domain name ccTLDs — those in non-Latin scripts — have been delegated in the meantime.

But South Sudan is the world’s newest country. It formed in 2011 following an independence referendum that saw it break away from Sudan.

It was recognized by the UN as a sovereign nation in July that year and was given the SS delegation by the International Standards Organization on the ISO 3166-2 list a month later.

The country has been wracked by civil war for almost all of its existence, which may well be a reason why it’s taken so long for a delegation request to come up for an ICANN vote. The warring sides agreed to a peace treaty last year.

South Sudan is among the world’s poorest and least-developed nations, with shocking levels of infant and maternal mortality. Having an unfortunate ccTLD is the very least of its problems.

The choice of .ss was made in 2011 by the new South Sudan government in the full knowledge that it has an uncomfortable alternate meaning in the global north, where the string denotes the Schutzstaffel, the properly evil, black-uniformed bastards in every World War II movie you’ve ever seen.

The Anti-Defamation League classifies “SS” as a “hate symbol” that has been “adopted by white supremacists and neo-Nazis worldwide”.

When South Sudan went to ISO for the SS delegation, then-secretary of telecommunications Stephen Lugga told Reuters

We want our domain name to be ‘SS’ for ‘South Sudan’, but people are telling us ‘SS’ has an association in Europe with Nazis… Some might prefer us to have a different one. We have applied for it anyway, SS, and we are waiting for a reply.

To be fair, it would have been pretty dumb to have applied for a different string, when SS, clearly the obvious choice, was available.

There’s nothing ICANN can do about the string. It takes its lead from the ISO 3166 list. Nor does it have the authority to impose any content-regulation rules on the new registry.

Unless the new South Sudan registry takes a hard line voluntarily, I think it’s a near-certainty that .ss will be used by neo-Nazis who have been turfed out of their regular domains.

The vote of ICANN’s board is scheduled to be part of its main agenda, rather than its consent agenda, so it’s not yet 100% certain that the delegation will be approved.

KSK vote was NOT unanimous

Kevin Murphy, September 18, 2018, Domain Policy

ICANN’s board of directors on Sunday voted to approve the forthcoming security key change at the DNS root, but there was some dissent.

Director Avri Doria, a Nominating Committee appointee, said today that she provided the lone vote against the DNSSEC KSK rollover, which is expected to cause temporary internet access problems for potentially a couple million people next month.

I understand there was also a single abstention to Sunday’s vote.

Doria has released a dissenting statement, in which she said the absence of an external, peer-reviewed study of the risks could prove a problem.

The greatest risk is that out of the millions that will fail after the roll over, some that are serious and may even be critical, may occur; if this happens the lack of peer reviewed studies may be a liability for ICANN, perhaps not legal, but in terms of our reputation as protectors of the stability & security of internet system of names.

She added that she was concerned about the extent that the public has been notified of the rollover plan, and questioned whether the current risk mitigation plan is sufficient.

Doria said she found comments filed by Verisign (pdf) particularly informative to her eventual vote, as well as comments from the At-Large Advisory Committee (pdf), Business Constituency (pdf) and Registries Stakeholder Group (pdf).

These groups had called for more study and data, better outreach, more clearly defined success/failure benchmarks, and more delay.

Doria noted in her dissenting statement that the ICANN board did not have a chance to quiz any of the minority of the members of the Security and Stability Advisory Committee who had called for further delay.

The board’s resolution, apparently arrived at after two hours of formal in-person discussions in Brussels at the weekend, is expected to be published shortly.

The rollover, which has already been delayed a year, is now scheduled to go ahead October 11.

Any impact is expected to be felt within a couple of days, as the change ripples out across the DNS.

ICANN says that any network operator impacted by the change has a simple fix: turn off DNSSEC. Then, if they want, they can update their keys and turn it back on again.

ICANN CTO: no reason to delay KSK rollover

Kevin Murphy, August 15, 2018, Domain Tech

ICANN’s board of directors will be advised to go ahead with a key security change at the DNS root — “the so-called KSK rollover” — this October, according to the organization’s CTO.

“We don’t see any reason to postpone again,” David Conrad told DI on Monday.

If it does go ahead as planned, the rollover will see ICANN change the key-signing key that acts as the trust anchor for the whole DNSSEC-using internet, for the first time since DNSSEC came online in 2010.

It’s been delayed since last October after it emerged that misconfigurations elsewhere in the DNS cloud could see potentially millions of internet users see glitches when the key is rolled.

Ever since then, ICANN and others have been trying to figure out how many people could be adversely affected by the change, and to reduce that number to the greatest extent possible.

The impact has been tricky to estimate due to patchy data.

While it’s been possible to determine a number of resolvers — about 8,000 — that definitely are poorly configured, that only represents a subset of the total number. It’s also been hard to map that to endpoints due to “resolvers behind resolvers behind resolvers”, Conrad said.

“The problem here is that it’s sort of a subjective evaluation,” he said. “We can’t rely on the data were seeing. We’re seeing the resolvers but we’re not seeing the users behind the resolvers.”

Some say that the roll is still too risky to carry out without better visibility into the potential impact, but others say that more delays would lead to more networks and devices becoming DNSSEC-compatible, potentially leading to even greater problems after the eventual rollover.

ICANN knows of about 8,000 resolver IP addresses that are likely to stop working properly after the rollover, because they only support the current KSK, but that’s only counting resolvers that automatically report their status to the root using a relatively new internet standard. There’s a blind spot concerning resolvers that do not have that feature turned on.

ICANN has also had difficulty reaching out to the network operators behind these resolvers, with good contact information apparently only available for about a quarter of the affected IP addresses, Conrad said.

Right now, the best data available suggests that 0.05% of the internet’s population could see access issues after the October 11 rollover, according to Conrad.

That’s about two million people, but it’s 10 times fewer people than the 0.5% acceptable collateral damage threshold outlined in ICANN’s rollover plan.

The 0.05% number comes from research by APNIC, which used Google’s advertising system to place “zero-pixel ads” to check whether individual user endpoints were using compatible resolvers or not.

If problems do emerge October 11 the temporary solution is apparently quite quick to implement — network operators can simply turn off DNSSEC, assuming they know that’s what they’re supposed to do.

But still, if a million or two internet users could have their day ruined by the rollover, why do it at all?

It’s not as if the KSK is in any danger of being cracked any time soon. Conrad explained that a successful brute-force attack on the 2048-bit RSA key would take longer than the lifetime of the universe using current technology.

Rather, the practice of rolling the key every five years is to get network operators and developers accustomed to the idea that the KSK is not a permanent fixture that can be hard-coded into their systems, Conrad said.

It’s a problem comparable to new gTLD name collisions or the Y2K problem, instances where developers respectively hard-coded assumptions about valid TLDs or the century into their software.

ICANN has already been reaching out to the managers of open-source projects on repositories such as Github that have been seen to hard-code the current KSK into their software, Conrad said.

Separately, Wes Hardaker at the University of Southern California Information Sciences Institute discovered that a popular VPN client was misconfigured. Outreach to the developer saw the problem fixed, reducing the number of users who will be affected by the roll.

“What we’re trying to avoid is having these keys hardwired into firmware, so that that it would never be changeable,” he said. “The idea is if you exercise the infrastructure frequently enough, people will know the that the key is not permanent configuration, it’s not something embedded in concrete.”

One change that ICANN may want to make in future is to change the algorithm used to generate the KSK.

Right now it’s using RSA, but Conrad said it has downsides such as rather large signature size, which leads to heavier DNSSEC traffic. By switching to elliptical curve cryptography, signatures could be reduced by “orders of magnitude”, leading to a more efficient and slimline DNS infrastructure, Conrad said.

Last week, ICANN’s Root Server Stability Advisory Committee issued an advisory (pdf) that essentially gave ICANN the all-clear to go ahead with the roll.

The influential Security and Stability Advisory Committee has yet to issue its own advisory, however, despite being asked to do so by August 10.

Could SSAC be more cautious in its advice? We’ll have to wait and see, but perhaps not too long; the current plan is for the ICANN board to consider whether to go ahead with the roll during its three-day Brussels retreat, which starts September 14.

Hacker hostage crisis at ICANN secret key ceremony! (on TV)

Kevin Murphy, March 24, 2017, Gossip

One of ICANN’s Seven Secret Key-Holders To The Internet got taken out as part of an elaborate heist or something on American TV this week.

In tense scenes, a couple of secret agents or something with guns were forced to break into one of ICANN’s quarterly root zone key signing ceremonies to prevent a hacker or terrorist or something from something something, something something.

The stand-off came after the secret agents or whatever discovered that a hacker called Mayhew had poisoned a guy named Adler, causing a heart attack, in order to secure his position as a replacement ICANN key-holder and hijack the ceremony.

This all happened on a TV show called Blacklist: Redemption that aired in the US March 16.

I’d be lying if I said I fully understood what was supposed to be going on in the episode, not being a regular viewer of the series, but here’s the exposition from the beginning of the second act.

Black List

Botox Boss Lady: Seven keys control the internet? That can’t be possible.

Neck Beard Exposition Guy: They don’t control what’s on it, just how to secure it. All domain names have an assigned number. But who assigns the numbers?

Soap Opera Secret Agent: Key holders?

Neck Beard Exposition Guy: Seven security experts randomly selected by ICANN, the Internet Corporation for Assigned Names and Numbers.

Bored Secret Agent: Max Adler’s wife mentioned a key ceremony.

Neck Beard Exposition Guy: Yeah, four times a year the key holders meet to generate a master key and to assign new numbers, to make life difficult for hackers who want to direct folks to malicious sites or steal their credit card information.

Botox Boss Lady: But by being at the ceremony, Mayhew gets around those precautions?

Neck Beard Exposition Guy: Oh, he does more than that. He can route any domain name to him.

That’s the genuine dialogue. ICANN, jarringly, isn’t fictionalized in the way one might usually expect from US TV drama.

The scene carries on to explain the elaborate security precautions ICANN has put in place around its key-signing ceremonies, including biometrics, smart cards and the like.

The fast-moving show then cuts to the aforementioned heist situation, in which our villain of the week takes an ICANN staffer hostage before using the root’s DNSSEC keys to somehow compromise a government data drop and download a McGuffin.

Earlier this week I begged Matt Larson, ICANN’s VP of research and a regular participant in the ceremonies (which are real) to watch the show and explain to me what bits reflect reality and what was plainly bogus.

“There are some points about it that are quite close to how the how the root KSK administration works,” he said, describing the depiction as “kind of surreal”.

“But then they take it not one but two steps further. The way the ceremony happens is not accurate, the consequences of what happens at the ceremony are not accurate,” he added.

“They talk about how at the ceremony we generate a key, well that’s not true. It’s used for signing a new key. And then they talk about how as a result of the ceremony anyone can intercept any domain name anywhere and of course that’s not true.”

The ceremonies are used to sign the keys that make end-to-end DNSSEC possible. By signing the root, DNSSEC resolvers have a “chain of trust” that goes all the way to the top of the DNS hierarchy.

Black ListThe root keys just secure the bit between the root at the TLDs. Compromising them would not enable a hacker to immediately start downloading data from the site of his choosing, as depicted in the show. He’d then have to go on to compromise the rest of the chain.

“You’d have to create an entire path of spoofed zones to who you wanted to impersonate,” Larson said. “Your fake root zone would have to delegate to a fake TLD zone to a fake SLD zone and so on so you could finally convince someone they were going to the address that you wanted.”

“If you could somehow compromise the processes at the root, that alone doesn’t give you anything,” he said.

But the show did present a somewhat realistic description of how the ceremony rooms (located in Virginia and California, not Manhattan as seen on TV) are secured.

Among other precautions, the facilities are secured with smart cards and PINs, retina scans for ICANN staff, and have reinforced walls to prevent somebody coming in with a sledgehammer, Larson said.

Blacklist: Redemption airs on Thursday nights on NBC in the US, but I wouldn’t bother if I were you.