DNSSEC claims another victim as entire TLD disappears

A country’s top-level domain disappeared from the internet for many people yesterday, apparently due to a DNSSEC key rollover gone wrong.

All domains in Fiji’s ccTLD, .fj, stopped resolving for anyone behind a strict DNSSEC resolver in the early hours of the morning UTC, afternoon local time, and stayed down for over 12 hours.

Some domains may still be affected due to caching, according to the registry and others.

The University of the South Pacific, which runs the domain, said that it had to contact ICANN’s IANA people to get the problem fixed, which took a while because it had to wait for IANA’s US-based support desk to wake up.

IANA head Kim Davies said that in fact its support runs 24/7 and in this case IANA took Fiji’s call at 2.47am local time.

Analyses on mailing lists and by Cloudflare immediately pointed to a misconfiguration in the country’s DNSSEC.

It seems Fiji rolled one of its keys for the first time and messed it up, meaning its zone was signed with a non-existent key.

Resolvers that implement DNSSEC strictly view such misconfigurations as a potential attack and nix the entire affected zone.

It happens surprisingly often, though not usually at the TLD level. That said, a similar problem hit thousands of Sweden’s .se domains, despite the registry having a decade’s more DNSSEC experience than Fiji, last month.

Domain Incite had a similar problem recently when its registrar carried on publishing DNSSEC information for the domain long after I’d stopped paying for it.

UPDATE: This post was updated with comment from IANA.

Thousands of domains hit by downtime after DNSSEC error

Sweden saw thousands of domains go down for hours on Friday, after DNSSEC errors were introduced to the .se zone file.

Local ccTLD registry IIS said in a statement that around 8,000 domains had a “technical difficulty” that started around 1530 local time and lasted around seven hours:

On the afternoon of 4/2, a problem was discovered that concerned approximately 8,000 .se domains. The problem meant that services, such as email and web, that are linked to the affected domains in some cases could not be used or reached. In total, there are approximately 1.49 million .se domains, of which approximately 8,000 were affected.

During the afternoon and evening, a thorough work was done with the troubleshooting and the error could be fixed for the affected .se domains at approximately 22.25.

The problem is believed to have been caused by incorrect DNSSEC signatures being published in the .se zone file. Any machine using a DNSSEC-validating resolver would have seen the errors and flat-out refused to resolve the domain.

This is probably the key drawback of DNSSEC — typically resolvers will treat badly signed domains as if they do not exist, rather than fail over to an unsigned, but resolving, response.

Sweden is not a DNSSEC newbie — .se was the first TLD to deploy the technology, all the way back in 2005, with services for domain holders coming a couple of years later.

Do young people know how to use domain names?

If you’re reading this blog, chances are you’re a fan of domain names. Prepare to be irritated by this TikTok “influencer”.

@timotechanut

Hire a freelancer for any task #fiverr #freelance

♬ original sound – Timoté Chanut

If the video isn’t embedded properly, it’s probably because your browser is blocking third-party cookies from tiktok.com.

It’s part of a long series in which a guy called Timoté Chanut tips off his bewilderingly large audience about useful web sites, largely fun-looking content creation tools.

What’s baffling about these videos is how he teaches his viewer to navigate to the web site in question.

“If you search fiverr.com and click the first link, you can find a freelancer to do just about anything,” he says in the above video.

He demonstrates this by typing the domain name of the web site, in this case fiverr.com, into the Google search bar on the Chrome home page, then clicking the top link in the search results page, which in this case is a Google ad paid for by Fiverr.

Chanut’s TikTok feed is filled with examples of this bizarre navigation technique.

An encouragingly large number of web sites he promotes via his videos are built on new gTLDs such as .earth, .space and .online, or repurposed ccTLDs such as .co, .ai and io. There’s no .com bias here.

But this method of using domain names sure is a head-scratcher.

Is this how kids are using the internet nowadays? Do they not understand how a browser address bar works? Do they not realize that you can just type the goddamn domain into the browser and go right where you want to go, without feeding the Google beast?

Lest you think I’m randomly picking on some 20-year-old French kid, I’ll point out that Chanut has 2.3 million followers on TikTok and runs his own social media consultancy. He’s an “influencer”.

I’ll give him the benefit of the doubt and assume Chanut does know what domain names are and how to use them. Does this imply that he assumes his audience of TikTok-using youngsters do not?

I’ve been asked for over a decade whether domain names are becoming less relevant as apps and search become more popular, and my stock response is to explain that domains are not just about navigation, they’re about identity.

There can be little doubt the navigation component is less relevant than it used to be, but I had no idea it had got so bad.

Verisign saw MASSIVE query spike during Facebook outage

Verisign’s .com and .net name servers saw a huge spike in queries when Facebook went offline for hours last October, Verisign said this week.

Queries for facebook.com, instagram.com, and whatsapp.net peaked at over 900,000 per second during the outage, up from a normal rate of 7,000 per second, a more than 100x increase, the company said in a blog post.

The widely publicized Facebook outage was caused by its IP addresses, including the IP addresses of its DNS servers, being accidentally withdrawn from routing tables. At first it looked to outside observers like a DNS failure.

When computers worldwide failed to find Facebook on their recursive name servers, they went up the hierarchy to Verisign’s .com and .net servers to find out where they’d gone, which led to the spike in traffic to those zones.

Traffic from DNS resolver networks run by Google and Cloudflare grew by 7,000x and 2,000x respectively during the outage, Verisign said.

The company also revealed that the failure of .club and .hsbc TLDs a few days later had a similar effect on the DNS root servers that Verisign operates.

Queries for the two TLDs at the root went up 45x, from 80 to 3,700 queries per second, Verisign said.

While the company said its systems were not overloaded, it subtly criticized DNS resolver networks such as Google and Cloudflare for “unnecessarily aggressive” query-spamming, writing:

We believe it is important for the security, stability and resiliency of the internet’s DNS infrastructure that the implementers of recursive resolvers and public DNS services carefully consider how their systems behave in circumstances where none of a domain name’s authoritative name servers are providing responses, yet the parent zones are providing proper referrals. We feel it is difficult to rationalize the patterns that we are currently observing, such as hundreds of queries per second from individual recursive resolver sources. The global DNS would be better served by more appropriate rate limiting, and algorithms such as exponential backoff, to address these types of cases

Verisign said it is proposing updates to internet standards to address this problem.

ICANN takes the lamest swipe at Namecheap et al over blockchain domains

ICANN has come out swinging against blockchain domains and the registrars that sell them. And by “come out” I mean it’s published a blog post. And by “swinging” I mean “offered the weakest criticism imaginable”.

The post starts off well enough, observing that services marketed as “domain names” that are not automatically compatible with the global DNS are probably not a great purchase, because they don’t work like regular domains.

Using these alternatives requires something like a browser plug-in or to reconfigure your device to use a specialist DNS resolver network, the post notes, before concluding with a brief caveat emptor message.

All good stuff. ICANN has been opposed to alt-root domain efforts for at least 20 years, and the policy is even enshrined in so-called ICP-3, which nobody really talks about any more but appears to still be the law of ICANN Land.

So, which domain-alternatives is ICANN referring to here, and which registrars are selling them? The post states:

Name resolution systems outside the DNS have existed for a long time. One could mention the Sun Microsystem Network Information Service (NIS), the Digital Object Architecture (DOA), or even the Ethereum Name Service (ENS)…

With some ICANN-accredited registrars now selling NIS, DOA, or other similar domains alongside standard domain names, the potential for confusion among unsuspecting customers seems high.

You may be asking: what the heck (or, if you’re like me, fuck) are NIS and DOA domains, and which registrars are selling them?

Great questions.

NIS is an authentication protocol (a bit like LDAP) for Unix networks developed in 1985 (the same year the original DNS standard was finalized) by Sun Microsystems, a company that hasn’t existed in over a decade.

To the best of my knowledge they’ve never been marketed as an alternative to regular domain names. Nobody’s ever used them to address a publicly available web site. Nobody sells them.

DOA, also known as the Handle System, is a more recent idea, first implemented in 1994, before some of you were born. Handles are mostly numeric strings used to address digital objects such as documents. Libraries use them.

The main thing to know about Handles for the purposes of this article is that they’re specifically designed to convey no semantic information whatsoever. They’re not designed to look like domain names and they’re not used that way.

So how many registrars are selling NIS/DOA domains? I haven’t checked them all, but I’m going to go out on a pretty sturdy limb and guess the answer is “none”, which is a lot less than the “some” that ICANN asserts.

But ICANN also mentions the Ethereum Name Service, a much newer and sexier way of cybersquatting, based on the Ethereum cryptocurrency blockchain.

ENS allows people to buy .eth domain names (which do not function in the consensus DNS) for the Ethereum equivalent of about $5. As far as I can tell, you can only buy them through ens.domains, and no ICANN-accredited registrar is functionally capable of selling them.

The ICANN post also contains a brief mention of “Handshake”, and this appears to be what ICANN is actually worried about.

Handshake domains, also known as HNS, look like regular domain names and a handful of ICANN-accredited registrars are actually selling them.

Handshake is also based on blockchain technology, but unlike ENS it also allows people to create their own TLDs (which, again, do not function without special adaptations). Registrars including Namecheap, 101domain and EnCirca sell them.

It’s Namecheap’s storefront hover text, warning that HNS domains don’t work in the regular DNS, that ICANN appears to be paraphrasing in its blog post.

The registrar has a lengthy support article explaining some of the ways you can try to make a Handshake domain work, including an interactive comment thread in which a Namecheap employee suggests that DNS resolvers may choose to resolve HNS TLDs instead of conflicting TLDs that ICANN approves in future.

That’s the kind of thing that should worry ICANN, but it’s got a funny way of expressing that concern. Sun Microsystems? Digital Object Architecture? What’s the message here?

Twenty years ago, I interviewed an ICANN bigwig about New.net, one of the companies attempting to sell alt-root domains at the time. He told me bluntly the company was “breaking the internet” and “selling snake oil”, earning ICANN a snotty lawyer’s letter.

Today’s ICANN post was ostensibly authored by principal technologist Alain Durand, but I’m going to give him the benefit of the doubt and assume comms and legal took their knives to it before it was published.

While some things haven’t changed in the last two decades, others have.

CentralNic gets into artificial intelligence

CentralNic has formed a business unit dedicated to big data and artificial intelligence.

The new Data and Artificial Intelligence Group will be headed by chief data scientist Pawel Rzeszucinski.

The company said that the group will be tasked with leveraging the “vast” amounts of data it generates as a registrar, registry, DNS resolution provider and domain monetization service.

CentralNic said in a press release:

CentralNic stores, manages, and is exposed to huge datasets that can be used for advanced analysis. Examples include; navigation data on tens of millions of daily DNS queries, ad-tech data on tens of millions of domain advertisements, site usage data on hundreds of millions of unique visits and millions of monthly clicks, and similarly extensive data on transactions and registrations.

These extremely large data sets lend themselves perfectly to AI and machine learning applications that can be used to provide a large array of initiatives which will benefit both the Company and our customers. These include; improved customer service, optimised business operations and decision making, enhanced marketing, reduced customer churn and automated detection of non-compliant customer activity.

There’s no mention of licensing its data to third parties, and the company notes that its initiatives will be compliant with current and future privacy rules from the public and private sectors, such as GDPR.

Donuts offers name spinner to show potential attacks

Donuts has launched a tool to show off its TrueName offering, which blocks potential phishing attacks at the domain registry level.

It’s like a regular name spinner, but instead of showing you available domains it shows you visually confusingly similar domains — homographs — that it will block if you register said name in any of Donuts’ portfolio of 2xx (subs, please check) TLDs.

For example, spinning truename.domains returns results such as trʋenɑme.domains (xn--trenme-exc57b.domains) and trᵫname.domains (xn--trname-xk6b.domains), which could be used in phishing attacks.

How many strings get blocked depends largely on what characters are in your name. The letters I and O have a great many visually confusing variants in other non-Latin scripts, and each instance exponentially increases the potential attack vectors.

For example, if I were to register “domainincite” in one of Donuts’ TLDs, Donuts would block 767 homographs at the registry level, but if I were to register “kevinmurphy”, it would only need to block 119.

It only blocks the homographs in the same TLD as the original name. It’s not a replacement for brand protection in other TLDs.

Donuts doesn’t charge anything extra for this service. It’s included in the price of registration and offered as a unique perk for Donuts’ selection of gTLDs.

I gave TrueName a brief post when it launched last year, but I have to say I really like the idea. It’s a rare example of true innovation, rather than simple money-grubbing, that has come from the new gTLD program.

If Verisign were to roll out something similar in .com, it would eliminate a bunch of phishing and cut down on legal fees for big brands chasing phishers and typosquatters through UDRP or the courts.

It was born out of Donuts’ Domain Protected Marks List product, which allows trademark owners to block their brands and homographs across the whole Donuts stable for less money than defensively registering the names individually.

The downside of the spinner tool is of course that, if you’re a bad guy, it simplifies the process of generating samples of homograph Punycode (the ASCII “xn--” string) that can be used in any non-Donuts TLD that supports internationalized domain names.

The tool is limited to 10 domains per spin, however, which limits the potential harm.

Try it out here.

ICANN name servers come under attack

ICANN’s primary name servers came under a distributed denial of service attack, the Org said earlier this week.

The incident appears to have gone largely unnoticed outside of ICANN and seems to have been successfully mitigated before causing any significant damage.

ICANN said on its web site:

ICANN was subjected to a Distributed Denial of Service (DDoS) attack targeting NS.ICANN.ORG. This event did not result in harm to the organization. It was mitigated by redirecting traffic flows through a DDoS scrubbing service.

ns.icann.org is the address of ICANN’s name servers, which handle queries to ICANN-owned domains such as icann.org and iana.org.

The servers are also authoritative for Ugandan ccTLD .ug for some reason, and until a few years ago also handled the .int special-purpose TLD and sponsored gTLD .museum.

ICANN did not disclosed the exact date of the attack, nor speculate about whether it was targeted and why it might have happened.

DNS genius and ICANN key-holder Dan Kaminsky dies at 42

Security researcher Dan Kaminsky, best known for uncovering the so-called “Kaminsky Bug” DNS vulnerability, has reportedly died at the age of 42.

It has been widely reported that Kaminsky’s niece confirmed his death from serious complications from his longstanding diabetes.

On Twitter, she rebutted emerging conspiracy theories that his death was linked to the coronavirus vaccine, which he had received April 12, saying her uncle would “laugh” at such views.

During his career as a white-hat hacker, Kaminsky worked for companies including Cisco, Avaya, and IOActive.

He occasionally spoke at ICANN meetings on security issues, and was since 2010 one of IANA’s seven Recovery Key Share Holders, individuals trusted to hold part of a cryptographic key that would be used to reboot root zone DNSSEC in the case of a massive disaster.

But he was best known for his 2008 discovery of a fundamental flaw in the DNS protocol that allowed cache poisoning, and therefore serious man-in-the middle attacks, across millions of name servers worldwide. He worked with DNS software vendors in private to help them with their patches before the problem was publicly disclosed.

His discoveries led in part to the ongoing push for DNSSEC deployment across the internet.

The vulnerability received widespread attention, even in the mainstream media, and quickly came to bear his name.

For me, my standout memory of Kaminsky is one of his series of annual “Black Ops” talks, at the Defcon 12 conference in Las Vegas in 2004, during which he demonstrated to a rapt audience of hackers how it was possible to stream live radio by caching small chunks of audio data in the TXT fields of DNS records and using DNS queries to quickly retrieve and play them in sequence.

As well as being a bit of a DNS genius, he knew how to work a stage: the crowd went mental and I grabbed him for an interview soon after his talk was over.

His death at such a young age is a big loss for the security community.

Universal Acceptance – making the internet work for everyone [Guest Post]

Editor’s note: this is a guest post written by Aman Masjide, head of compliance at new gTLD registry Radix.

Back in 2014, to foster innovation and to better the choice in domain names, ICANN introduced new generic top-level domains through its New gTLD Program. It was a monumental move that enabled businesses, individuals, and communities across the globe to mark their presence on the internet.

Allowing users to be present digitally in their chosen language (non-ASCII characters and scripts) gave opportunities to local businesses, civil societies, and governments to better serve their communities.

Analysys Mason conservatively estimates that there is scope of $9.8 billion growth in potential revenue from both; existing users who are using new domain names and from new internet users coming online through Internationalized Domain Names (IDNs).

To achieve this, Universal Acceptance of new gTLDs and IDNs is critical in making the Internet more accessible to the next billion users. Founded in February 2015, the Universal Acceptance Steering Group (UASG) undertakes activities to promote Universal Acceptance of all valid domain names and email addresses.

Through its ambassadorship and local Initiative programs, UASG promotes Universal Acceptance globally. Their efforts are divided and executed through five working groups that include:

• Technology Working Group
• Email Address Internationalization Working Group
• Communications Working Group
• Measurement Working Group
• Local Initiatives Working Group

Before we get into the acceptance of new domain extensions (nTLDs), we must first understand what acceptance means and how it’s measured.

The Universal Acceptance Steering Group’s mission sums up acceptance in one short statement: “All domain names and all email addresses work in all software applications.”

While this is a simple understanding of the concept, for an end user of an nTLD, this statement further branches out into multiple questions such as:

• Will my domain name work on all platforms/applications–online or offline?
• Will my email address on a new domain extension get accepted on all websites/platforms and pass all the validation tests?
• Will my emails on new domain extensions, once accepted, stop going into the junk folder?
• Will I be able to use all the features of a website/platform irrespective of my domain extensions? For example, will a social media platform accept a new domain extension in the bio, comments, posts, messenger, etc, and process it exactly like any other legacy TLD?

The Universal Acceptance (UA) of all domain names and email addresses requires that every piece of software is able to accept, validate, process, store, and display them correctly and consistently.

As a new domains registry, it was critical for us to understand what the gaps were and how to close them so that the internet operates the same for nTLD users as it does for the legacy TLD users.

Initial research concluded that UA readiness issues occur when applications are not able to handle the following categories of a domains name or email addresses:

Domain Names

• New short top-level domain names: example.fun, example.site
• New long top-level domain names: example.berlin, example.space
• Internationalized Domain Names: παράδειγμα.ευ

Email Addresses

• ASCII@ASCII; new short or long TLD: ekrem@misal.istanbul
• ASCII@IDN: john@société.org
• Unicode@ASCII: 测试@example.com
• Unicode@IDN: ईमेल@उदाहरण.भारत
• Unicode@IDN; right to left scripts: لیم@لاثم.عقوم ای

For Universal Acceptance to succeed, it needs to be examined holistically.

Over the years, UASG working group members have conducted several gap analysis on programming languages and frameworks, networking command-line tools, web browsers, websites, and have made great strides in acceptance of new domain extensions.

According to UASG’s FY 2020 report, tests conducted on top websites showed that

• The acceptance rate of emails on short nTLDs has increased from 91% in 2017 to 98.3% in 2020.
• The acceptance rate of emails on long nTLDs has increased from 78% in 2017 to 84.8% in 2020.

Note: The table above compares the 2020 results to the earlier 2017 and 2019 testing results.

Two important caveats should be remembered in this case:

• Different email addresses were tested (but they were of the same type).
• The websites tested in 2020 were different from previous ones as they were the 50 most popular in the 20 countries rather than the 1,000 most popular globally.

However, these results may still be used to compare overall trends.

Universal Acceptance Readiness Report 2020 (pdf) also segregated test websites as per different categories such as eCommerce, government, education, etc and the results were promising.

Such studies help UASG ambassadors and advocates to identify and focus on websites of a specific category that require immediate attention. We conducted a similar study at Radix where we analysed top websites belonging to different categories. These were the results (click to enlarge):

While the acceptance rates for new short and new long cases is more than 80% under most categories, we see a drastic dip when a domain is on an IDN TLD. Such comparisons highlight problem areas and provide direction to ambassadors and members who are advocating for Universal Acceptance.

Radix’s contribution to UASG

UA is something that affects nTLD users the most. This is why it’s crucial to focus on the feedback that we receive from them. At Radix, we work closely with our users to ensure we have the first hand information on any UA related issues faced by the customer.

The feedback could be about linkification, validation or acceptance of emails on nTLDs on different websites and platforms. Radix also actively invests its resources in gap analysis by testing various websites and social media platforms. We are also part of the ambassadorship program promoting and supporting local and global UA initiatives.

Here are some of the UASG initiatives that Radix is part of:

• Participating in the Technical And Measurement working group
• Participating in workshops on Universal Acceptance such as:
  – India Internet Week 2020 Online Edition
  – FICCI – Bhashantara – Panel Discussions
• Sharing gaps and issues reported by customers to UASG

At Radix, our objective is to ensure that nTLDs are accepted across websites and platforms. To achieve this, we actively work with UASG and share as many issues and gaps noticed and reported by customers.

Contribution by other registries

A key objective for most registries is to ensure great customer experience when it comes to their nTLDs and I’ve always admired it when registry operators have actively taken initiative and participated in the five UASG groups mentioned above.

One of the ways to do this is to capture all the queries and complaints reported by their customers/registrar partners and share it with UASG. This will help their support team direct their resources in solving the problems and encouraging those websites to become UA compliant.

Contribution by registrars

When it comes to UA-related issues, registrars are the first in chain to receive a complaint or feedback from the user. Therefore, it’s crucial that their support teams have all the necessary information needed on how to best handle such complaints.

For now, they can:

• Inform the customer about the potential UA issue and raise a request on behalf of the customer with UASG. Issues can be logged at – https://uasg.tech/global-support-center/
• Report these instances to the Registry Operator so that they can connect and follow up with UASG.
• Join any of the five working groups and participate.

The path ahead

The UASG is consistently compiling and sharing all the important information needed for organizations and developers to become UA ready. This is not only about ensuring the readiness of a system to accept certain TLDs or emails, but also about realising the full potential of an organization by connecting with people and businesses that might not be even on it’s radar.

Every successful step taken by an organization towards UA readiness is also a step towards equality and inclusiveness on the internet.

Guest poster Aman Masjide leads compliance and abuse mitigation at Radix.