Verisign saw MASSIVE query spike during Facebook outage

Verisign’s .com and .net name servers saw a huge spike in queries when Facebook went offline for hours last October, Verisign said this week.

Queries for facebook.com, instagram.com, and whatsapp.net peaked at over 900,000 per second during the outage, up from a normal rate of 7,000 per second, a more than 100x increase, the company said in a blog post.

The widely publicized Facebook outage was caused by its IP addresses, including the IP addresses of its DNS servers, being accidentally withdrawn from routing tables. At first it looked to outside observers like a DNS failure.

When computers worldwide failed to find Facebook on their recursive name servers, they went up the hierarchy to Verisign’s .com and .net servers to find out where they’d gone, which led to the spike in traffic to those zones.

Traffic from DNS resolver networks run by Google and Cloudflare grew by 7,000x and 2,000x respectively during the outage, Verisign said.

The company also revealed that the failure of .club and .hsbc TLDs a few days later had a similar effect on the DNS root servers that Verisign operates.

Queries for the two TLDs at the root went up 45x, from 80 to 3,700 queries per second, Verisign said.

While the company said its systems were not overloaded, it subtly criticized DNS resolver networks such as Google and Cloudflare for “unnecessarily aggressive” query-spamming, writing:

We believe it is important for the security, stability and resiliency of the internet’s DNS infrastructure that the implementers of recursive resolvers and public DNS services carefully consider how their systems behave in circumstances where none of a domain name’s authoritative name servers are providing responses, yet the parent zones are providing proper referrals. We feel it is difficult to rationalize the patterns that we are currently observing, such as hundreds of queries per second from individual recursive resolver sources. The global DNS would be better served by more appropriate rate limiting, and algorithms such as exponential backoff, to address these types of cases

Verisign said it is proposing updates to internet standards to address this problem.

ICANN takes the lamest swipe at Namecheap et al over blockchain domains

ICANN has come out swinging against blockchain domains and the registrars that sell them. And by “come out” I mean it’s published a blog post. And by “swinging” I mean “offered the weakest criticism imaginable”.

The post starts off well enough, observing that services marketed as “domain names” that are not automatically compatible with the global DNS are probably not a great purchase, because they don’t work like regular domains.

Using these alternatives requires something like a browser plug-in or to reconfigure your device to use a specialist DNS resolver network, the post notes, before concluding with a brief caveat emptor message.

All good stuff. ICANN has been opposed to alt-root domain efforts for at least 20 years, and the policy is even enshrined in so-called ICP-3, which nobody really talks about any more but appears to still be the law of ICANN Land.

So, which domain-alternatives is ICANN referring to here, and which registrars are selling them? The post states:

Name resolution systems outside the DNS have existed for a long time. One could mention the Sun Microsystem Network Information Service (NIS), the Digital Object Architecture (DOA), or even the Ethereum Name Service (ENS)…

With some ICANN-accredited registrars now selling NIS, DOA, or other similar domains alongside standard domain names, the potential for confusion among unsuspecting customers seems high.

You may be asking: what the heck (or, if you’re like me, fuck) are NIS and DOA domains, and which registrars are selling them?

Great questions.

NIS is an authentication protocol (a bit like LDAP) for Unix networks developed in 1985 (the same year the original DNS standard was finalized) by Sun Microsystems, a company that hasn’t existed in over a decade.

To the best of my knowledge they’ve never been marketed as an alternative to regular domain names. Nobody’s ever used them to address a publicly available web site. Nobody sells them.

DOA, also known as the Handle System, is a more recent idea, first implemented in 1994, before some of you were born. Handles are mostly numeric strings used to address digital objects such as documents. Libraries use them.

The main thing to know about Handles for the purposes of this article is that they’re specifically designed to convey no semantic information whatsoever. They’re not designed to look like domain names and they’re not used that way.

So how many registrars are selling NIS/DOA domains? I haven’t checked them all, but I’m going to go out on a pretty sturdy limb and guess the answer is “none”, which is a lot less than the “some” that ICANN asserts.

But ICANN also mentions the Ethereum Name Service, a much newer and sexier way of cybersquatting, based on the Ethereum cryptocurrency blockchain.

ENS allows people to buy .eth domain names (which do not function in the consensus DNS) for the Ethereum equivalent of about $5. As far as I can tell, you can only buy them through ens.domains, and no ICANN-accredited registrar is functionally capable of selling them.

The ICANN post also contains a brief mention of “Handshake”, and this appears to be what ICANN is actually worried about.

Handshake domains, also known as HNS, look like regular domain names and a handful of ICANN-accredited registrars are actually selling them.

Handshake is also based on blockchain technology, but unlike ENS it also allows people to create their own TLDs (which, again, do not function without special adaptations). Registrars including Namecheap, 101domain and EnCirca sell them.

It’s Namecheap’s storefront hover text, warning that HNS domains don’t work in the regular DNS, that ICANN appears to be paraphrasing in its blog post.

The registrar has a lengthy support article explaining some of the ways you can try to make a Handshake domain work, including an interactive comment thread in which a Namecheap employee suggests that DNS resolvers may choose to resolve HNS TLDs instead of conflicting TLDs that ICANN approves in future.

That’s the kind of thing that should worry ICANN, but it’s got a funny way of expressing that concern. Sun Microsystems? Digital Object Architecture? What’s the message here?

Twenty years ago, I interviewed an ICANN bigwig about New.net, one of the companies attempting to sell alt-root domains at the time. He told me bluntly the company was “breaking the internet” and “selling snake oil”, earning ICANN a snotty lawyer’s letter.

Today’s ICANN post was ostensibly authored by principal technologist Alain Durand, but I’m going to give him the benefit of the doubt and assume comms and legal took their knives to it before it was published.

While some things haven’t changed in the last two decades, others have.

CentralNic gets into artificial intelligence

CentralNic has formed a business unit dedicated to big data and artificial intelligence.

The new Data and Artificial Intelligence Group will be headed by chief data scientist Pawel Rzeszucinski.

The company said that the group will be tasked with leveraging the “vast” amounts of data it generates as a registrar, registry, DNS resolution provider and domain monetization service.

CentralNic said in a press release:

CentralNic stores, manages, and is exposed to huge datasets that can be used for advanced analysis. Examples include; navigation data on tens of millions of daily DNS queries, ad-tech data on tens of millions of domain advertisements, site usage data on hundreds of millions of unique visits and millions of monthly clicks, and similarly extensive data on transactions and registrations.

These extremely large data sets lend themselves perfectly to AI and machine learning applications that can be used to provide a large array of initiatives which will benefit both the Company and our customers. These include; improved customer service, optimised business operations and decision making, enhanced marketing, reduced customer churn and automated detection of non-compliant customer activity.

There’s no mention of licensing its data to third parties, and the company notes that its initiatives will be compliant with current and future privacy rules from the public and private sectors, such as GDPR.

Donuts offers name spinner to show potential attacks

Donuts has launched a tool to show off its TrueName offering, which blocks potential phishing attacks at the domain registry level.

It’s like a regular name spinner, but instead of showing you available domains it shows you visually confusingly similar domains — homographs — that it will block if you register said name in any of Donuts’ portfolio of 2xx (subs, please check) TLDs.

For example, spinning truename.domains returns results such as trʋenɑme.domains (xn--trenme-exc57b.domains) and trᵫname.domains (xn--trname-xk6b.domains), which could be used in phishing attacks.

How many strings get blocked depends largely on what characters are in your name. The letters I and O have a great many visually confusing variants in other non-Latin scripts, and each instance exponentially increases the potential attack vectors.

For example, if I were to register “domainincite” in one of Donuts’ TLDs, Donuts would block 767 homographs at the registry level, but if I were to register “kevinmurphy”, it would only need to block 119.

It only blocks the homographs in the same TLD as the original name. It’s not a replacement for brand protection in other TLDs.

Donuts doesn’t charge anything extra for this service. It’s included in the price of registration and offered as a unique perk for Donuts’ selection of gTLDs.

I gave TrueName a brief post when it launched last year, but I have to say I really like the idea. It’s a rare example of true innovation, rather than simple money-grubbing, that has come from the new gTLD program.

If Verisign were to roll out something similar in .com, it would eliminate a bunch of phishing and cut down on legal fees for big brands chasing phishers and typosquatters through UDRP or the courts.

It was born out of Donuts’ Domain Protected Marks List product, which allows trademark owners to block their brands and homographs across the whole Donuts stable for less money than defensively registering the names individually.

The downside of the spinner tool is of course that, if you’re a bad guy, it simplifies the process of generating samples of homograph Punycode (the ASCII “xn--” string) that can be used in any non-Donuts TLD that supports internationalized domain names.

The tool is limited to 10 domains per spin, however, which limits the potential harm.

Try it out here.

ICANN name servers come under attack

ICANN’s primary name servers came under a distributed denial of service attack, the Org said earlier this week.

The incident appears to have gone largely unnoticed outside of ICANN and seems to have been successfully mitigated before causing any significant damage.

ICANN said on its web site:

ICANN was subjected to a Distributed Denial of Service (DDoS) attack targeting NS.ICANN.ORG. This event did not result in harm to the organization. It was mitigated by redirecting traffic flows through a DDoS scrubbing service.

ns.icann.org is the address of ICANN’s name servers, which handle queries to ICANN-owned domains such as icann.org and iana.org.

The servers are also authoritative for Ugandan ccTLD .ug for some reason, and until a few years ago also handled the .int special-purpose TLD and sponsored gTLD .museum.

ICANN did not disclosed the exact date of the attack, nor speculate about whether it was targeted and why it might have happened.

DNS genius and ICANN key-holder Dan Kaminsky dies at 42

Security researcher Dan Kaminsky, best known for uncovering the so-called “Kaminsky Bug” DNS vulnerability, has reportedly died at the age of 42.

It has been widely reported that Kaminsky’s niece confirmed his death from serious complications from his longstanding diabetes.

On Twitter, she rebutted emerging conspiracy theories that his death was linked to the coronavirus vaccine, which he had received April 12, saying her uncle would “laugh” at such views.

During his career as a white-hat hacker, Kaminsky worked for companies including Cisco, Avaya, and IOActive.

He occasionally spoke at ICANN meetings on security issues, and was since 2010 one of IANA’s seven Recovery Key Share Holders, individuals trusted to hold part of a cryptographic key that would be used to reboot root zone DNSSEC in the case of a massive disaster.

But he was best known for his 2008 discovery of a fundamental flaw in the DNS protocol that allowed cache poisoning, and therefore serious man-in-the middle attacks, across millions of name servers worldwide. He worked with DNS software vendors in private to help them with their patches before the problem was publicly disclosed.

His discoveries led in part to the ongoing push for DNSSEC deployment across the internet.

The vulnerability received widespread attention, even in the mainstream media, and quickly came to bear his name.

For me, my standout memory of Kaminsky is one of his series of annual “Black Ops” talks, at the Defcon 12 conference in Las Vegas in 2004, during which he demonstrated to a rapt audience of hackers how it was possible to stream live radio by caching small chunks of audio data in the TXT fields of DNS records and using DNS queries to quickly retrieve and play them in sequence.

As well as being a bit of a DNS genius, he knew how to work a stage: the crowd went mental and I grabbed him for an interview soon after his talk was over.

His death at such a young age is a big loss for the security community.

Universal Acceptance – making the internet work for everyone [Guest Post]

Editor’s note: this is a guest post written by Aman Masjide, head of compliance at new gTLD registry Radix.

Back in 2014, to foster innovation and to better the choice in domain names, ICANN introduced new generic top-level domains through its New gTLD Program. It was a monumental move that enabled businesses, individuals, and communities across the globe to mark their presence on the internet.

Allowing users to be present digitally in their chosen language (non-ASCII characters and scripts) gave opportunities to local businesses, civil societies, and governments to better serve their communities.

Analysys Mason conservatively estimates that there is scope of $9.8 billion growth in potential revenue from both; existing users who are using new domain names and from new internet users coming online through Internationalized Domain Names (IDNs).

To achieve this, Universal Acceptance of new gTLDs and IDNs is critical in making the Internet more accessible to the next billion users. Founded in February 2015, the Universal Acceptance Steering Group (UASG) undertakes activities to promote Universal Acceptance of all valid domain names and email addresses.

Through its ambassadorship and local Initiative programs, UASG promotes Universal Acceptance globally. Their efforts are divided and executed through five working groups that include:

• Technology Working Group
• Email Address Internationalization Working Group
• Communications Working Group
• Measurement Working Group
• Local Initiatives Working Group

Before we get into the acceptance of new domain extensions (nTLDs), we must first understand what acceptance means and how it’s measured.

The Universal Acceptance Steering Group’s mission sums up acceptance in one short statement: “All domain names and all email addresses work in all software applications.”

While this is a simple understanding of the concept, for an end user of an nTLD, this statement further branches out into multiple questions such as:

• Will my domain name work on all platforms/applications–online or offline?
• Will my email address on a new domain extension get accepted on all websites/platforms and pass all the validation tests?
• Will my emails on new domain extensions, once accepted, stop going into the junk folder?
• Will I be able to use all the features of a website/platform irrespective of my domain extensions? For example, will a social media platform accept a new domain extension in the bio, comments, posts, messenger, etc, and process it exactly like any other legacy TLD?

The Universal Acceptance (UA) of all domain names and email addresses requires that every piece of software is able to accept, validate, process, store, and display them correctly and consistently.

As a new domains registry, it was critical for us to understand what the gaps were and how to close them so that the internet operates the same for nTLD users as it does for the legacy TLD users.

Initial research concluded that UA readiness issues occur when applications are not able to handle the following categories of a domains name or email addresses:

Domain Names

• New short top-level domain names: example.fun, example.site
• New long top-level domain names: example.berlin, example.space
• Internationalized Domain Names: παράδειγμα.ευ

Email Addresses

• ASCII@ASCII; new short or long TLD: ekrem@misal.istanbul
• ASCII@IDN: john@société.org
• Unicode@ASCII: 测试@example.com
• Unicode@IDN: ईमेल@उदाहरण.भारत
• Unicode@IDN; right to left scripts: لیم@لاثم.عقوم ای

For Universal Acceptance to succeed, it needs to be examined holistically.

Over the years, UASG working group members have conducted several gap analysis on programming languages and frameworks, networking command-line tools, web browsers, websites, and have made great strides in acceptance of new domain extensions.

According to UASG’s FY 2020 report, tests conducted on top websites showed that

• The acceptance rate of emails on short nTLDs has increased from 91% in 2017 to 98.3% in 2020.
• The acceptance rate of emails on long nTLDs has increased from 78% in 2017 to 84.8% in 2020.

Note: The table above compares the 2020 results to the earlier 2017 and 2019 testing results.

Two important caveats should be remembered in this case:

• Different email addresses were tested (but they were of the same type).
• The websites tested in 2020 were different from previous ones as they were the 50 most popular in the 20 countries rather than the 1,000 most popular globally.

However, these results may still be used to compare overall trends.

Universal Acceptance Readiness Report 2020 (pdf) also segregated test websites as per different categories such as eCommerce, government, education, etc and the results were promising.

Such studies help UASG ambassadors and advocates to identify and focus on websites of a specific category that require immediate attention. We conducted a similar study at Radix where we analysed top websites belonging to different categories. These were the results (click to enlarge):

While the acceptance rates for new short and new long cases is more than 80% under most categories, we see a drastic dip when a domain is on an IDN TLD. Such comparisons highlight problem areas and provide direction to ambassadors and members who are advocating for Universal Acceptance.

Radix’s contribution to UASG

UA is something that affects nTLD users the most. This is why it’s crucial to focus on the feedback that we receive from them. At Radix, we work closely with our users to ensure we have the first hand information on any UA related issues faced by the customer.

The feedback could be about linkification, validation or acceptance of emails on nTLDs on different websites and platforms. Radix also actively invests its resources in gap analysis by testing various websites and social media platforms. We are also part of the ambassadorship program promoting and supporting local and global UA initiatives.

Here are some of the UASG initiatives that Radix is part of:

• Participating in the Technical And Measurement working group
• Participating in workshops on Universal Acceptance such as:
  – India Internet Week 2020 Online Edition
  – FICCI – Bhashantara – Panel Discussions
• Sharing gaps and issues reported by customers to UASG

At Radix, our objective is to ensure that nTLDs are accepted across websites and platforms. To achieve this, we actively work with UASG and share as many issues and gaps noticed and reported by customers.

Contribution by other registries

A key objective for most registries is to ensure great customer experience when it comes to their nTLDs and I’ve always admired it when registry operators have actively taken initiative and participated in the five UASG groups mentioned above.

One of the ways to do this is to capture all the queries and complaints reported by their customers/registrar partners and share it with UASG. This will help their support team direct their resources in solving the problems and encouraging those websites to become UA compliant.

Contribution by registrars

When it comes to UA-related issues, registrars are the first in chain to receive a complaint or feedback from the user. Therefore, it’s crucial that their support teams have all the necessary information needed on how to best handle such complaints.

For now, they can:

• Inform the customer about the potential UA issue and raise a request on behalf of the customer with UASG. Issues can be logged at – https://uasg.tech/global-support-center/
• Report these instances to the Registry Operator so that they can connect and follow up with UASG.
• Join any of the five working groups and participate.

The path ahead

The UASG is consistently compiling and sharing all the important information needed for organizations and developers to become UA ready. This is not only about ensuring the readiness of a system to accept certain TLDs or emails, but also about realising the full potential of an organization by connecting with people and businesses that might not be even on it’s radar.

Every successful step taken by an organization towards UA readiness is also a step towards equality and inclusiveness on the internet.

Guest poster Aman Masjide leads compliance and abuse mitigation at Radix.

After 20 years, DomainTools takes its first VC dough

DomainTools has taken a “significant” investment from a venture capital firm, the first outside funding its received in its 20-year history.

The amount of the investment is undisclosed, but DomainTools said its investor is Battery Ventures.

Battery already owns stakes in numerous software and technology companies, but this appears to be its first foray into the domain name space.

Its principal, Jordan Welu, and partner Dave Tabors will join DomainTools’ board of directors and Andy Rothery, a Battery “executive-in-residence”, will become its executive chairman.

DomainTools said in a press release:

This investment will drive more rapid innovation in DomainTools’ platform capabilities for machine learning-based threat analytics and predictive risk scoring, along with enhanced product development around automating threat intelligence and incident response workflows.

The company is all about the “threat intelligence” nowadays, no doubt partly due to the fact that its original mission of aggregating the world’s Whois data will become decreasingly useful in light of privacy laws such as GDPR.

As a private company its financial position is unknown, but I’ll note that it did take a big chunk of change out of the US taxpayers’ pocket earlier this year under a government coronavirus-related corporate-relief program.

Coronavirus could cause “high risk of widespread outages”, ICANN says

There’s a “high risk of widespread outages” in the DNS if ICANN can’t get enough people in the same room for its next root DNSSEC ceremony because of the coronavirus pandemic.

That’s according to ICANN’s own board of directors, which yesterday published a contingency plan that — in the worst case scenario — could see parts of the internet come to a screeching halt in July.

The problem is with the elaborate “ceremonies” that ICANN and its IANA/PTI unit uses to make sure the internet can support DNSSEC — the secure version of the DNS protocol — all the way from the root servers down.

Every quarter, ICANN, Verisign and a select few “Trusted Community Representatives” from all over the world meet in person at one of two secure US-based facilities to generate the public Zone Signing Keys for the root.

In addition to the complex cryptographic stuff happening in the computers, there’s a shedload of physical security, such as retinal scans, PIN-based locks, and reinforced walls.

And the “secret key-holders”, memorably fictionalized in a US spy drama a few years ago, actually have physical keys that they must bring to these ceremonies.

The events are broadcast live and archived on YouTube, where they typically get anything from a few hundred to a few thousand views.

Obviously, with the key-holders dotted all over the globe and most under some form of coronavirus-related lockdown, getting a quorum into the same facility at the same time — originally, Culpeper, Virginia on April 23 — isn’t going to be possible.

So IANA has made the decision to instead move the ceremony to the facility in El Segundo, California, within easy driving distance of ICANN’s headquarters, and have it carried out almost entirely by ICANN staff, wrapped in personal protective equipment and keeping their distance from each other.

The TCRs for El Segundo live in Mauritius, Spain, Russia, Tanzania, Uruguay and on the east-coast of the US, according to ICANN.

Four of these key-holders have mailed their keys to different IANA staff “wrapped in opaque material” and sealed in “tamper-evident bags”. These IANA employees will stand in for the TCRs, who will be watching remotely to verify that nothing fishy is going on.

Verisign and the independent auditors will also be watching remotely.

That’s the current plan, anyway, and I’ve no reason to believe it won’t go ahead, but ICANN’s new contingency plans do provide four alternatives.

It’s already discarded the first two options, so if the current, third, plan for the ceremony can’t go ahead before June 19 for some reason, all that would be left is the nuclear option.

Option D: Suspend signing of the DNS root zone

This is the final option if there is no conceivable way to activate the KSK and perform signing operations. There would need to be a massive education campaign at short notice to have resolver operators disable DNSSEC validation. There is a high risk of widespread outages as it is not possible to ensure global implementation, and high risk this will fatally compromise trust in DNSSEC in general as a technology.

This is considered highly unlikely, but nonetheless the final option. Without exercising the option, in the absence of a successful key signing ceremony, DNSSEC validation would be unsuccessful starting in July 2020.

The reason for this scenario is that DNSSEC keys have a finite time-to-live and after that period expires they stop functioning, which means anyone validating DNSSEC on their network may well stop resolving the signed zones.

ICANN typically generates the keys one quarter in advance, so the current key expires at the start of July.

However, the planned April 23 ceremony will generate three quarters worth of keys in advance, so the root should be good until the end of March 2021, assuming everything goes according to plan.

Clearly, the idea that half the planet might be on the verge of lockdown wasn’t taken into consideration on February 12, the last ceremony, when ICANN’s biggest problem was that it couldn’t get into one of its safes.

If you’re interested in more about the ceremony and the coronavirus-related changes, info can be found here.

Go here to help fight against coronavirus abuse

A coalition of over 1,000 security experts, domain name providers and others have got together to help coordinate efforts to combat abusive coronavirus-related domains.

A workspace on the collaboration platform Slack has been growing steadily since it was created a week ago, enabling technology professionals to exchange information about the alarming number of sites currently trying to take advantage of the pandemic.

You can join the channel via this link. Thanks to Theo Geurts of RealtimeRegister.com for passing it along.

The collection of chat rooms appears to have been created by Joshua Saxe, chief scientist at security software firm Sophos, March 19. There are currently 1,104 members.

There’s a channel devoted to malicious domains, which is being used to share statistical data and lists of bad and good coronavirus-related domains, among other things.

Across the workspace, a broad cross-section of interested parties is represented. Current members appear to come from security companies, governments, law enforcement, registries, registrars, ICANN, healthcare providers, and others.

It seems like a pretty good way for the technical members of the domain name industry to keep track of what’s going on during the current crisis, potentially helping them to put a stop to threats using domains they manage as they emerge.