Amazon has reversed, at least temporarily, its decision to yank its free list of the world’s most popular domains, after an outcry from researchers.
The daily Alexa list, which contains the company’s estimate of the world’s top 1 million domains by traffic, suddenly disappeared late last week.
The list was popular with researchers in fields such as internet security. Because it was free, it was widely used.
DI PRO uses the list every day to estimate the relative popularity of top-level domains.
After deleting the list, Amazon directed users to its Amazon Web Services portal, which had started offering the same data priced at $0.0025 per URL.
That’s not cheap. The cost of obtaining same data suddenly leaped from nothing to $912,500 per year, or $2,500 per day.
That’s beyond the wallets, I suspect, of almost every Alexa user, especially the many domain name tools providers (including yours truly) that relied on the data to estimate domain popularity.
Even scaling back usage to the top 100,000 URLs would be prohibitively expensive for most researchers.
While Amazon is of course free to price its data at whatever it thinks it is worth, no notice was given that the file was to be deleted, scuppering without warning goodness knows how many ongoing projects.
Some users spoke out on Twitter.
The quiet death of the @Alexa_Support top million sites is a grievous blow to internet researchers everywhere. $2500 per pull now.
— April King (@aprilmpls) November 21, 2016
Removing the top 1M list is a HUGE mistake. It was extremely useful to assess the impact of new security vulnerabilities. 🙁 @Alexa_Support
— Benjamin Beurdouche (@beurdouche) November 22, 2016
@Alexa_Support I'm disappointed, but I hope you reconsider. The Top 1M list is a standard reference in research. It's simply irreplaceable.
— Santiago Zanella (@xEFFFFFFF) November 22, 2016
I spent most of yesterday figuring out how to quickly rejigger DI PRO to cope with the new regime, but it seems I may have been wasting my time.
After an outcry from fellow researchers, Amazon has restored the free list. It said on Twitter:
Thanks to customer feedback, the top 1M sites is temporarily available again. We’ll provide notice before updating the file in the future
— Alexa Support (@Alexa_Support) November 22, 2016
It seems clear that the key word here is “temporarily”, and that the the restoration of the file may primarily be designed to give researchers more time to seek alternatives or wrap up their research.
Oracle has signed a deal to buy DNS services provider Dyn for an undisclosed amount probably in the nine-figure range.
The software giant said it plans to integrate Dyn’s services into its existing cloud computing platform. For the moment, existing Dyn customers are unaffected.
Dyn provides distributed DNS resolution services mainly to the enterprise market, where it has about 3,500 customers.
But it also provides redundant DNS to some TLD registries, notably Uniregistry.
Knowing how ruthlessly opportunistic Oracle can be when it comes to M&A, I have to wonder how much impact the recent denial of service attack against Dyn had on the timing of the deal being signed.
Dyn customers including Twitter and Netflix found themselves inaccessible for millions of North American internet users a couple of weeks ago.
Customers that may have been reconsidering their DNS options following the downtime may feel more reassured now that Dyn is about to become part of a much larger company.
While the acquisition price was not disclosed, it’s certainly going to be in the hundreds of millions.
Just six months ago, Dyn received $50 million in venture capital, following on from a $38 million round in 2012.
Next year’s Domaining Europe conference will be held in Berlin, organizers announced today.
The three-day event is slated to start May 14, 2016, at the Steigenberger Hotel, covering the usual mix of sales, development and legal issues.
“This time we are going back to the roots,” organizer Dietmar Stefitz said in an email, “the majority of the panels will discuss about monetization and Market-places.”
This year’s Domaining Europe took place in the Netherlands, after taking place for a couple of years in Spain.
Full-price tickets will be €650 (currently about $705) but there’s an early-bird discount to €350 for anyone buying before December 15.
The conference is being managed in cooperation with ECO, the Germany internet industry association.
Former Famous Four Media VP of sales Richard Downs has launched a new consultancy business aimed at new gTLD registry operators.
The new company, GTLD Systems is offering a multitude of services but is mainly a way for smaller registries to outsource their sales and marketing operations.
Downs told DI an early success was a recent $400,000 deal, selling a few FFM premiums (in .review and .download) to a single end user. He says he has a pipeline that he hopes will bring his total sales to $1 million before the end of the year.
He said he’s sold over $3 million in premiums over the last few years at FFM.
Spain-based Downs said that he has three employees, one a Chinese-speaker, in three different western-European countries.
Among the services on offer are premium list creation and sales, registrar channel management, Chinese regulatory approval consulting, supplier negotiations and marketing consulting.
Downs was with FFM for about three years. Before that, he was in digital recruitment.
Registry operators are challenging an ICANN decision to force them to launch a new Whois-style service, saying it will cost them too much money.
The Registries Stakeholder Group has filed a Request for Reconsideration — a low-level appeal — of a decision asking them to launch RDAP services to complement their existing Whois.
RDAP, Registration Data Access Protocol, is being broadly touted as the successor to Whois.
It offers the same functionality — you can query who owns a domain — but the data returned is more uniformly structured. It also enables access control, so not every user would have access to every field.
The RySG now claims that ICANN is trying to sneak an obligation to implement RDAP into its registry agreements through a “backdoor” in the form of the new Consistent Labeling and Display Policy.
That policy, which originated in a formal, community-driven GNSO Policy Development Process, seeks to normalize Whois (or Registration Data Services, in its generic not protocol-specific wording) output to make it easier to machine-read.
It applies to all gTLDs except .com, .net and .jobs (which are “thin” registries) and would come into effect February 1 next year.
Registries appear happy to implement the CL&D policy, but not as currently written. It now contains, almost as an aside, this requirement:
The implementation of an RDAP service in accordance with the “RDAP Operational Profile for gTLD Registries and Registrars” is required for all gTLD registries in order to achieve consistent labeling and display.
The RySG argues in its RfR (pdf) that implementing RDAP was never part of the community-endorsed plan, and that it is not “commercially feasible” to do so right now.
The 2012 new gTLD Registry Agreement specifies that implementation of the protocol now known as RDAP be commercially feasible before it’s required. The RySG can’t even respond as to whether it’s feasible or not since no reasoning to that regard was provided in the notice to implement such services.
Furthermore, some of our members are on record stating that since the RDAP profile replicates the known deficiencies of WHOIS – which is currently being studied by a PDP WG – so it’s not commercially feasible to deploy it to mimic a flawed system.
The introduction of RDAP represents an additive requirement for Registries to operate a new (additive) service. As there are no provisions for the sunset of the legacy Whois service, it’s unclear how this additional requirement can be considered commercially feasible.
In other words, the registries think it could be too costly to deploy RDAP and Whois at the same time, especially given that RDAP is not finished yet.
It’s yet another case of domain companies accusing ICANN the organization of slipping in requirements without community support.
Whether the RfR will be successful is debatable. There’s only been a few Reconsideration requests that have been approved by the ICANN board in the history of the mechanism.
However, the board may be feeling especially diligent when it comes to look at this particular RfR, due to the spotlight that was recently shone on the Reconsideration process by an Independent Review Process panel, which determined that the board just rubber-stamped decisions written by house lawyers.