ICANN has revealed details of a security problem on its web site that could have allowed new gTLD registries to view data belonging to their competitors.
The bug affected its Global Domains Division customer relationship management portal, which registries use to communicate with ICANN on issues related to delegation and launch.
ICANN took GDD down for three days, from when it was reported February 27 until last night, while it closed the hole.
The vulnerability would have enabled authenticated users to see information from other users’ accounts.
ICANN tells me the issue was caused because it had misconfigured some third-party software — I’m guessing the Salesforce.com platform upon which GDD runs.
A spokesperson said that the bug was reported by a user.
No third parties would have been able to exploit it, but ICANN has been coy about whether any it believes any registries used the bug to access their competitors’ accounts.
ICANN has ‘fessed up to about half a dozen crippling security problems in its systems since the launch of the new gTLD program.
Just in the last year, several systems have seen downtime due to vulnerabilities or attacks.
A similar kind of privilege escalation bug took down the Centralized Zone Data Service last April.
The RADAR service for registrars was offline for two weeks after being hacked last May.
A phishing attack against ICANN staff in December enabled hackers to view information not normally available to the public.
Go Daddy has rushed out a fix to a security bug in its web site that could have allowed attackers to steal valuable domain names.
Security engineer Dylan Saccomanni found several “cross site request forgery” holes January 17, which he said could be used to “edit nameservers, change auto-renew settings and edit the zone file entirely”.
He reported it to Go Daddy (evidently with some difficulty) and blogged it up, with attack code samples, January 18. Go Daddy reportedly patched its site the following day.
A CSRF vulnerability is where a web site fails to adequately validate data submitted via HTTP POST. Basically, in this case Go Daddy apparently wasn’t checking whether commands to edit name servers, for example, were being submitted via the correct web site.
Mitigating the risk substantially, attackers would have to trick the would-be victim domain owner into filling out a web form on a different site, while they were simultaneously logged into their Go Daddy accounts, in order to exploit the vulnerability, however.
In my experience, Go Daddy times out logged-in sessions after a period, reducing the potential attack window.
Being phishing-aware would also reduce your chance of being a victim.
I’m not aware of any reports of domains being lost to this attack.
It just keeps getting worse.
ICANN’s TLD Application System security bug could have revealed file names belonging to 105 new gTLD applicants to 50 other applicants on 451 occasions, according to the organization.
With 1,268 applicants in the system, those numbers certainly fit with the “a minority of applicants” description previously given, but it still shows that the bug was widespread.
The supplied numbers are “approximate”, but ICANN said it is “continuing to review system logs and packet-level traffic to confirm how many viewings actually did occur.”
The latest news means, for example, that 50 new gTLD applicants may have had the ability to see information belonging to other applicants on average nine times each.
While the new data may not strongly suggest that the bug was deliberately exploited by any applicant(s), it’s not inconsistent with that scenario.
It could mean that one applicant saw the details of 56 others (suggesting exploitation), but it could also mean that 50 applicants saw about two third-party file names each (suggesting accidental viewing).
Without further information, it’s impossible to know.
ICANN has not revealed, and is unlikely to reveal in the short term, whether any applicant was able to view the metadata of another applicant for the same gTLD.
The organization has however started to notify affected applicants whether they were affected as victim or beneficiary, according to the latest update from chief operating officer Akram Atallah.
Atallah also revealed that TAS had 95,000 file attachments in the system when it was taken down April 12.
At an average of 75 files per TAS account, this would support the idea that, on average, each TAS account was being used to file more than one application.
ICANN still plans to wrap up the notification process before next Tuesday, May 8, but there’s no word yet on when TAS will reopen for the final five days of the application window.
ICANN may not reveal its list of new generic top-level domain applications until as late as the last week of June, according to CEO Rod Beckstrom.
In his first interview since ICANN took its TLD Application System offline due to a security bug, Beckstrom told DI that he “hopes” to host the Big Reveal before he steps down as ICANN’s CEO.
He said he expects to have the new gTLD program back on track before he hands the reins to the organization over to his successor at the end of the ICANN 44 meeting in Prague, June 29:
I’d like to see us obviously get the technical issues resolved, notify applicants, reopen the window and publish the strings before I pass the baton in Prague. That’s not a commitment at this point in time, it’s an indication as CEO that it’s absolutely my intention to push for a timely resolution of this issue… If we can get things done sooner, then the sooner the better.
That’s two months away, a full month later than anyone was expecting.
But Beckstrom would not commit even to the Prague date. He said:
That’s my hope as a CEO, to get these issues resolved by that time-frame and have the string reveal in that time-frame. I haven’t committed the organization, I’m indicating to you volitionally my desire as CEO and the person who’s running the organization.
He framed the issue as a blip on a nine-year process (six years of policy development, one year of outreach and application filing, and up to two years of evaluation). He said:
In the context of nine-year program, a delay of between here and Prague of a few months is undesirable, it’s not what we want to have happen, but the quality of this program is more important to everyone involved than the specific date and time. We’re all focused on quality here and not just doing things in a hurry. This program is too important.
He said he is “sympathetic” to applicants that are burning through start-up funding waiting for ICANN to sort this out, but he noted that the same concerns have been raised over the years whenever the program has previously missed a launch deadline.
We know that some parties have been very patient and we know it’s got to be frustrating right now to see any delay in the program. At the same time, I’m sure that those parties are very concerned that this be done well and that the program be reopened and administered successfully.
Beckstrom reaffirmed ICANN’s promise to notify all applicants whether or not they were affected by the TAS bug – which revealed user names and file names to other TAS users – by May 8.
But TAS will not, it seems, reopen immediately after the notifications have been sent. As well as the log audit, ICANN is also working on performance upgrades.
While Beckstrom confirmed that the plan is to open TAS for five business days, to give applicants a chance to finish uploading their applications and confirm that their data has not been corrupted, he would not say when this window is due to open.
We’re going to share more precise dates when we have them. What I can tell you precisely right now is that the key thing we’re working on is combing through this large data set we have so that the parties that were affected are notified within the seven days. When we have clarity on the next milestone in the process we’ll communicate that openly.
We’re still doing system testing, we’re still looking at some of the performance issues. We have a whole set of things to do and feel comfortable that we’re ready and have full internal sign off. We’ll notify you and other parties when we have that clarity. Right now we have the clarity that we’re going to get the notification done in seven days – that’s the key dating item at this time.
We have very strong reason to believe we understand the bug and we’ve fixed the bug, but every day that we continue to test we gain a higher level of confidence in the system that this specific issue will not reappear.
While the first report of the bug was received March 19, it was not until April 12 that ICANN managed to “connect the dots” and figure out that the problem was serious and recurring, Beckstrom said.
ICANN saw the bug show up again repeatedly on April 12, as many TAS users logged in to finish off their applications, which was why it chose to take the system down with just 12 hours to go before the filing deadline.
ICANN is currently analyzing a 500GB log containing a record of every data packet that went into and out of the TAS between January 12 and April 12, to reconstruct every user session and determine who could see what and when, Beckstrom said.
He refused to comment on whether this analysis has revealed any attempts by TAS users to deliberately exploit the bug for competitive intelligence on other applicants.
He also declined to comment on whether ICANN has discovered instances of data leakage between two applicants for the same gTLD string.
The full packet capture system was introduced following a third-party security audit of the system conducted late last year, he said.
That audit, of course, did not reveal the data leakage vulnerability that continues to delayed the program.
When I put it to him that this is precisely the kind of problem ICANN wanted to avoid, due to the confidentiality of the applications, Beckstrom played down the seriousness of the bug.
Let’s be clear here: some user names and file names were visible, not the contents of applications and not the contents of those files. I think that if that had occurred it would be an even more undesirable situation and we have no indication that that occurred.
I wouldn’t call this a security issue, I’d call this… every major software system we use has bugs in it or bugs that are discovered over time. Whether that’s our operating systems or desktop applications or specific applications, you conduct the best tests you can. You assemble a testing suite, you assemble testers, you take various methods, but there’s never a guarantee that software is bug-free. The issue is that if and when bugs are encountered you deal with them appropriately, and that’s what we’re doing right now.
But Beckstrom admitted that the problem is embarrassing for ICANN, adding that sorting out the mess is currently the top priority.
Obviously any time you have a software problem or technical problem with any program you come under enhanced scrutiny and criticism, and I think that’s understandable, that’s fair. What we’re focused on is resolving this successfully and I think ICANN has dealt with many challenges in its past successfully and we’re committed to resolve this issue professionally.
I should tell you that this is our top priority right now internally right now. The resolution of this issue is our number one priority, the number one issue for me as CEO, number one for most members of the executive management team and for a large part of the organization. We’re extremely focused on this.
ICANN plans to reveal how many applicants were affected by the bug at the same time as it notifies applicants, Beckstrom said. It will not publish information about who could see what, he said.
Unfortunately for applicants, it seems they will have to wait well into next week before they have any more clarity on the timetable for TAS coming back online and the application window finally closing.
With Prague now emerged as a potential deadline for the reveal, the delays could in fact be much worse than anyone was expecting.
DI PRO subscribers can read a full transcript of the 30-minute interview.
ICANN has announced yet another delay in its new generic top-level domains program.
Last night’s much-anticipated update on its efforts to deal with the fallout of the TLD Application System security bug merely deferred resolution of the problem for a week. Again.
The whole program is now essentially a month behind schedule.
Chief operating office Akram Atallah said in a statement:
ICANN will notify all applicants within the next seven business days whether our analysis shows they were affected by the technical glitch in the TLD application system.
Shortly after the notification process has been completed, we will announce the schedule for reopening the application system and completing the application period. We are mindful of the need to allow sufficient time during the reopening period for applicants to confirm the completeness of their submissions.
The seven business days for applicant notifications takes us to May 8.
It’s not clear whether TAS would reopen immediately after this, but I suspect we’re probably looking at a buffer of at least a day or two between the end of notifications and TAS coming back online.
ICANN has previously said that TAS will be open for five business days, to enable applicants to finish off their applications. This brings us to, at the very earliest, May 15.
The Big Reveal of the list of applications, I estimate, will come one to two weeks after that.
We’re essentially looking at a late May or early June finish to a process that should have ended in late April.
As a result, the entire timetable for evaluating, approving and delegating new gTLDs will likely also be pushed out by a month.
For applicants, the anticipated November 12 date for the completion of the first-batch Initial Evaluation phase is now likely to come some time in mid-December instead.
Unhelpfully, the deadlines for filing objections and requesting Extended Evaluation for first-batch applicants is now likely to fall around about January 1, 2013.
That’s assuming we do not see any more delays, of course, which I think would be optimistic.