A couple of times this year, ICANN has supplemented its regular email public comment periods with quicker, easier point-and-click surveys.
I assume they were designed to more clearly communicate the issues and increase response rates. In some respects they seem to have been a success, in others, not so much.
First, the Win.
Of the two comment periods I’ve looked at – on the Post-Expiration Domain Name Recovery policy and ICANN’s public meetings strategy – response rates were definitely up on what you would normally expect to see.
The PEDNR report solicited 10 regular email comments and a whopping 412 survey responses. The meetings survey had three emailed comments and 297 survey responses.
Judging from the PEDNR summary and analysis report, which breaks down the actual responses, the majority of respondents had something of value to say for most of the questions posed.
(It also reveals that I was the first person to take the survey. Take that, Kirikos!)
The majority of responses came from Germanic-sounding names, so I’m guessing there was some outreach in a German-language forum somewhere. Due to this fact, some respondents confessed to not fully understanding some of the English-only questions.
Now, the Fail.
The survey conducted to gauge feeling on ICANN policy for deciding meeting locations has produced some incomprehensible data. Take this, for example:
Considerations in choosing a host city
246 voters replied:
591 or 40% felt that Facilities were most important
399 or 27% felt that Convenience of meeting location and cost for both ICANN and attendees most important
495 or 33% felt that Safety and Security were most important
So… 246 people voted, but 1,485 votes are tallied?
A few more questions had similar anomalies in their results, rendering the data pretty much worthless. It’s reproduced by ICANN regardless, and without comment.
I suspect the survey provider, BigPulse, may have some of the blame, although it can’t have helped that the ICANN survey itself was reportedly highly confusing.
The PEDNR survey was conducted using Zoomerang. I do not recall having any trouble using it.
On balance I’d say that, teething troubles aside, surveys make a nice complement to public comment periods, provided that, due to the risk of “framing”, they do not carry too much weight.