August 25, 2011
Want A Good Survey? Think Balance: Love and Hate
“Balance plays an important part in yielding accurate results in a survey.”
So says the Floodlight blog. They’re correct, of course. They’re also assuming you want a balanced survey.
This is not an automatic assumption. Hidden within the methodology is the difference between a survey “showing” that 99.9 percent of all voters would rather cut off their right arms than not vote for Jack Frack McCracken.
“Without balance, a survey is inherently skewed in one direction or another and a waste of both the researchers’ time and the respondents’ time,” Floodlight says correctly. Of course it might be a tremendously beneficial investment for whoever would like people to think the survey’s accurate.
Floodlight notes that there is an instance where a survey could be not designed to yield accurate results but calculated to yield specific results: “Often, these questionnaires are used to support particular arguments.”
In other words, about 93 percent of the surveys you’ll see between now and Election Day 2012 will support particular arguments. And not just for the particular politicos themselves: “It’s vital for politicians to have data to back up their platforms,” Floodlight says. One thinks of the surveys showing commanding majorities of the American electorate pining for ObamaCare, but we digress.
And it’s not like it’s brain surgery or rocket science to skew surveys. “Getting the results they want is actually subtler and easier than you’d think. All it takes is a little imbalance in the questions, specifically those questions that include rating scales,” Floodlight says, giving the example of a survey asking “How much do you like the color blue?…A) It’s okay; B) I like it; C) I love it.”
In other words, there’s no option for “I hate it.” Not that anybody hates the color blue, of course, except Republicans on Election Night, but the principle holds up.
It’s called including “the appropriate extreme,” and it’s something you need to get an honest, accurate survey. If that’s what you’re interested in.
David Sims is a contributing editor for TMCnet. To read more of David’s articles, please visit his columnist page. He also blogs for TMCnet here.
Edited by Juliana Kenny