'No Opinion' as a Question Choice
Posted by Jeffrey Henning on Fri, Aug 07, 2009
When writing
closed-end questions should you include a choice for “Don’t know”, “Not applicable” or “No opinion”? The fear is that including one of these as an option will give respondents an easy way out (e.g.,
survey satisficing) rather than actually thinking through their best answer to the question.
According to many seasoned survey researchers, offering a no-opinion option should reduce the pressure to give substantive responses felt by respondents who have no true opinions. By contrast, the survey satisficing perspective suggests that no-opinion options may discourage some respondents from doing the cognitive work necessary to report the true opinions they do have. We address these arguments using data from nine experiments carried out in three household surveys. Attraction to no-opinion options was found to be greatest:
- among respondents lowest in cognitive skills (as measured by educational attainment),
- among respondents answering secretly instead of orally,
- for questions asked later in a survey,
- and among respondents who devoted little effort to the reporting process.
The quality of attitude reports obtained (as measured by over-time consistency and responsiveness to a question manipulation) was not compromised by the omission of no-opinion options. These results suggest that inclusion of no-opinion options in attitude measures may not enhance data quality and instead may preclude measurement of some meaningful opinions.
Use of no-opinion responses is greater for respondents “answering secretly instead of orally” - i.e., for respondents doing self-administered surveys such as paper, web and kiosk surveys, rather than responding to telephone or face-to-face surveys. The reason for this difference is that every respondent sees the no-opinion choice on a written survey, but in an oral survey that response is typically not read aloud to a respondent but is kept in reserve, checked off only if the respondent brings it up.
One way to approximate this in an online survey is to not include such a response in the choice list for a question, if an answer is not required. The respondent therefore doesn’t see the “Don’t know” option but, upon consideration of the available choices, can simply skip answering the question altogether. Accordingly, I prefer to only use no-opinion responses in choice lists only if the question is required, and only if the required question may be hard to answer for respondents: for instance, when asking about specific details about a past transaction or when asking for details about the respondent’s organization that they simply might not know.
When analyzing no-opinion responses it is often handy to omit such responses from pie charts and frequency percentages. Survey software applications may let you assign a code to a choice (e.g., in Vovici v4, you can assign a choice to a precoded meaning “Not applicable”, “Don’t know” or “Refused”) with such coded values then omitted from pie charts and the percentage column of frequency tables. Rather then say, “70% said yes, 20% said no and 10% didn’t know” with this option you can say “78% who knew said yes, 22% said no.”
I hope this gives you enough information to now have an opinion on the use of no-opinion responses!