Long Surveys Turn Respondents into Liars
Posted by Jeffrey Henning on Tue, May 05, 2009
Add to the list of ways to corrupt people with surveys - offering incentives to respondents and bonuses to employees - boring people to death with long questionnaires. Actually, you just bore them to the point where they pay less and less attention to the quality of their answers, until finally they're cheating.
When respondents first begin the questionnaire, they're trying to give you the most appropriate answer to each question (optimizing). A little ways into the survey and they are engaging in what Krosnick calls weak satisficing: selecting the first choice that appears reasonable and answering "Yes" to be agreeable. Further into the survey and they are strong satisficing: failing to differentiate between ratings and selecting "don't know" rather than giving an opinion. After subjecting them to many pages of a questionnaire you've lost them and their good will; if the questions are required, they are just randomly selecting responses to be done with the damn thing.
What can you do about this?
- Remind respondents in the introduction of the importance of accurate answers
- Keep the questionnaire length appropriate to the purpose of the survey
- Follow our six strategies for shortening questionnaires
- Rotate the order of sections of the questionnaire, so that no section suffers disproportionately from having respondents satisfice
- Minimize the use of required questions, so that the truly uninterested respondent can skip the question without fabricating an answer
- Keep the questions engaging and relevant.
What other tips can you think of to keep respondents engaged for the duration of a long questionnaire?