Successful Surveys FAQ - Part 2
Posted by Jeffrey Henning on Thu, Apr 15, 2010
As mentioned yesterday, we had a lot of good questions to last week's webinar,
The Seven Steps to Highly Successful Surveys. See
yesterday's post for answers to questions on the first three steps. Here is the Q&A for
the remaining four steps.
Order Questions Logically
Q: What's your thought of making all the questions mandatory before allowing user to submit?
A: Mandatory questions increase your abandonment rate. I use as few as possible.
Q: Is it a good idea to have "pattern" in the way questions are worded, or is it better to mix it up? E.g. "It was easy to do ------." Mixed with: "Doing ---------was easy."
A: The more consistent you can be, the easier it is for respondents and the higher quality results you will get.
Write Objective Questions
Q: Is it best to label a ratings question (disagree, somewhat disagree, neutral, etc.) or just label the end points (disagree and agree)?
A: Label every point of a rating scale for highest reliability and validity.
Q: Do you also recommend labeling each point on a 10 point scale?
A: I don't recommend 10-point scales for web surveys (see these rating scale best practices); 5-point scales are fine. 10 points don't add reliability. You can talk me into 0-10 for phone surveys, though.
Q: Any thoughts on the theory that we should not include a "middle" ground option in a survey such as "neutral". I heard that you should lean customers to answer in a positive fashion or negative?
A: Yes, for bipolar scales you will get greater reliability with a neutral rating, but try to avoid bipolar scales if you can.
Q: With "choice questions", is it always a good idea to add "other" as an answer, with an opportunity for the respondent to fill-in the "other" answer?
A: Yes, I always like to add an Other. Here are my best practices for closed-ended questions.
Q: Back to the slide about number of responses, is it better to have many choices or few?
A: Better to have too many than too few! See how short choice lists lead to wrong answers.
Q: I am new to surveys. Are there any "cheat sheets" for what to say and not say in a survey?
A: Not that I am aware of. I would encourage you to download my ebook, Survey Software Success, and read the chapter on writing survey questions.
Q: Can you give your opinion of Net Promoter Score?
A: I used to be a big fan of NPS but find it less successful the more I use it. Some posts I've written about it:
Shorten the Survey
Q: Is there an optimal number of questions to ask?
A: The recommended survey length depends in part on the type of survey you are doing and who you are researching.
Q: What about the utilization of a progress/completion bar?
A: I always recommend it as a courtesy to respondents -- it will lower your response rate on long surveys, though.
Q: Is there any truth to adding a question that you may not 'need' per se, but is used to 'build logic' for the responder?
A: It is worthwhile adding a question that is entertaining or interesting near the end of the survey to keep respondents engaged.
Close the Feedback Loop
Q: How can you customize surveys so that you get actionable feedback but still keep it broad enough to be able to compare between departments?
A: Define a standard library of questions with the same wording and choices and use that across different surveys. Then have the other sections be tailored to the different departments.
Q: Just curious, what vehicle do you use to conduct virtual focus groups?
A: We are no longer doing virtual focus groups but are instead building market research online communities using our own platform, Vovici Community Builder.
Got additional questions? Feel free to comment below, contact me at @jhenning on Twitter or email me at jhenning at the vovici.com domain.