Welcome to the Listening Post!

The "Shoulds" & "Should Nots" of Question Wording

Tweet

 

At the 2011 AMA Applied Research Methods conference, Michael Zicha of Zicha & Associates shared the “5 Shoulds” and “11 Should Nots” of wording questions, taken from the text Marketing Research by Burns and Bush.

The Five “Shoulds” of Question Wording

1. The question should be focused on a single issue or topic. Make sure to set the context appropriately for a question.
2. The question should be interpreted the same way by all respondents. When terms can be interpreted differently, use more precise words instead (for instance, instead of “drink” use “alcoholic beverage”).
3. The question should use the respondent’s core vocabulary. Avoid trade jargon that will be unfamiliar to some respondents, as it will introduce a source of error.
4. The question should be a grammatically simple sentence if possible. Longer, more complex sentences seem more formal but often hinder a reader’s ability to understand the question.
5. The question should be brief. Eschew needless words, where possible when not violating the other “shoulds”.

The Eleven “Should Nots” of Question Wording

1. The question should not assume criteria that are not obvious. For instance, a common mistake is to say “How important is it that…” when what is intended is “How important is it to you that…”
2. The question should not be beyond the respondent’s ability or experience. Questions should be about the respondent’s activities and behavior rather than those of others in dissimilar circumstances.
3. The question should not use a specific example to represent a general case. “Did you see any advertising for Sears in the last week, such as newspaper inserts?” inadvertently reframes the question to focus on the example. Instead, omit examples or provide a comprehensive list of examples: “Did you see any newspaper, television, radio, online or mailed advertising for Sears in the last week?”
4. The question should not request the respondent to recall specifics when only generalities will be remembered. Questionnaires about service experience often include lists of specific employee behaviors that respondents may no longer recall: “Did the hotel clerk call you by name? Did the clerk describe the complimentary breakfast? Did the clerk invite you to join the rewards program?”
5. The question should not require the respondent to guess a generalization. Rather than say “If you bought a digital camera at retail, would you ask the clerk about its warranty?” instead say “…how likely would you be to ask the clerk about its warranty?”
6. The question should not ask for details that cannot be related. “How many brands of bread did you compare before selecting the loaf you purchased?” is too specific; instead ask, “Did you compare any brands of bread before selecting the loaf you purchased?”
7. The question should not contain words that overstate the condition. Avoid overstatement such as used in this example: “How much do you think you would pay for a pair of sunglasses that will protect your eyes from the sun's harmful ultraviolet rays, which are known to cause blindness?”
8. The question should not have ambiguous wording. The question “When your puppy misbehaves, do you discipline it?” assumes respondents will have a shared understanding of “misbehaves” and “disciplines”, when in practice each might describe quite different ranges of activities.
9. The question should not be “double-barreled”. The question “How easy was it to reach someone to help?” conflates two separate ideas – ease of reaching someone, and that the representative was helpful. Ask each separately.
10. The question should not lead the respondent to a particular answer. “Don’t you see some dangers in the new policy?” is the type of leading question a political push poll would ask.
11. The question should not have “loaded” wording or phrasing. Loaded questions are more subtle than leading questions: “Should people be allowed to protect themselves from harm by using Mace as self-defense?” is a loaded question; “Should people be allowed to carry Mace?” is more neutral.

As Mike summarized, writing good questions is hard, and questionnaire design is “more art than science”, being a skill acquired mainly through experience. Always try to understand potential respondents’ cognitive processes, point of view and language.