Welcome to the Listening Post!

Question Library as an Embodiment of Best Practices

Tweet

 

It's far easier to write surveys the way you always have rather than writing them the best way that you can today, given the most recent research on research. In fact, when I train business researchers, I often get pushback on the academic research that I am presenting. "That's not how we do it." "We've always done it this way." "That won't work for us." Ironically, for people devoted to data-driven decisions, many researchers rely on their gut when structuring surveys rather than on researched best practices.

Since each researcher has his or her own preferences, organizations often field surveys using very different ways of asking for the same information. When we reviewed 29 customer-satisfaction questions used by one of our partners, we identified eight different scales in use with nine different ways of wording the question. And that was just for one kind of question.

This uncomfortable fact was brought home to me last week by Ray Poynter's description of the recent Australian Market and Social Research Society's summer conference:

Pete Cape, of Survey Sampling, gave a good presentation on the way that online questions should be constructed with some really good examples and sound advice. For example, Pete suggested picking an academic, such as Jon Krosnick, and simply going with what they suggested, rather than letting every researcher in an organisation create every scale from scratch and according to their own preferences or ignorance. Pete was not specifically endorsing Krosnick over other relevant academics, but simply asserting that a company should adopt a house style and that the house style should be based on some formal reasoning.

This is excellent advice. Gather your researchers together and have them review their current methods. You will enable a lively debate, and your researchers will learn from one another. Ask them to back up their preferences with best practices rather than past practices, with well researched methods rather than long cherished habits. The output of their debate and eventual consensus should be a common question library to be used throughout the organization. (Most enterprise feedback management platforms, including ours, support a shared question library.) 

Here are some of the business benefits of using shared best practices:

• Your questions will have higher accuracy and reliability.
• Surveys will be less confusing for respondents when you use common rating scales.
• Consistency will make it easier to use questions for internal benchmarking across studies.
• Standard questions will enable you to make connections across studies that you never could have before.
• Business users writing their own surveys will have proven questions to choose from, rather than trying to write their own.

As researchers, we need to leave our comfort zone, to realize that some of the ways we conduct surveys today are outdated and suboptimal. Let go of yesterday's bad habits and construct shared guidelines for tomorrow.