Rating Scale Best Practices
Posted by Jeffrey Henning on Fri, Aug 28, 2009
Most survey researchers pay little mind to the research into the efficacy of different scales, frequently using obsolete rating scales in their surveys: changing scales can be a real bugbear.
That said, here are some well-researched best practices when it comes to scales:
- Use 5-point scales for unipolar scales and 7-point scales for bipolar scales:
"To explore the relation between scale length and reliability, we conducted a meta-analysis of the results of many past studies. Our data consist of results from 706 tests of reliability taken from thirty different between-subject studies. We combined various measures of reliability and various sample sizes, controlling for these and other factors in determining the relation of scale length to reliability. In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five." - Jon Krosnick, professor of communication at Stanford, "The Optimal Length of Rating Scales to Maximize Reliability and Validity"
- Use fully labeled scales without showing respondents numeric ratings. Such scales are preferred by respondents and have higher reliability and predictive validity than numeric scales.
- Exclude “Don’t know” and “No opinion” as a choice when presenting your scale.
- The 0-to-10 rating scale for Net Promoter has the lowest reliability and predictive validity of four scales tested.
The above findings are backed up by scientific research. The following best practices, on the other hand, are my personal preferences, for which I was not able to find supporting data:
That said, factoring in the research and my recommendations, this is what I consider to be the best
CSAT scale:
What is your overall satisfaction with our company?
- Not at all satisfied
- Slightly satisfied
- Moderately satisfied
- Very satisfied
- Completely satisfied
When a study mixes different lengths of scales, consider
standardizing the scales in survey analysis, for instance by mapping scales to a 0 to 10 scale. This can make reports of the results easier to understand. While respondents dislike numeric scales, fully labeled scales are typically analyzed numerically, and the 0-to-10 mapping can aid analysis.
Most organizations fail to
standardize on rating scales, making it difficult to compare the results from study to study, from department to department. If you haven’t yet done so, please consider coming up with standard practices to guide your research. To contradict Emerson, when it comes to rating scales, a foolish
inconsistency is the hobgoblin of little minds.