Survey Software, Web Survey, Online Surveys, and Enterprise Feedback Management solutions from Vovici

Voice of Vovici Blog
Survey Research & Enterprise Feedback Management

   Contact Us       Customer Login       Support    Blog  
 

Experience Vovici :: request a call
Experience Vovici :: request a call
Experience Vovici :: request a demo

Subscribe to our Blog


Your email address

VISION 2010 :: Vovici User Conference :: Save the Date :: May 10-12, 2010

Download your free copy our eBook: Survey Software Success

Free EBook!

We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys.

> Download your free copy

Solutions For:

Online Survey Solutions Voice of the Customer SolutionsMarket Research Solutions Customer Support Solutions Voice of the Employee Solutions Government Solutions

Subscribe to our blog

Your email:

Survey Research & Enterprise Feedback Management

Current Articles | RSS Feed RSS Feed

Rating Scale Comparison: Weighing Different Scales for Survey Research

 | Submit to Digg digg it | Submit to Reddit reddit | Add to delicious delicious | Submit to StumbleUpon StumbleUpon | Share on Facebook Facebook | Share on Twitter Twitter | Share on LinkedIn LinkedIn 

Rating Scale Comparison - Weighing Different Scales for Survey ResearchIf there is one thing I dislike, it is giving people advice about scales. As Brian Koma, our vice president of marketing puts it, "People will defend which scale they use like they're fighting a holy war."

So here's some advice on why the three-point scale, the five-point scale, the seven-point scale, the ten-point scale and the eleven-point scale are all the absolute best rating scale for you to use!

  • Three-item scale: "Using only three answer options takes up a lot less visual space and is far less daunting than five options.  Thus, we always try to limit the response options to three, as that number offers the robustness to capture what we need while still remaining visually inviting." - Ziggy Zubric, owner of Marketing Endeavors, "Less Really is More When it Comes to Response Scales"
  • Five-point scale & seven-point scale: "To explore the relation between scale length and reliability, we conducted a meta-analysis of the results of many past studies. Our data consist of results from 706 tests of reliability taken from thirty different between-subject studies. We combined various measures of reliability and various sample sizes, controlling for these and other factors in determining the relation of scale length to reliability. In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five." - Jon Krosnick, professor of communication at Stanford, "The Optimal Length of Rating Scales to Maximize Reliability and Validity"
  • Ten-point scale: "A five-point scale is totally inappropriate for customer satisfaction studies. Why? It lacks enough granularity and robs companies of a burning desire to take corrective action. It commonly leads executives to believe that ‘80% rate us four or five; that's great, let's move on,' without realizing that it simply means that 80% are at least somewhat satisfied. Further, many people will never rate anything a ‘five,' resulting in ‘four' including those who are really very satisfied and those who are only somewhat satisfied. To avoid this topping effect, use at least a 10-point scale and count nine and 10 ratings as fully satisfied. This will also allow easier analysis of what bottom-line effects satisfaction has, since such tools as regressions work better with a more granular score." - Brad Bortner, principal analyst with Forrester Research, "Best Practices: Why Customer Satisfaction Studies Fail"
  • Eleven-point scale: "The 0-to-10 scale has many significant advantages: Customers find that the scale makes intuitive sense..., most of the world already uses the metric system..., customers may refuse to give anybody a perfect score..., customers will transpose the top and bottom on a 1-to-10 scale..., scales with fewer points seem more susceptible to grade inflation..., the 0-to-10 standard is being adopted by many of the world's leading companies." - Fred Reichheld, Fellow with Bain & Company, The Ultimate Question, p. 98-99

What's your favorite size for a rating scale, and why do you prefer it?

Comments

Appropriate scale length depends on the extent respondents can differentiate different attribute levels in the stimuli they are rating. So: for concepts tests in which respondents are being exposed to something new, a 5- or 7-point scale will be best. For a customer satisfaction study, where respondents are rating products with which they are familiar, then an 11-point scale (0 to 10) is best. In all cases, use a balanced scale. If you go to the “bible” of psychometric theory—Jum Nunnally’s Psychometric Theory—scales measure the amount of attributes. Consequently, they have to have a midpoint.
Posted @ Wednesday, June 17, 2009 8:59 AM by Terry Grapentine
I am not the greatest writer but I do alot of Marketing research. I recently had to convince "management" about switching from a 5 to 7 point scale. I found an simple article from Quirks and though it would be Relevent to this discussion. 
 
basically there is no real conclusive evidence to suggest a one scale system. the question know becomes unipolar or bipolar- It seems Maritz uses the unipolar more 
 
 
 
Balanced scales have equal numbers of positive and negative points: completely satisfied; mostly satisfied; mostly dissatisfied; completely dissatisfied. Although there are exceptions, a best practice with respect to balanced scales is that the use of modifiers should be symmetrical on the positive and negative ends of the scale. 
 
 
 
Unbalanced scales attempt to get greater discrimination on one side of the scale than on the other. If past experience suggested that most respondents are satisfied with a certain product or service, a researcher might want to “stretch out” the positive side of the scale, as in this extreme example (five positives and only two negatives): completely satisfied; very satisfied; mostly satisfied; somewhat satisfied; barely satisfied; mostly dissatisfied; completely dissatisfied. 
 
 
 
Whether a scale should be balanced or unbalanced usually depends on whether we’re measuring a unipolar or a bipolar concept of satisfaction. A unipolar satisfaction scale might range from “not satisfied” to “completely satisfied” (i.e., it doesn’t measure any more extreme dissatisfaction at all). In contrast, a bipolar scale would range from “completely dissatisfied” to “completely satisfied” (i.e., it measures extremes of both satisfaction and dissatisfaction). 
 
 
 
When we don’t know ahead of time whether most respondents will tend to be satisfied or dissatisfied, or if we expect high levels of dissatisfaction, a balanced bipolar scale is appropriate. If we know from past experience to expect low levels of dissatisfaction, an unbalanced unipolar scale will be better. In our firm’s experience, most studies show low levels of dissatisfaction, so our most frequent scale recommendation is for an unbalanced satisfaction scale. 
 
 
 
Some people think that satisfaction and dissatisfaction should be measured separately and are different entities. Although Maritz has observed that satisfaction and dissatisfaction may sometimes have different drivers, and there may be nonlinear relationships between satisfaction and other variables, we have seen no compelling reason to measure them separately. 
 
 
 
C. Midpoints 
 
 
 
A midpoint communicates neutrality on a balanced scale. “Neither satisfied nor dissatisfied” serves as the midpoint in this balanced bipolar scale: completely satisfied; mostly satisfied; neither satisfied nor dissatisfied; mostly dissatisfied; completely dissatisfied. 
 
 
 
There seems to be no difference in quality between scales that have a midpoint and those that do not. For bipolar scales, however, Maritz Research follows the advice of Sudman and Bradburn (1982) to “include the middle category unless there are persuasive reasons not to do so.” Krosnick (in press) suggests the same. 
 
 
 
D. Number of scale points 
 
 
 
Despite the strong opinions of some scale enthusiasts, there just isn’t powerful empirical evidence that a single number of scale points are always the best. A literature review by Cox (1980) summarizes the consensus regarding self-administered surveys: “Seven, plus or minus two, appears to be a reasonable range for the optimal number of response alternatives.” Internal research has not indicated a consistent difference in quality between 5- and 10-point scales. Krosnick (in press) finds that 5-point scales have the greatest test-retest reliability in paper-and-pencil surveys. Maritz’ experience with mail studies has shown that 5-point scales provide better dispersion of responses (and are, therefore, more discriminating) than other scales. We expect these results would generalize to other visual survey modes (e.g., PC or Web-based). 
 
 
 
The findings about the superiority of 5-point scales apply specifically to visually perceived scales and may or may not generalize to questions that respondents hear rather than see. Ten-point response scales are common in telephone surveys and have also been found to provide some advantages: greater statistical precision, statistical power, and room for improvement (Wittink and Bayer, 1994). We have also seen evidence that respondents from some ethnic groups shy away from using the extreme endpoints and may answer more accurately if they have a greater number of scale points from which to choose. Hence, while our basic recommendation is to use 5-point scales, we are comfortable using 10-point scales for some surveys, particularly those conducted via telephone. 
 
 
 
Posted @ Wednesday, July 08, 2009 3:29 PM by Stephen
For a seven point bi-polar scale is there consistent way to label the points?  
 
 
 
Extremely Satisfied 
 
Satisfied 
 
Somewhat Satisfied 
 
Neutral 
 
Somewhat Dissatisfied 
 
Dissatisfied 
 
Extremely Dissatisfied
Posted @ Monday, August 17, 2009 9:54 AM by Bob Klaus
That certainly works, though I prefer to start survey scales with the negative choice first. The 7-point satisfaction scale I use: 
- Completely dissatisfied 
- Mostly dissatisfied 
- Somewhat dissatisfied 
- Neither satisfied nor dissatisfied 
- Somewhat satisfied 
- Mostly satisfied 
-Completely satisfied 
You can label the midpoint “Neutral” for succinctness, if you prefer. 
Posted @ Monday, August 17, 2009 10:07 AM by Jeffrey Henning
Question: I am a graduate student conducting a survey to determine worksite safety culture from a construction workers perspective. I intend to use scaled questions in my survey. Example questions are the following: 
 
 
 
Please rate your level of agreement to the following statement: 
 
1. My coworkers support the company's safety program 
 
2. Employees correct other employees unsafe acts 
 
 
 
I considered doing a 5 point scale to measure agreement, but then considered using a 4 point balanced scale. My thinking is that either workers support the safety program or they don't. There is no "neither." As my sister would say, "Either you are pregnant, or you are not." 
 
 
 
I would appreciate any advice regarding this.
Posted @ Wednesday, October 07, 2009 12:29 AM by Jared Runge
Typically agreement scales are bipolar (disagree to agree), and the best bipolar scales have seven items. I'd use: Completely disagree, Disagree, Somewhat disagree, Neither agree nor disagree, Somewhat agree, Agree, Completely agree 
 
Many respondents select "Neither agree nor disagree" (sometimes "Neutral" is substituted) as a "No opinion" measure.
Posted @ Friday, October 09, 2009 4:22 PM by Jeffrey Henning
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics

Receive email when someone replies.