Welcome to the Listening Post!

Multi-Item Scale (Index) Construction in Survey Research

In the past week, I was asked to give advice to a prospect who wanted to create their own multi-item measure (also known as an index or latent variable). A multi-item or summated scale consists of a number of ratings combined into a single value. Traditionally, multi-item scales are used to represent complex psychological constructs that can't be summarized in a single question: attitudes, stress levels, personality, loyalty and satisfaction, among others.

Multi-item measures are more reliable and less volatile than single-item questions (see Customer Satisfaction - The customer experience through the customer's eyes). For instance, if a respondent misunderstands one part of the question, it will affect part of the measure rather than all of the measure. As a result of this increased stability, multi-item scales make excellent benchmarks:

1. The ACSI Score is a superb example of a multi-item satisfaction scale.
2. For those of our customers who use the Apostle Model, we advise them to use two indices: one for satisfaction and one for loyalty.
3. An ISP (Internet Service Provider) developed a Renewal Likelihood Index consisting of three questions designed to measure the probability of an individual customer renewing when their annual service contract was up. As the predictive value of this measure proved itself over time, all surveys to customers included these three questions and a low score would send an email trigger to alert a service representative.

A more affordable approach than developing your own the way that ISP did is to adopt a proven index used by others. I've had a copy of the first edition of Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behavior Research at hand for 15 years now for specifically this reason. While you could define for yourself a multi-item measure (latent variable) simply by summing or averaging the answers to two or more questions (manifest or observable variables) that you selected, it is unlikely that you would have hit upon a very effective measurement. According to Robert Peterson in Constructing Effective Questionnaires, a good index needs to be unidimensional, reliable, valid and generalizable:

• Unidimensional - A multi-item scale needs to measure only one attribute or behavior.
• Reliable - The scale needs to produce similar results over time, and its internal measures need to be consistent with one another.
• Valid - The scale needs to demonstrate that it actually measures what it is meant to measure: if it predicts that customers will recommend your brand, successive research should demonstrate that those customers do in fact recommend your brand.
• Generalizable - The scale should work in multiple collect modes (web, paper, telephone, face-to-face) and across the target population.

I point out these requirements primarily to discourage you from attempting to develop your own multi-scale measures: this is actually a rigorous science known as psychometrics. If your organization can benefit from a proprietary measure of satisfaction or loyalty, then you are best served by hiring a psychometrician to develop the measure for you.

And that was the advice I gave our prospect, who wanted to develop a metric of satisfaction and loyalty (which would not have been unidimensional, since satisfaction and loyalty are independent).