"You're the Top! You're the Coliseum": Top-Box Scores vs. Means
Posted by Vovici Blog on Mon, Mar 07, 2011
You're the top!
You're the Coliseum.
You're the top!
You're the Louvre Museum….
I'm a worthless check, a total wreck, a flop,
But if, baby, I'm the bottom, you're the top!
- Cole Porter, “You’re the Top”
You have hundreds of responses to your customer-satisfaction survey, and you need to sum them up in a single number. Do you share the arithmetic mean (the average) or the top box (the percent of respondents who gave you the highest rating)?
The two numbers are quite different. Statisticians call the mean “a measure of central tendency,” as it describes a middle point of all responses; they call the top-box score “a measure of extremes,” as it describes the proportion of respondents who choose the highest point on a rating scale.
If the two measures were to serenade one another about which was better, the Arithmetic Mean would sing these praises of the Top Box:
- “You’re easy for managers to understand”
- “You correlate to the highest level of loyalty”
- “You better reflect the skew of satisfaction data”
- “You’re the king of all royalty”
And the Top Box, in its turn, would sing these praises of the Arithmetic Mean:
- “You’re unambiguous”
- “You include the entire distribution of responses”
- “You compare better across cultures”
- “You’re a chandelier in a world of sconces”
Now, backstage, the statistics would complain about one another. The Arithmetic Mean would have these mean things to say about Top Box:
- “You’re a pushover: you’re supposed to be the Top Box, not the Top 2 Box or the Top 3 Box (which have lower correlations to loyalty behaviors).”
- “You’re easily gamed: organizations coach customers to give the highest rating rather than to provide an honest assessment.”
- “You’re the bottom, baby, and I’m the top.”
And Top Box would try to top the Arithmetic Mean:
- “You’re abstract: what does moving from 3.7 to 3.8 actually mean?”
- “You’re shapeless: you don’t differentiate between mediocrity and extremes of satisfaction and dissatisfaction.”
- “You’re the middle, baby, and I’m the top.”
Ironically, I occasionally see firms flip from one to the other, as the weaknesses of their current favored metric become apparent—disregarding the weaknesses of the metric they’re moving to.
As Derek Allen and Tanniru Rao write in their book Analysis of Customer Satisfaction Data:
The choice of a univariate tracking metric is in reality not a statistical decision but all too often a political one. In the final analysis, organizations choose metrics with which they are comfortable and familiar… Nonetheless, it seems clear that in North America, the top box or top two boxes are overwhelming preferred.
Just like Ethel Merman and William Gaxton singing in Anything Goes, Top Box and Arithmetic Mean are better together. There’s no need for either of them to sing solo.