Welcome to the Listening Post!

Halo Error: Lose the Halo to Gain More Accurate Insights

Tweet

 

What makes cheesesteaks and Tastykakes® taste even better? At Lincoln Field in Philadelphia, Aramark stadium surveys found that fans like the food 11% more when the Eagles win. Victory tastes delicious.

As far back as 1920, survey researchers discovered that respondents fail to rate individual attributes in isolation, but let overall impressions affect their ratings. The halo effect is when satisfaction inflates ratings, and the reverse halo effect is when dissatisfaction lowers ratings. Halo error is a form of satisficing, of respondents going through the motions rather than carefully thinking through each answer.

Halo error can lead you to make bad decisions from the survey data you've gathered. Jochen Wirtz of the National University of Singapore, who has done extensive research into halo error, identifies three ways that respondents who give in to the halo effect can be false prophets:

• Halo error leads to confusion about the true strengths and weaknesses of products and services.
• Halo error makes benchmarking attributes across competing brands and products unreliable (brands are well documented as introducing halo effects).
• Halo error can lead to the misinterpretation of satisfaction attributes.

In "Halo in Customer Satisfaction Measures," Wirtz used a number of experiments to identify three ways of minimizing halo error.

1. Introduce the survey as being used for improvement rather than as a performance assessment of employees. Respondents don't want to be responsible for negative feedback about staff. "When the subjects believed that their input could help the service firm to improve and deliver better service in the future, they were more willing to participate in the evaluation, discriminate among the attributes, and rely less on general impressions.
• Do introduce a survey like this: "Your response will be used to help fast food restaurants to improve their service quality. It will also be used to develop better training programs for employees in fast food restaurants."
• Don't introduce a survey like this: "Your response will be used to help fast food restaurant managers monitor and evaluate their branch performance. It will also facilitate their task in identifying employees who have performed below par."
2. Have respondents rate more items rather than fewer. Wirtz tested two versions of the questionnaire, one with five attributes to be rated and one with ten. His findings replicated the results of earlier research: "Subjects expended more cognitive effort in order to discriminate between the attributes when there were more rather than less attributes, and thereby decreased halo caused by inadequate discrimination."
3. Contrast the ratings of highly involved customers with those of less involved customers. Wirtz measured how involved respondents were with the service category (fast-food restaurants) using the ten-item Revised Product Involvement Inventory. Respondents with above-average involvement showed much less halo error than respondents with below-average involvement. Your most involved customers (those who find your product to be important, relevant, interesting and fun) will be able to more clearly separate their overall impression of you from that of specific attributes.

Fortunately, halo error is not difficult to acknowledge and address when designing a questionnaire. It is certainly much easier than winning football games. I have to imagine, given the Eagles' trade of Donovan McNabb, that the snacks at Lincoln Field aren't going to taste quite as appetizing this year.