Welcome to the Listening Post!

Analyzing Customer Satisfaction Surveys

Tweet

 

1. "How am I doing?" Measurements capture attributes as numbers, which are easier to manipulate and which facilitate objective inferences; however, such measurements are only meaningful if the manipulations "preserve truth". We're making customers recall experiences, translate the question we ask and then pick the most appropriate choice from our rating scale. "All those steps only work if we preserve truth." Back up your interpretation of measures by analyzing verbatim comments. For instance, use sentiment analysis on your verbatim responses and crosstabulate against your rating scale, to ensure that you have the correct interpretation of the scale; answers of "somewhat satisfied" are "nearly met" are typically accompanied by negative comments, indicating these are negative ratings, not positive or neutral ratings.
2. "What are the major themes in customers' opinions?" Questions in a specific survey may be redundant and overlap with one another, getting in the way of higher level analysis. Use factor analysis to correct for poorly written questionnaires and to identify a small number of questions that explain most of the variation in the responses. For instance, separate ratings for "representative's knowledge", "representative's courtesy" and "representative's efficiency" were shown by factor analysis to measure a common underlying factor (which could be thought of as "representative's professionalism"). Factor analysis can also show that an attribute is "double barreled"; for instance, "easy to reach someone to help" is a nice example of a double-barreled question without the word "and" - factor analysis showed that it measured the representative ("someone to help", who might not have been the first person to answer the phone) as well as timeliness.
3. "What are my company's priorities?" Susan has observed four stages of an organization's sophistication when it comes to understanding priorities: stage 1-areas of poor performance, stage 2-important areas of poor performance, stage 3-important areas of poor performance that present significant opportunities to improve customer loyalty, ad stage 4-areas of competitive disadvantage. Common modeling approaches to determine priorities include multiple regression, structural equations, logistic regression and classification trees. Good models enable the analyst to build simulators that can be used to estimate how improving specific ratings will improve overall customer satisfaction or loyalty.
4. "Who is my chief competitor and how do I compare?" Conducting a relationship-style survey of your customers and your competitors' customers provides important measures of relative performance. If you use the CVA (Customer Value Added) model, you can evaluate if your company is a better or worse value (quality for price) than competitors. CVA also illustrates companies that aren't competitive with you--downmarket and upmarket brands which offer fair value but outside the market you serve. Going beyond CVA, you can use the same survey to provide a detailed a comparison of the relative performance of your company on a range of attributes.
5. "What process threshold triggers dissatisfaction or delight?" Survey data is not the only measure that businesses use to track performance. Understanding internal measures and linking them to overall satisfaction is important for many reasons. It's easy for staff to dismiss survey results, for reasons of sample size or question design, in favor of internal metrics they are comfortable with, such as call hold times or repeats (subsequent requests for service). By quantifying the effect an internal measure has on overall loyalty, you demonstrate the validity of both measures, and you present staff with a way of understanding and influencing overall satisfaction.
6. "How do I set credible objectives?" Customer satisfaction objectives are driving compensation plans at more and more large organizations. As researchers, we need to provide input to this process, helping the organization set objectives that support a strong competitive position and offer guidance to the factors with the greatest impact on overall satisfaction, while assigning accountability fairly. The CVA research can be used to benchmark "best in class" performance, and simulators that link performance indicators to satisfaction can be used to help identify objectives. Make sure that measurements of progress towards the objective account for the larger sampling error that comes from smaller sampling sizes for departments and regions. Help the organization set an achievable goal; an unachievable goal will simply be ignored.