Free EBook!
 We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys. Download your free copy here.
|
Survey Research & Enterprise Feedback Management
|
 RSS Feed
Posted by Jeffrey Henning on Wed, Oct 21, 2009
Jeremy Whyte, director of customer feedback and reporting with Oracle Corporation, presented details of Oracle's extensive Voice of the Customer research program in a Vovici research webinar yesterday. Oracle has achieved five years of year-over-year growth in customer satisfaction, even as revenues have grown from under $14 billion in 2005 to over $23 billion as 53 acquisitions have been merged into the Oracle product line. To evaluate and improve customer satisfaction, Oracle has developed a linkage analysis that ties operational measures, transactional satisfaction, customer satisfaction/loyalty and financial outcomes into a coherent model that can be used for ROI analysis:
Operational measures often have a direct impact on transactional satisfaction. For instance, Oracle identified a clear negative correlation between the total time required to resolve a service request and the overall satisfaction with that service request: the longer it took to close the ticket, the less satisfied the customer was.
Transactional satisfaction in turn impacts customer satisfaction. Customers who were more satisfied with service requests were, according to the relationship survey, more satisfied overall with support services and product effectiveness. Such customers also reported higher value received from Oracle and greater loyalty.
Customer satisfaction and loyalty in their turn impact financial outcomes. Oracle was able to identify how much additional revenue for licenses and maintenance renewals would result from increasing the Executive Customer Loyalty Index by one rating point.
Financial outcome modeling lets you link customer behavior to the bottom line, validating the benefits of customer loyalty. Properly refined, the model can be used to predict the ROI of improvements, helping prioritizing those initiatives and guiding resource allocation. For instance, determining how much can be invested in improving response time based on the ultimate financial outcome deriving from greater transactional satisfaction.
For a company as large as Oracle, this is not a one-size-fits-all linkage model. Segmentation is important, as the model varies by customer segment and by product line. Segmented linkage analyses have increased the sponsorship and engagement of senior leaders with the Voice of the Customer research and has proven itself year in, year out, as customer satisfaction with Oracle has steadily increased.
Posted by Jeffrey Henning on Mon, Sep 28, 2009
 In my recent post, Your Half-Human, Half-Vulcan Customers, I discussed affective commitment (the emotional pleasure your customer takes in doing business with you: the Human half) and calculative commitment (the cold-blooded evaluation that governs why your customer has a business relationship with you: the Vulcan half). After reading that post, the logical question for you to ask is how to measure your customers' affective and calculative commitment.
My past review of the literature hadn't shown much in the way of consensus, though I did find two papers using the following measures:
On a scale of 1 to 10, where 1 means "strongly disagree" and 10 means "strongly agree", please indicate your level of agreement or disagreement with each of the following about your telecommunications provider.
[Affective Commitment]
1. I take pleasure in being a customer of the company.
2. The company is the operator that takes the best care of their customers.
3. There is a presence of reciprocity in my relationship with the company.
4. I have feelings of trust toward the company.
[Calculative Commitment]
5. It pays off economically to be a customer of the company.
6. I would suffer economically if the relationship were broken.
7. The company has location advantages versus other companies.
Source: Anders Gustafsson, Michael D. Johnson, & Inger Roos, "The Effects of Customer Satisfaction, Relationship Commitment Dimensions, and Triggers on Customer Retention".
Sadly, there's a lot to dislike about this measurement, as it doesn't reflect modern rating scale best practices: it uses a 10-point bipolar scale rather than a 7-point bipolar scale, it uses numbers rather than labels, it's a Likert scale and subject to acquiescence response bias, and it has some complex items (e.g., "There is a presence of reciprocity in my relationship with the company").
Despite those weaknesses, I have used it with minor modification in some of my research (changing #2 to "[vendor] is the company that takes the best care of their customers" and changing #7 to "The company has advantages versus other companies"). I'm continuing to experiment: for B2B and more expensive consumer goods (e.g., appliances and cars), calculative commitment is an important predictor of loyalty (i.e., repurchase likelihood); for inexpensive consumer goods, affective commitment is more important for its impact on loyalty.
If your customers are "Human" (driven by emotion), consider adding the affective commitment index to your research. If your customers are "Vulcan" (driven by logic), consider adding the calculative commitment index. And if your customers are half-Human, half-Vulcan, use both measures. And never let them try to pinch you on the neck!
Posted by Jeffrey Henning on Thu, Sep 03, 2009
Inspired by the folk Apostle Model, TNS has its own customer-loyalty segmentation, the Champion Model, which is a quadrant analysis contrasting likelihood to recommend and likelihood to repurchase. Note: I swapped the axes from the customary presentation that TNS uses to make the segmentation more clearly match up with the Apostle Model. The four segments of this model: - Champion - high repurchase intentions, high likelihood to recommend – true apostles of your brand, who practice what they preach
- Moral Supporter – low repurchase intentions, high likelihood to recommend – false prophets of your brand
- Rebel - low repurchase intentions, low likelihood to recommend – these apostates won’t be customers long
- Captive - high repurchase intentions, low likelihood to recommend – these hostages can’t switch because of corporate policy or financial considerations
The actual questions, taken from the TNS CLI template: - Likelihood to recommend - "Based on your experience, how likely would you be to recommend [organization/brand] to a friend or colleague looking for [product/service category]? Definitely would, Probably would, Might or might not, Probably would not, Definitely would not"
- Likelihood to repurchase – "For your next similar purchase of [product/service category], how likely would to be to buy from [organization/brand] again? Definitely would, Probably would, Might or might not, Probably would not, Definitely would not"
This champion model is most well-known to Microsoft partners, for whom TNS will conduct this customer-loyalty segmentation for free (e.g., at Microsoft’s expense). The segmentation seems to suffer from not segmenting enough. In segmentations made public by two Microsoft partners, each showed samples sizes of two dozen respondents, with 100% champions; the partner average itself currently shows 88.6% of customers are champions. This model is great in its focus on loyalty behaviors, and it is certainly better than the traditional Apostle Model in aligning its segmentations with actual behavior: the Apostle Model itself segments a group into “Loyalists/Apostles” but doesn’t have any measure of likelihood to recommend, a key behavior that would be expected of a group that is evangelizing a particular vendor. (Though Bob E. Hayes, among others, shows how CSAT can drive advocacy loyalty.) Best Practices for Using the Champion ModelTNS uses a five-point bipolar scale instead of a five-point unipolar scale, which will have greater reliability and validity. Accordingly, I suggest using this scale instead: Not at all likely, Slightly likely, Moderately likely, Very likely, Completely likely. A segmentation that classifies almost 90% of your customers as champions isn’t going to inspire your organization to improve. In truth, you want customers who are completely likely to repurchase and recommend: only classify the “completely likely” as champions. Your customer satisfaction initiatives should be designed around driving that type of behavior. Instead of using a traditional quadrant analysis, I refactor the analysis to be analogous to the Apostle Model: This refactoring lowered one analysis I did from 71% champions to 21% champions! Remember, though, it's not about the score: it's about understanding how the highest level of promoters differ from the lowest so that you can make continuous improvement.
Posted by Jeffrey Henning on Mon, Aug 17, 2009
 TNS, a Vovici partner, has a CLI (Customer Loyalty Index) it uses in its business-to-business satisfaction research that measures loyalty across four questions: That last question is unique to the index and reflects the focus on B2B sales.
Here is a generic version of the TNS CLI template: We'd like to ask you about your overall satisfaction with [organization/brand]. Considering everything you know about this company, its relationship with you and its products, services, and/or support would you say you are:
- Very dissatisfied
- Somewhat dissatisfied
- Somewhat satisfied
- Very satisfied
Based on your experience, how likely would you be to recommend [organization/brand] to a friend or colleague looking for [product/service category]?
- Definitely would not
- Probably would not
- Might or might not
- Probably would
- Definitely would
For your next similar purchase of [product/service category], how likely would you be to buy from [organization/brand] again?
- Definitely would not
- Probably would not
- Might or might not
- Probably would
- Definitely would
In general, how would you rate the competitive advantage provided to your company by using [organization/brand] rather than using any other company that provides similar solutions, services, products, support, etc.?
- No advantage at all
- Only a slight advantage
- Some advantage
- Big advantage
- Vital advantage
The TNS CLI is an average that can range from 0% to 100%. It is the average of % Satisfied (scores 3 or 4 on the 4-point scale), the % Likely to Recommend (scores 4 or 5 on the 5-point scale), the % Likely to Repurchase (scores 4 or 5) and the % Competitive Advantage (score 4 of 5, “Big advantage” or “Vital advantage”).
One of the keys to note about the TNS CLI is that it, like many loyalty indices, integrates satisfaction directly into the measure. Update (9/3): The Champion Model is a TNS customer segmentation built on top of the TNS CLI.
Posted by Brian Koma on Wed, Aug 12, 2009
 Customer loyalty is essential to any organization seeking to maintain existing revenue or trying to create revenue growth. Holding onto existing customers can dramatically reduce the amount of time, effort and money required to grow the organization. Customer retention rates can be low – or as high as 100%. Lower customer retention/renewal rates are problematic because they are expensive and inefficient to counteract, requiring a very high investment in sales and marketing programs to drive new customer acquisition. Higher customer retention rates can dramatically lower these costs and can enable a company’s growth to far outpace its rivals at much lower cost. Winning new customers while losing a significant share of existing customers is like filling a bathtub with the drain open. The examples below show the stark differences between a 74% customer retention rate and a 90% customer retention rate. At a 74% customer retention rate, a $30 million business will lose $7.8 million in revenue year-over-year and will need to sell almost $17 million in new business to reach $40 million in sales. At a 90% customer retention rate, a business will reduce its year-over-year revenue loss to $3 million and will only need to sell an additional $12 million to reach $40 million in sales. As these illustrations show, loyal customers can make a dramatic difference in an organization’s overall financial health and dramatically reduce the cost of growth, even in a challenging economy.
Posted by Jeffrey Henning on Wed, Jul 22, 2009
 Business Over Broadway (BOB) developed two loyalty indices by analyzing the results from two satisfaction surveys that included eight satisfaction and loyalty questions: - Overall satisfaction
- Likelihood to choose again for the first time
- Likelihood to recommend
- Likelihood to continue purchasing same products/services
- Likelihood to purchase different products/services
- Likelihood to increase purchase size
- Likelihood to increase frequency of purchasing
- Likelihood to switch to a different provider
BOB than conducted a factor analysis on the results to determine which of three types of loyalty a question would better measure: - Advocacy Loyalty—reflecting the degree to which customers will recommend the company to others
- Purchasing Loyalty—reflecting the degree to which customers will increase their purchasing behavior
- Defection Loyalty—reflecting the degree to which customers will switch to a different company
This factor analysis showed which factor each question best fit with: | | Advocacy Loyalty | Purchasing Loyalty | Defection Loyalty | | Overall satisfaction | X | | | | Likelihood to choose again for the first time | X | | | | Likelihood to recommend | X | | | | Likelihood to continue purchasing same products/services | X | | | | Likelihood to purchase different products/services | | X | | | Likelihood to increase purchase size | | X | | | Likelihood to increase frequency of purchasing | | X | | | Likelihood to switch to a different provider [scale reversed for analysis] | | | X | Each index is calculated simply by averaging the rating of each component. Since no benchmark information is provided, you are free to use whichever scales you find appropriate. Unfortunately, the original BOB scales use a 0-10 bipolar scale for satisfaction and 0-10 unipolar scale for likelihood. Add this to the growing body of research that shows that likelihood to recommend is not a unique measure of advocacy. Two counterintuitive findings from the research: - Overall satisfaction was found to be a factor of loyalty, despite significant research into how satisfaction and loyalty can differ. That said, part of the disconnect can be explained by the fact that satisfaction contributes primarily to one type of loyalty: advocacy rather than purchasing loyalty.
- The likelihood to repurchase the same products was less an indicator of purchasing loyalty than it was of advocacy loyalty.
Posted by Jeffrey Henning on Tue, Jun 23, 2009
Often when people talk about “ customer satisfaction”, they really are interested in “customer loyalty”. While CSAT has been a commonly used customer-satisfaction question for years, different organizations often use quite different loyalty measures, depending on their industry and past experience.
When considering loyalty metrics, Forrester Research is a useful firm to start with, because Forrester doesn’t provide one overriding loyalty index but instead looks at three questions to determine types of loyalty behavior:
- Willingness to consider the provider for another purchase
- Likelihood to recommend the provider to a friend or colleague
- Reluctance to switch business away from the provider
The actual question text for each is proprietary to Forrester and has not been published. The following are generic versions of these questions.
- Willingness to repurchase: How likely are you to repurchase from us? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
- Likelihood to recommend: How likely is it that you would recommend us to a friend or colleague? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
- Reluctance to switch: How reluctant are you to switch from us to another provider for these products or services? Not at all reluctant, Slightly reluctant, Somewhat reluctant, Moderately reluctant, Very reluctant
For each measure, Forrester looks at the percent that answer 4 or 5 (“Moderately *” or “Very *” in the scales above). Bruce Temkin has published some Forrester loyalty results using these measures on his blog.
To speed up the questionnaire for respondents, you can use the same scale as the prior two questions by inverting the last question:
3. How likely are you to switch from us to another provider for these products or services? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
If you do that, then for consistency of analysis with the other two measures, reverse the coding of this one question so that “Very likely” is 1 and “Not at all likely” is 5.
These measures are unusual in that Forester elected not to roll them up into a multi-item scale but kept them separate. Next time around I’ll look at a loyalty index developed by a Vovici partner.
Posted by Jeffrey Henning on Wed, Jun 10, 2009
 Perhaps you could have safely ignored CEM (Customer Experience Management) three years ago, but you certainly can't now:
- Customer experience is growing up; it's been important to firms for long enough that your competitors are differentiating themselves on experience.
- The recession is strengthening the correlation between customer experience and customer loyalty, across all 12 B2C industries Forrester Research studied in its report "Customer Experience Correlates to Loyalty" (published February 17, 2009).
As CEM comes of age, most businesses will cut other areas disproportionately more than they will cut areas that impact customer experience. So customer experience is more important than ever, but how should you measure your customers' perceptions of the experience you provide? The service quality gap model is more useful for identifying areas that need improvement than for tracking overall sentiment. For this purpose, you might want to consider the Forrester CxPi, which is a succinct measure for evaluating customer experience that you can readily adapt to your own research. Forrester calculates it customer-experience index using the net score for three key questions:
- Usefulness - Thinking about your recent interactions with these firms, how effective were they at meeting your needs? 1 - Didn't meet any of my needs to 5 - Met all of my needs
- Ease of Use - Thinking about your recent interactions with these firms, how easy was it to work with these firms? 1 - Very difficult to 5 - Very easy
- Enjoyability - Thinking about your recent interactions with these firms, how enjoyable were the interactions? 1 - Not at all enjoyable to 5 - Very enjoyable
The individual indexes for each measure are calculated by subtracting the percentage of a firm's customers that reported a bad experience from the percentage that reported a good experience: take the percentage of consumers who selected one of the top two choices (4 or 5) and subtract the percentage of consumers who selected the bottom two choices (1 or 2).
To calculate the overall CxPi, Forrester Research averages the net scores for all three questions. An example:
Forrester provides the following guidelines for judging a CxPi:
- Excellent: 85%+
- Good: 75% to 84%
- Okay: 65% to 74%
- Poor: 55% to 64%
- Very poor: <55%
What makes the CxPi an outstanding metric is its strong correlation to loyalty. Depending on industry, Forrester reports a very high correlation to willingness to repurchase, a high correlation to likelihood to recommend, and a medium correlation to reluctance to switch.
The minor disappointments with the index relate to the choice of scales. Unfortunately, the choice of scales is inconsistent: ease of use is a bipolar scale instead of a unipolar scale like usefulness and enjoyability. And scales that label each point have greater reliability and validity, especially among less educated consumers, than scales that label only the endpoints, as done by Forrester here. But those are minor quibbles. While the CxPi was only created in 2007, it has already emerged as a valuable and important index.
Forrester makes available a complimentary copy of the 2008 results to the Customer Experience Index. This report provides the results for 113 organizations in a dozen different industries, making it an excellent document to use for benchmark comparisons of the customer experience your organization provides.
Posted by Jeffrey Henning on Fri, May 29, 2009
 Many organizations use the following question, often called CSAT, to measure customer satisfaction:
What is your overall satisfaction with our company?
1. Very dissatisfied
2. Somewhat dissatisfied
3. Neither satisfied nor dissatisfied
4. Somewhat satisfied
5. Very satisfied
This question has the twin advantages of brevity and familiarity; it is recognized and easily answered by most respondents. Any adult who has taken a survey has most likely answered a form of this question before. CSAT is traditionally analyzed by tracking over time the percentage of "Satisfied" respondents, i.e., the percent who answer 4 or 5. Many organizations are happy to see that 70-80% of their customers are satisfied and feel little sense of urgency to make improvements so that satisfaction exceeds this level. When phrased as above, the satisfaction question ignores significant research into how to structure rating scales to provide the greatest reliability and validity (see the abstract of "The Optimal Length of Rating Scales to Maximize Reliability and Validity" by Jon Krosnick and Alex Tahk, which studied 706 tests). The CSAT question should instead be asked in one of the two following ways, with no numbers presented: What is your overall satisfaction with our company? - Not at all satisfied
- Slightly satisfied
- Moderately satisfied
- Very satisfied
- Completely satisfied
Or: What is your overall satisfaction with our company?
- Completely dissatisfied
- Mostly dissatisfied
- Somewhat dissatisfied
- Neither satisfied or dissatisfied
- Somewhat satisfied
- Mostly satisfied
- Completely satisfied
Mere satisfaction alone is not enough: the key top-line number is the percent of respondents reporting themselves to be "Completely satisfied". This is a far more important metric. In the seminal paper, "Why Satisfied Customers Defect" by Thomas O. Jones and W. Earl Sasser, Jr. (covered in this description of the "folk" Apostle Model), the authors report that for Xerox completely satisfied customers (rating of 5) were six times more likely to repurchase over the next 18 months than somewhat satisfied customers (ratings of 3-4).
Your first survey to report this statistic will certainly report a much lower percentage than you had hoped for, giving you a meaningful metric to track your performance against. Use the top-two percentage on the five-point scale for marketing purposes, and to compare yourself to other firms' self-reported numbers, but use the "completely satisfied" percentage to measure the results of your initiatives and innovation. If satisfaction is just one part of the customer experience and loyalty survey you are conducting, the CSAT question can be an effective measure. By itself, though, a single question can be very volatile from measurement period to measurement period. As a result, most professionally commissioned customer-satisfaction reports provide a customer-satisfaction index, derived from two to four questions. The American Customer Satisfaction Index is the most famous of these and is useful if the principle concern of your research is customer satisfaction.
Posted by Jeffrey Henning on Fri, May 15, 2009
 There’s something devilish in the air. Wednesday night my wife and I watched a Dr. Who episode involving a demon, Thursday afternoon I attended a webinar where Fred Reichheld said market researchers thought he was the devil, and this morning my son complained that his camping trip meant he was going to miss tonight's premiere of Angels & Demons.
Since you probably don’t care about my family’s viewing habits, let’s turn to Fred’s presentation. Fred showed a slide with his face superimposed onto a cartoon devil, claiming that he has become the enemy of traditional market researchers. Among the criticisms he reported:
 Net Promoter Score® oversimplifies customer loyalty NPS is a score that doesn’t provide solutions Market researchers have proprietary indices with greater validity NPS is hazardous to your business NPS does not correlate to growth Fred is dumb or lying Well, presenting an oversimplification of legitimate criticism of NPS is one way to attempt to inoculate your listeners from paying attention to that criticism. I thought it was amusing, but I will play along. In fact, I will play devil’s advocate and give Fred halos instead of horns for these contributions to loyalty research:
 Discovering that customer retention is a key driver of profitability (q.v., Reichheld, Frederick F. (1993), “ Loyalty-Based Management,” Harvard Business Review, 71 (2), 64–73) Promoting the importance of very short surveys Championing the need for a common loyalty measure across the enterprise Pointing out that a 0-10 scale is an intuitive scale for telephone research, since—unlike a 1-10 scale—the 0 is unambiguously the worst score Advocating the firing of staff who cheat on their customer-feedback numbers and discouraging organizations from tying that feedback to compensation Making NPS free That last is important. As he said: “NPS is an 'open source' system: you are all welcome to use it! You don't have to pay me to use it.” (I could have done without his next statement: “Therefore its bad business for market research, and why they call me a liar.”) But if you’ve ever gone to implement the five questions of the Secure Customer Index® or the one question of the Customer Effort Score™ and realized you couldn’t because they are proprietary, you’ll appreciate that there are no such restrictions on the Net Promoter Score.
Oh, and even if you liked the book (Angels & Demons, not The Ultimate Question), my advice is to skip the movie. This review of Angels & Demons said it all for me: "Hanks returns to the dullest role of his career, under the direction of Howard, who takes the material as seriously as a kidney stone on the way out." As for Fred, I’ll leave it to you to decide whether he is an angel or demon.
[All trademarks are properties of their respective holders. Vovici makes no claim to any of these registered or common-law marks.]
Error sending email
Email sent successfully
|