Survey Software, Web Survey, Online Surveys, and Enterprise Feedback Management solutions from Vovici

Subscribe to our Blog


Your email address


Download your free copy our eBook: Survey Software Success

Free EBook!

We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys.

> Download your free copy

Subscribe to our blog

Your email:

Solutions For:

Online Survey Solutions Voice of the Customer SolutionsMarket Research Solutions Customer Support Solutions Voice of the Employee Solutions Government Solutions

Current Articles | RSS Feed RSS Feed

Survey Compensation for Employees Gone Awry

 | Share on Facebook Facebook | Share on Twitter Twitter | Share on LinkedIn LinkedIn 

I'm adamantly against financial compensation for staff based on customer-service survey results. Invariably, it leads to customer-service agents gaming the system; this happens far more than executives are willing to acknowledge. For instance, a K-mart cashier posted a sign at the register saying, "For a chance to win a $2,500 gift card, log onto KmartFeedback.com and rate our customer service 9 or 10. Thanks for shopping at your 1 Penn Plaza Kmart!" Of course, people have a chance to win no matter what rating they give.

Kmart survey request
(photo: bcurrie)

A post on this example at The Consumerist led to a rash of comments giving examples for other major brands:

  • CVS - "Yesterday I went to CVS for a prescription. When I checked out, the cashier looked at my receipt and stated that I had been selected to participate in a survey. Yeah, yeah, yeah, I know - this happens all the time, but she then showed me a mini candy bar and said, ‘you get a free Take5 candy bar so that will remind you to give us all 5's on the survey!' I thought it was pretty smart but I still didn't do the survey..."
  • Enterprise - "I was asked to rate an Enterprise rental by the customer rep, on a scale of 1-10 (or whatever it was). I said 9 because I figure one should save 10 for over-the-top service; this particular rental went smoothly, but was nothing special. He kept after me, wanting to know what was wrong, why I didn't give them a 10, until I got the point that to them ‘10' means ‘normal service'."
  • Home Depot - "The Home Depots around me [hand you a flyer] telling you the almost exact same thing."
  • Nissan - "This reminds me of whenever I buy a car. The Nissan salesperson reminds me that I will receive a survey in the mail, and that I'm supposed to return that survey with all 5's (the best possible mark), or else their dealership will be dinged or knocked down some points or whatever. I don't generally return surveys anyway, and I certainly don't when someone tells me what my answers should be anyway."
  • Old Navy - "An Old Navy store stapled a piece of paper to my receipt with a similar suggestion - something like ‘rate us a 10 and get 10% off'. Of course you get 10% off for filling out the survey no matter what, so I filled out the survey honestly, with perhaps a slight negative influence from the suggestion."
  • Sears - "Did the same thing when I worked at Sears but it said all 10s and we were told to tell them all 10s or we'd fail. It would be nice if they used the surveys to better themselves."
  • Target - "I have had plenty of cashiers at Target say that sort of thing too."
  • Toyota - "A Toyota dealer actually told me to bring in the survey so we could 'fill it out together'. In exchange - he would fill my gas tank."
My recommendations:
  • Don't financially incentivize any staff based on survey results. Doing so will change the results, no matter how strong your corporate culture is (for instance, Enterprise would discipline the representative mentioned above). Compensating only managers will lead to managers asking representatives to ask for higher scores. The most important thing is to gather authentic feedback about current satisfaction.
  • Don't use survey results as a crutch for measuring employees. It's vital that you share rich, relevant feedback with agents so that they can serve customers better in the future. Their managers need to mentor them not overmeasure them. (See Employee-Customer Engagement Best Practices.)
  • Don't use the Net Promoter Score as a transactional measure. From these and other anecdotes, it is clear that the NPS in customer-service settings leads to a collapse in the range and validity of the scale and an obsession with the top score, even for organizations like Enterprise that try to counterbalance this.
  • Don't use numbers. They are arbitrary; in some pure mathematical realm they are better for averaging than the results of fully labeled scales but the lower interrater reliability of numeric scales negates that. What an 8 means on a 10-point scale varies from respondent to respondent. Use labels, as in these common rating scales, and make the highest labeled point hard to achieve: Not at all satisfied, Slightly satisfied, Moderately satisfied, Very satisfied, Completely satisfied.

Do you have any examples of being asked to provide artificially high ratings to surveys?

 Experience Vovici    Request a Call from a Vovici Rep     Request a Demo with a Vovici Rep

Comments

Verizon is another one I just experiences. The rep at the store thanked me for my business and asked me to log onto their site and give him a 10.
Posted @ Thursday, November 12, 2009 1:03 PM by Tim
Great post. Have you watched this video from TED Talks - Dan Pink on the surprising science of motivation. 
 
 
 
http://www.ted.com/talks/dan_pink_on_motivation.html 
 
 
 
It is a little long, but worth watching. 
 
 
 
Leslie
Posted @ Thursday, November 12, 2009 3:34 PM by Leslie Pagel
The BMW dealer came down the last $500 in exchange for a perfect survey score. The dealers are obsessed with survey score, though maybe he would have come down to that regardless. My guess is that he holds the survey out until the end.
Posted @ Thursday, November 12, 2009 4:42 PM by Erik Cortright
Excellent post. What these surveys seem to have in common are that the surveys are administered by employees of the companies carrying out the research. Would you be happy for financial compensation to be based on surveys carried out by an independent third party i.e. a research agency?
Posted @ Friday, November 13, 2009 3:54 AM by Gary Austin
Outstanding post! And unbelievably timely. Our team has been talking about this too, and is sharing some best/worst examples of reps informing customers about company surveys. 
 
 
 
And your point about over-measuring is so true! It's hard to believe how many metrics employees are forced to endure these days.  
 
 
 
It's overwhelming for the staff, and ultimately unproductive for the organization.  
 
 
 
Great stuff!
Posted @ Friday, November 13, 2009 6:51 AM by Ziggy Zubric
Gary, I don't think it matters. I think the behavior is the same regardless of whether the program is run externally or internally. The Kmart program, for instance, is run by a firm called the Link Group. Some of the other examples are managed by third parties as well.
Posted @ Friday, November 13, 2009 7:29 AM by Jeffrey Henning
I have also run into this with Ford Motors on a recent purchase. The problem to me seems to be more of upper management using this research as a way to beat down their dealerships. At least that was the impression that I received from the person begging me to give them all 5's. But it could be that the car dealership is financially motivated to have higher scores and thus game the system. I actually sent Ford corporate an email with my concern but never heard back from them.
Posted @ Monday, November 16, 2009 5:29 PM by Ken
Jeffrey, 
 
 
 
It is really unfortunate when organizations inappropriately influence customer feedback ratings in an effort to trigger some type of performance reward or payoff. You’re right that these systems get gamed by employees far too often. 
 
 
 
The fact is, though, when you establish a monetary or non-monetary incentive to encourage some type of employee behavior, there’s always a risk that it could backfire. Take, for example, companies that reward staff for hitting certain productivity or timeliness goals. Even these measures, when linked to reward and recognition, can result in unintended consequences – like getting an organization to focus too much on speed, and not enough on quality. 
 
 
 
One could argue that it’s not just customer surveys that may be tainted by this dynamic. Any internal or external management metric/goal could be similarly affected. So if you’re not going to financially incentivize staff based on survey results, then you probably shouldn’t incentivize them based on these other metrics, either. 
 
 
 
But that doesn’t seem right. Monetary and non-monetary awards do motivate employees. They provide carrots that can be very powerful catalysts for modifying organizational behaviors. 
 
 
 
It seems to me that, if the answer is to not link survey results with performance awards, that undermines reward-for-performance philosophies across the board. 
 
 
 
Is there not a middle ground that tempers the risk without eliminating the carrot? For example, inserting questions on the survey or conducting random observations that would help spot when customers are being inappropriately coached by survey stakeholders?
Posted @ Tuesday, November 17, 2009 9:45 AM by Jon Picoult
Jeffrey, I liked this post, and because of it, I just filled out a survey from a fast food restaurant. I agree w/ you -- a Likert scale that makes sense to the user is the way to go.  
 
In this survey, I was offered a Likert scale w/ numbers, then later w/ text. The troubling question was, was the restaurant in good repair. Good repair? This needs to be a more definitive term.  
 
Lastly, spend some time on a good layout. Your user is looking for reasons to leave; don't make layout one of them. My survey was 24 questions, but they weren't numbered.  
 
Thanks, Jeffrey.  
Posted @ Wednesday, November 18, 2009 1:16 PM by Tom Sakell
Jon, I think how to financially reward employees with incentives is its own fascinating topic, with decisions reflective of the culture of different employers. Except for sales staff, I personally prefer profit sharing to individual employee incentives.
Posted @ Wednesday, November 25, 2009 10:35 AM by Jeffrey Henning
This recently happened to me at a Sprint location as I purchased my new palm pre. The Rep informed me that someone would be calling me to complete a survey and to please give him the highest ratings. They did call and my experience was in fact a good one. What interest me the most was at the end of the survey the last questions was "did the sales rep ask you to provide high marks on this survey?" to which I replied "yes". It was a shame that a good experience could have a negative impact on someone because they are so worried about rankings.
Posted @ Thursday, December 10, 2009 11:44 AM by KC Stratton
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics

Receive email when someone replies.
Vision 2010 Vovici User Conference

VISION 2010 :: Vovici User Conference :: Save the Date :: May 10-12, 2010