Welcome to the Listening Post!

Random Sampling Insufficient When Assessing Employees

As Brian Koma wrote on employee-customer engagement:

According to the CE IQ study, organizations with the most loyal customers not only measure and monitor employee interactions with customers but then share that feedback with employees (a strong correlation of 0.54 to the study's loyalty index). This creates a closed feedback loop that allows customer-facing staff members to understand the impact of their interactions on customers and enables the creation of programs that allow employees to improve those interactions over time.  

If your organization had been conducting a traditional transactional survey, you will need to expand that survey to support this level of feedback. It requires a number of changes. To begin using transactional surveys for employee assessment, you will need to:

• Move from random sampling to an attempted census. Probability sampling--e.g., surveying customers from one out of every five interactions--is fine for measuring the overall service levels you are providing and is typical of much transactional research, as it minimizes the number of survey invites per customer. That said, for most organizations, random sampling just won't work once you begin cross-tabulating results by employee. You won't have enough responses per employee to be able to compare and contrast employee performance with a high degree of confidence. As a result, you will need to begin inviting almost every customer to rate their interaction: we do encourage the use of touch-management rules to limit the invitation for this type of survey to once every 60 days.
• Develop new employee reports. I prefer to give employees a standard weekly report that shows their results for the prior week compared against other employees at their level and compared against the overall average. For organizations where weekly sample sizes are small, I prefer to report a rolling average of the rating; I have recommended four- to eight-week rolling averages, with longer averages for smaller weekly sample sizes. I have occasionally been pressed for daily reports, but I have never seen sufficient sample sizes to warrant daily reporting rather than weekly reporting.
• Develop new management reports. Management reports need to be defined that list all employee averages for a manager and provide the verbatim comments by employee. On our blog post about the Domino's transactional survey, which is used for employee reports, Pam Snodgrass makes a very insightful comment: "We seem to have differing opinions among our managers about sharing survey information with employees. One manager wants to make it clear to customers that the information will only be shared with management, and another would like the survey results to be emailed directly to employees. I think it should be somewhere in the middle-such as share weekly or monthly statistics and useful open-ended comments, but not individual survey results from individual customers." I think Pam's middle approach is best, as management can cherry-pick illustrative comments to make their point.
• Develop HR guidelines for the use of these employee reports. The employee-employer relationship is the second most regulated relationship in the United States (after the spousal relationship). With that in mind, it is important that the human-resource department be actively involved in the creation of these employee reports. Certainly there will be occasions where ongoing negative customer feedback will lead to an organization deciding to terminate the employment of specific individuals. But that is not the primary purpose or even a goal of this research: the customer feedback should be used to mentor employees, not berate and bludgeon them. Successful employee assessments help employees better understand and emphathize with the customer viewpoint, so that they can serve them better.

Whew! Is all this work worth it? I believe so. Out of the 24 best practices we studied, providing employees with customer assessments had the third highest positive correlation to customer loyalty. To me it is the logical next step in the evolution of transactional surveys.