Free EBook!
 We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys. Download your free copy here.
|
Survey Research & Enterprise Feedback Management
|
 RSS Feed
Posted by Jeffrey Henning on Sat, Jun 20, 2009
 Email triggers highlight two key aspects of enterprise feedback management: 1) transforming surveys from projects and integrating them into processes and 2) distributing survey data to employees. Typically, a web survey of customers is set up with triggers or alerts that are fired off to staff each time a respondent gives a low rating to customer satisfaction or loyalty questions. For instance, through an automated process, each call to a contact center generates a follow-up email survey within 24 hours of the issue being marked as resolved. This survey is short and—behind the scenes, thanks to CRM integration—includes data about the transaction being rated. If the customer rates the service poorly on one of several key measures, an email is triggered: this notification of a poor rating is sent to a contact center manager, and includes within it the customer’s answers to the survey as well as data about the customer, product line and call-center transaction. By receiving this notification moments after the customer has completed the survey, the manager is able to begin customer recovery with a call or email. The actual manager notified might vary depending on other fields contained within the survey, such as location of the contact center or location of the customer. Of course, the survey response is compiled and aggregated for reporting purposes, as with a traditional survey. However, thanks to the user of email triggers, measuring satisfaction has been transformed into intervening to improve that satisfaction.
Posted by Jeffrey Henning on Mon, Jun 08, 2009
One of the reasons your organization may need enterprise feedback management is to standardize on the same question wording and scale across your products, departments and divisions. Whichever measures you decide are best for your organization, use them consistently. You want to be able to compare your Net Promoter Scores®, your ACSI Scores, your CSAT scores, your Apostle Models to one another. This will enable you to compare and contrast different parts of the business. Internal benchmarking can help your different groups learn from one another.
Sad to say, such standardization is the exception rather than the rule. In a review of 29 customer-satisfaction questions used by one Vovici partner, we identified eight different scales in use with nine different ways of wording the question:
| Occurrences |
Question Wording |
| 13 |
How would you rate your company's satisfaction with... |
| 6 |
Please rate your satisfaction with... |
| 4 |
How satisfied are you with... |
| 3 |
How would you describe your level of satisfaction with... |
| 1 |
Please indicate your level of satisfaction with... |
| 1 |
Please describe your satisfaction with... |
| 1 |
In general how would your describe your level of satisfaction with... |
| Occurrences |
Scale |
| 12 |
1 (Poor) - 6 (Excellent) |
| 5 |
1 (Not Satisfied) - 5 (Very Satisfied) |
| 4 |
1 (Not At All Satisfied) - 5 (Extremely Satisfied) |
| 4 |
Very Dissatisfied, Dissatisfied, Somewhat Dissatisfied, Somewhat Satisfied, Satisfied, Very Satisfied, No Opinion |
| 1 |
Very Dissatisfied, Dissatisfied, Neither Satisfied Nor Dissatisfied, Satisfied, Very Satisfied |
| 1 |
1 (Extremely Unsatisfied) - 5 (Extremely Satisfied) |
| 1 |
Extremely Dissatisfied, Slightly Dissatisfied, Neither Satisfied Nor Dissatisfied, Slightly Satisfied, Extremely Satisfied |
| 1 |
1 - 4 [no labels specified!] |
Ironically, not a single rating scale followed the best practices of not showing numbers, of labeling each point and of using five points for unipolar scales and seven points for bipolar scales.
For individual groups that are reluctant to switch to the standard measure, allow them to run the new metrics alongside the old for a period of time so that they can understand the relationship between the two measures. Some Vovici clients have run the old and new methods side by side for a year, only retiring the old measure once year-over-year comparisons could be provided on the new measure.
Then your staff can say, "Measure by measure, what's mine is yours, and what is yours is mine." Well, at least the English majors can, and they were probably never too fond of the old numbers anyway.
Posted by Jeffrey Henning on Wed, May 27, 2009
 One of the factors that distinguishes Six Sigma from TQM (Total Quality Management) and earlier quality movements is its reliance on measurable data. Jiju Antony, in "Pros and cons of Six Sigma: an academic perspective", describes this difference like this:
Six Sigma emphasises the importance of decision making based on facts and data rather than assumptions and hunches. Six Sigma forces people to put measurements in place. Measurement must be considered as a part of the culture change. Surveys are a key tool for transforming hypotheses and hunches about customer attitudes and outlooks into numbers and metrics. As a result, surveys are useful throughout Six Sigma work.
- Early in its own deployment of Six Sigma, Caterpillar conducted a Six Sigma Supplier Survey with its partners to understand how they had deployed Six Sigma and what lessons they had learned (see Lean Six Sigma: Combining Six Sigma Quality with Lean Production Speed by Michael George).
- When General Electric began its own use of Six Sigma, each GE division conducted detailed customer surveys, asking customers to rate GE products and services on CTQ (Critical To Quality) issues and to rate best-in-class performance. This evolved into a quarterly customer-satisfaction process for many divisions, with low-scoring items in the quarterly updates becoming candidates for subsequent Six Sigma projects (see Managing Six Sigma: A Practical Guide to Understanding, Assessing, and Implementing the Strategy That Yields Bottom-Line Success by Forrest Breyfogle III, James Cupello and Becki Meadow).
- Voice-of-the-Customer research is often conducted as an input to QFD (Quality Function Deployment), with QFD transforming customer needs into engineering and quality assurance methods for developing new, high-quality products and services.
- One Six Sigma approach to web design involves an ongoing study of web-site effectiveness, which surveys visitors about their goals at the site and tracks the success rate of achieving those goals over time. Regular incremental improvements to the web site are evaluated by their effect on improving goal completion rates.
- Another organization uses an employee survey to identify bottlenecks and excessive bureaucracy that reduce employee productivity, to highlight and prioritize areas for internal process improvement.
- The book Managing Six Sigma is noteworthy among Six Sigma books because it actually practices what it preaches and includes within itself a readership satisfaction survey! The authors assert the results of this survey will help them prepare the next edition of the book.
How have you used surveys in your Six Sigma projects?
Posted by Jeffrey Henning on Thu, May 14, 2009
 Most organizations get to the point where they are subjecting customers to too many surveys through the bottom-up adoption of survey tools. This frequently happens in an organization where no market-research function exists or where MR concentrates primarily on strategic or outsourced projects.
With MR uninvolved, surveying starts as an individual activity, where each individual may have selected a different survey software application to use. Over time, these individuals begin working together, and survey research within the department turns into a collaborative effort. The requirements become increasingly complex as users' knowledge of surveying increases.
Each department or division often evolves through four stages in its use of survey systems:
- Survey activity using simple survey tools is widely distributed.
- De facto survey-domain experts emerge through experience with these tools and through peer recognition. Only rarely do these experts have any formal training in market research.
- Survey development, deployment and analysis changes from a solitary effort to a collaborative approach within the department and occasionally across departments.
- The need quickly arises for more sophisticated survey systems, greater collaborative capability and wider information sharing.
Missing from this evolution, unfortunately, is a market-research department mentoring the users of these survey tools. While enterprise feedback management systems are intended to provide cross-departmental standardization, they are adapted as frequently for standardization within a department. Departments often have all the same problems as the enterprise does when it comes to survey research, only smaller in scale.
Posted by Jeffrey Henning on Mon, May 04, 2009
Ironically, large organizations seeking to better research customer experience don't even understand their customer's experience with their research.
Let's look at the feedback process from the perspective of the typical customer of a large organization: the customer is surrounded on all sides by survey requests.
In this example, survey invitations come from the training department, the customer service department, the marketing department and the sales department. Each department has selected its own survey tool with no central coordination of how frequently customers are invited to take part in research. In fact, one Vovici client had subjected customers to 450 different survey initiatives in the prior year. The result?
- Customers stopped taking requests for surveys seriously and responded less often
- As response rates dropped, individuals invited even more customers to take surveys
- Survey content often overlapped with that of other surveys
- Significant time and energy was invested in gathering survey results, but those results were not even shared within departments, let alone between departments
- Information was gathered inconsistently by different authors, limiting:
- the ability to study changes in customer attitudes over time
- the ability to compare results across departments and product lines
- Wasted investment in redundant tools and surveys
- No one had a true, holistic picture of the customer experience.
Any organization serious about understanding its customer experience needs to start by analyzing and standardizing its own research into that experience. A feedback assessment is a great place to start on the path to enterprise feedback management.
Posted by Jeffrey Henning on Mon, Apr 20, 2009
 Heidi Lo and Brad Bortner of Forrester have written a new independent report, "Role Insights: Market Researchers Struggle For Strategic Relevance":
Forrester surveyed 40 market researchers from business-to-business (B2B) and business-to-consumer (B2C) companies with annual revenues of more than $1 billion and annual market research budgets ranging from $30,000 to $8 million.... Results from Forrester's Q2 2008 Global Market Research Organizing For Influence Online Survey show that many market researchers have not been able to position their market research groups as strategic partners within their organizations. Market research leaders report finding themselves sidelined because they are often viewed more as service bureaus focused on tactical, not strategic, questions and research. To pull out of this rut and get on the road to strategic relevance, it is imperative that market research groups create processes to prove value, enhance their own operational efficiency, and improve alignment with their internal clients.
Table of Contents
- Most Market Research Groups Are Not Strategic Partners Yet
- Challenges In Funding And Operations Are Key Obstacles
- Many Market Research Groups Are Lean Extensions Of Marketing - Minus The Budget
- Recommendations
- Get Organized To Start Attaining Strategic Relevance
Forrester has prepared a sobering assessment of the state of market research within corporations. Given the strategic importance of customer research, it is surprising how project oriented most departments are. Yet MR departments are only addressing major projects-the minor projects are being addressed by line staff across the organization. Missing from Forrester's analysis is a discussion of the disintermediation of market researchers due to the widespread use of survey software, and the power of enterprise feedback management to help. Check out my post Rethinking the Role of the Market Research Department for more on this possible path for internal MR.
What do you see as the way forward for structuring market research within organizations?
Posted by Jeffrey Henning on Tue, Mar 31, 2009
Posted by Jeffrey Henning on Thu, Mar 05, 2009
 Typically, when consulting with large organizations about enterprise feedback management, one of the big issues I encounter is that they have too many surveys done with too many of their customers. For such organizations, using a centralized panel system with random sampling dramatically reduces the frequency with which any individual customer is surveyed.
I want to be clear, though, that when I am criticizing a census I am really criticizing needless attempts at a census. For me, the canonical example will always be the automobile manufacturer that started subjecting their 8,000 dealers to web surveys about all manner of issues, from the strategic to the tactical, simply because they could. Just like the tragedy of the commons, though, this wanton surveying ruined a good thing, as dealers stopped participating.
Although I describe this approach as a "census", it is really an attempt at a census, selecting everyone in the target population to participate but not reaching many of them. The 2000 United States Census surveyed 99.6% of the population, while census attempts by a corporation typically get much lower participation-as low as 20% for firms that overuse the technique.
Advantages of a Census
- Every customer or employee is invited to participate. This is important for annual measures of customer satisfaction and employee well-being, which are opportunities for everyone to provide their feedback.
- Maximizes exposure to negative feedback. By contacting everyone, and by setting up email triggers to alert you to negative responses, you will have the opportunity to intervene with more individuals to attempt to improve their satisfaction.
- Greater confidence levels in the results. For target populations with fewer than 1,000 individuals, the recommended sample size is so large as a proportion that you must typically attempt a census to gain enough responses to have a high degree of statistical confidence that your results represent the target population.
Disadvantages of a Census
- Limits opportunities for other surveys. If everyone in the target population has just been invited to participate in a survey, they are unlikely to respond to an immediate subsequent request to take an additional survey. We once had a client that realized it needed to do some additional unrelated research, right after it had emailed its house list of 60,000 for a tactical survey.
- Leads to declining response rates. A survey important enough to a large organization for it to be sent as a census is typically longer than average. Inviting everyone to take a long survey makes them less likely to participate in the next survey.
Sometimes you simply must do a census instead of a survey. Just make sure you are doing so because you have to. And, if you're in the States, remember to be kind to the census worker when he or she comes to your door next year.
Posted by Jeffrey Henning on Tue, Jan 27, 2009
 Many of our prospects for enterprise feedback management come to us after discovering, in horror, how terrible they are as an organization at surveying their customers. They survey all of their customers instead of taking a random sample. They send out surveys that are far too long. They send out surveys on topics that the organization has already researched. But, most of all, they just write bad questionnaires.
Meanwhile, most of these organizations have on staff experts at writing questionnaires, specialists who can tease out key insights from respondents, professional researchers with decades of mastering the survey process: experts who aren't available to their coworkers. In fact, that's how many of the organizations got into this jam in the first place: the market research department, which typically manages only the highest profile, most strategic research, was - by definition of its role - unavailable to assist coworkers with tactical surveys. As a result, those coworkers bought survey software and did the project themselves.
It's time for market research departments to rethink their mission. They should not simply be project managers, overseeing large outside research efforts; they should be mentors to their coworkers, assisting fellow employees with the deployment of surveys. To do this, MR departments need to expand, hiring additional staff with the appropriate research skills and knack for teaching. How to cost justify this expansion? Quantify the cost of bad research. What decisions are being made on the basis of results from amateur surveys? These surveys can lead the organization astray, often incorporating biased questions or unrepresentative samples. Conduct a feedback assessment to get an accurate view of how pervasive the problem is (it's always a bigger problem than you think!). Your organization is making a significant investment in this research, but it is often a hidden investment: the survey software is being purchased on a credit card or expensed, and no one is tracking how much time employees are spending doing surveys (an expenditure that far exceeds the cost of the technology). Since these are surveys that are worth doing, they are worth doing right.
Too often, the market research department has been disintermediated by the rise of survey software. It is best for your organization if the MR department reintermediates itself-still letting line staff do most of the work for tactical surveys, but reviewing questionnaires and invitation lists before the survey goes live, to provide results that staff can feel comfortable making decisions with.
Posted by Jeffrey Henning on Thu, Dec 18, 2008
Today we announced the release of Vovici v4, the latest expansion of our flagship feedback platform. Vovici v4 extends feedback throughout the enterprise: its assisted survey authoring empowers managers to write their own surveys in conjunction with internal mentors, while its powerful business intelligence empowers employees to develop and customize their own analysis of surveys reflecting their domain or geography.
Key features of Vovici v4 include:
- Uniquely combines Proprietary Panels, Online Surveys and Business Intelligence
- Embeds feedback into business applications and leading CRM systems to allow 360-degree views of customers
- Offers industry's most powerful analytics, data visualization and reporting
- Helps you manage survey creation workflows and approval cycles to protect your brand and eliminate feedback fatigue
Vovici v4 helps your organization:
- Improve strategy with deeper knowledge of customers and employees
- Drive enterprise efficiencies and cost reduction with robust, fast, accurate EFM
- Deliver enterprise-grade protection for your brand and critical business data
Vovici v4, formerly known as Vovici EFM Community and before that as Perseus SurveySolutions/EFM, is the fourteenth major release of the product that pioneered the Enterprise Feedback Management industry.
Error sending email
Email sent successfully
|