Free EBook!
We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys.
> Download your free copy
|  |
|
 RSS Feed
Posted by Jeffrey Henning on Wed, Nov 04, 2009
 At the MRA First Outlook Conference in San Diego, Reed Cundiff, senior director of central market research for Microsoft (and formerly an analyst with the Yankee Group), discussed how market research at the company has evolved and is evolving.
Prior to the creation of the Central Market Research Insights team, researchers had existed alone or in pairs in many different departments across the company. Job descriptions varied significantly; researchers had no career path within Microsoft.
Four years ago, the CMRI had just eight staff. It peaked at about 102 people in June, before being brought down by layoff to 97 staff. The upside of the downside, as it were, is that it has further accelerated the centralization of market research: departments that in the past were funding their own research are now turning to the central group instead. "We see that a lot of ad hoc research budgets have been cut; that is good for us, as there were many projects done outside of our research group." As a result, the organization is eliminating redundant and superfluous research expenditures.
The vision for the central research group is "to be a driving force behind Microsoft's business and product strategy" with the mission of delivering "strategic, fact-based insights that drive Microsoft's most essential business decisions."
The organization spends $80M to $110M annually on external research: the technology sector's largest research budget, according to Reed. Projects range from doing a market opportunity analysis for a v1 health care product to conducting a customer satisfaction survey with 100,000 respondents across 86 countries. CMRI devotes 3.5 FTEs to its research vendor management program, where they develop the preferred vendor list and do biannual reviews of the vendors (and ask the vendors to review the research managers they interact with). The result has been better use of outside vendors and consistent improvement. Before the new process, Microsoft would "run a pilot with a vendor, fall in love with that vendor, bury that vendor, then never do business with that vendor again!"
CMRI's strategy has three key components: to "deliver integrated insights" (primary research with market analysis), to "be a trusted advisor" and to "display business acumen". As a result, what Microsoft wants and expects from its internal researchers is changing [worth a blog post of its own!]: researchers need to be more consultative and need to specialize in a few focus areas.
 In its research on research, the CMRI has adopted Six Sigma. "In the past fiscal year, we went through the Six Sigma process and we are reducing the number of defects study by study by study. We averaged 12 defects per final report in a six-month period to 3 the next period down to 2 most recently."
"We need to seize the opportunity. We are moving through a lot of challenges but to drive a fact-based culture, the timing has never been better for us." In his concluding remarks, Reed said, "We spend millions of dollars that affects hundreds of millions of dollars in marketing spend that affects billions of dollars in revenue. We have to get it right."
Posted by Brian Koma on Fri, Oct 23, 2009
 According to the Vovici CE IQ study, organizations that have the most loyal customers have approached primary research with a coordinated strategy of feedback management.
Standardizing on a common feedback management solution throughout the enterprise provides many benefits for organizations. First, they can use the platform to coordinate activities across different departments and functional areas, helping to eliminate redundant work. Second, a common platform enables the use of "guidelines and guardrails" that empower users with the ability to capture data within pre-approved parameters and branding standards. Finally, it enables easy information sharing among internal stakeholders while eliminating redundant costs for multiple platforms.
Once the platform is in place, organizations must agree on a set of metrics that will be used to evaluate the customer experience. These metrics can be drawn from one of the many existing CEM metrics that already exist, such as Net Promoter, ACSI, the Apostle Model, Customer Heartbeat and many others, or they can utilize a proprietary model. While we can nitpick each measure, and some are better than others for different industries and different organizations, consider this: the most important element of success is to agree on a set of metrics and then apply them consistently throughout the organization.
To achieve the highest levels of loyalty, organizations must also coordinate their feedback efforts across all parts of the organization to avoid fatiguing customers with requests for information. One large customer of a Fortune 50 firm brought to the CEO's attention that they were receiving 100 survey invitations a year! After that Fortune 50 firm became a Vovici client, they realized that they were fielding 450 separate customer feedback initiatives. By eliminating duplicate requests for the same or similar sets of information, organizations can reduce the number of surveys that are conducted while ensuring that response rates will increase to higher levels and avoid the erosion that many organizations are facing due to "feedback fatigue."
Finally, organizations with the most loyal customers track the metrics they measure over a period of time to determine whether or not they are moving the needle in the right direction.
Posted by Jeffrey Henning on Mon, Sep 14, 2009
Today at the Gartner CRM conference, Jim Davies gave his presentation, "EFM: The Who, When, Why, Where & What of Surveying". Some highlights:
- What? I noticed in Jim’s other presentation this morning, he used “feedback management” and “survey” rather than “EFM”: it’s still an acronym that prompts quizzical looks. Jim described enterprise feedback management systems as the feedback “hubs” of their organizations, the central repository for survey data, across departments and across channels (web, paper, phone, etc.).
- Why? Without EFM systems, organizations trip all over themselves, oversurveying customers and missurveying them, because of a lack of attention to process and results. A big failure is the lack of personalization. Jim related the tale of a colleague who bought a Mini Cooper online: he was kept up to date with each stage of the manufacture and shipping of the car, to the point that “the car” emailed him e-postcards from its cruise across the Atlantic (“Playing shuffle board with the other Minis. Looking forward to meeting you!”). It was a great, personalized experience. Until the survey came! Ten pages, 80 questions, beginning by asking which car was ordered, what option package was selected, and so on. Apparently the survey researcher hadn’t played shuffleboard with the car.
- Who? Jim mentioned two firms as leaders in enterprise feedback management: ConfirmIt and Vovici. He said that there were 60-70 vendors, though most had less than $2M revenue. He mentioned Allegiance as an up-and-coming vendor for its acquisition of Inquisite; he also mentioned MarketTools. I thought it was notable that, having mentioned consolidation was coming to the industry, he discussed the four firms that have made acquisitions in this market: ConfirmIt, Vovici, MarketTools and Allegiance have each made at least one acquisition (Vovici has purchased three companies: Perseus, WebSurveyor and Surveyo, as well as Raosoft web-survey assets).
- When? Begin planning for EFM implementations now! As Jim said, “Your competitors are getting better at this!” Gartner estimates that EFM implementations will grow 15-20% in 2010 over 2009. When planning to implement an EFM system, Jim noted that “EFM functionality is more diverse and complex than many organizations perceive.”
- Where? By department and across the organization. In online communities. Set up an online community with member profiles containing information such as the member’s product portfolio, demographics and geographic location. Use forums for ad-hoc feedback, with text mining supporting the qualitative analysis. For the quantitative side, field surveys to not only community members but to customers who aren’t community members as well; let customers select their preferred modality.
So there you have the 5 W’s of EFM.
Posted by Roderick Morris on Wed, Aug 19, 2009
(Used by permission of Forrester.) As the pioneers of EFM, and the ones who named the category, we at Vovici are glad to see that Forrester has recognized the central role that enterprise feedback management can play in extending the capabilities of traditional CRM. Clearly, customer understanding is critical to each of the other segments: - Customer targeting works best when organizations understand the types of customers that are most satisfied and most loyal, as discovered through EFM surveys. One could argue that customer targeting without customer surveying is like shooting in the dark.
- Customer acquisition works best when deploying sales force automation. And sales force automation works best when you have deep understanding of win/loss and rep productivity through transactional surveys. Acquisition is also supported when EFM is used to audit, measure and alert issues in Contract Life-cycle Management (CLM) systems.
- Customer retention is a key focus of customer satisfaction and customer loyalty studies. By using surveys with email alerts and triggers, EFM systems help organizations intervene to improve customer loyalty…not just measure it.
- Customer collaboration has been a particular area of focus for EFM vendors. Vovici integrates with third-party customer forums and customer community platforms, as well as offers our own online community platform or MROC specific to research.
Back at the center of the hub, Vovici has long integrated with third-party BI systems, and—in our Feedback Intelligence product—has a BI system that uniquely combines survey and operational data. There is no more powerful tool on the market for creating highly analytical and segmented reports on the fly. EFM has come a long way since Vovici introduced the first enterprise feedback management system (under that label) in 2004. Expect the pace to accelerate as EFM solutions are more tightly woven into the fabric of complementary CRM applications.
Posted by Jeffrey Henning on Sat, Jun 20, 2009
 Email triggers highlight two key aspects of enterprise feedback management: 1) transforming surveys from projects and integrating them into processes and 2) distributing survey data to employees. Typically, a web survey of customers is set up with triggers or alerts that are fired off to staff each time a respondent gives a low rating to customer satisfaction or loyalty questions. For instance, through an automated process, each call to a contact center generates a follow-up email survey within 24 hours of the issue being marked as resolved. This survey is short and—behind the scenes, thanks to CRM integration—includes data about the transaction being rated. If the customer rates the service poorly on one of several key measures, an email is triggered: this notification of a poor rating is sent to a contact center manager, and includes within it the customer’s answers to the survey as well as data about the customer, product line and call-center transaction. By receiving this notification moments after the customer has completed the survey, the manager is able to begin customer recovery with a call or email. The actual manager notified might vary depending on other fields contained within the survey, such as location of the contact center or location of the customer. Of course, the survey response is compiled and aggregated for reporting purposes, as with a traditional survey. However, thanks to the user of email triggers, measuring satisfaction has been transformed into intervening to improve that satisfaction.
Posted by Jeffrey Henning on Mon, Jun 08, 2009
One of the reasons your organization may need enterprise feedback management is to standardize on the same question wording and scale across your products, departments and divisions. Whichever measures you decide are best for your organization, use them consistently. You want to be able to compare your Net Promoter Scores®, your ACSI Scores, your CSAT scores, your Apostle Models to one another. This will enable you to compare and contrast different parts of the business. Internal benchmarking can help your different groups learn from one another.
Sad to say, such standardization is the exception rather than the rule. In a review of 29 customer-satisfaction questions used by one Vovici partner, we identified eight different scales in use with nine different ways of wording the question:
| Occurrences |
Question Wording |
| 13 |
How would you rate your company's satisfaction with... |
| 6 |
Please rate your satisfaction with... |
| 4 |
How satisfied are you with... |
| 3 |
How would you describe your level of satisfaction with... |
| 1 |
Please indicate your level of satisfaction with... |
| 1 |
Please describe your satisfaction with... |
| 1 |
In general how would your describe your level of satisfaction with... |
| Occurrences |
Scale |
| 12 |
1 (Poor) - 6 (Excellent) |
| 5 |
1 (Not Satisfied) - 5 (Very Satisfied) |
| 4 |
1 (Not At All Satisfied) - 5 (Extremely Satisfied) |
| 4 |
Very Dissatisfied, Dissatisfied, Somewhat Dissatisfied, Somewhat Satisfied, Satisfied, Very Satisfied, No Opinion |
| 1 |
Very Dissatisfied, Dissatisfied, Neither Satisfied Nor Dissatisfied, Satisfied, Very Satisfied |
| 1 |
1 (Extremely Unsatisfied) - 5 (Extremely Satisfied) |
| 1 |
Extremely Dissatisfied, Slightly Dissatisfied, Neither Satisfied Nor Dissatisfied, Slightly Satisfied, Extremely Satisfied |
| 1 |
1 - 4 [no labels specified!] |
Ironically, not a single rating scale followed the best practices of not showing numbers, of labeling each point and of using five points for unipolar scales and seven points for bipolar scales.
For individual groups that are reluctant to switch to the standard measure, allow them to run the new metrics alongside the old for a period of time so that they can understand the relationship between the two measures. Some Vovici clients have run the old and new methods side by side for a year, only retiring the old measure once year-over-year comparisons could be provided on the new measure.
Then your staff can say, "Measure by measure, what's mine is yours, and what is yours is mine." Well, at least the English majors can, and they were probably never too fond of the old numbers anyway.
Posted by Jeffrey Henning on Wed, May 27, 2009
 One of the factors that distinguishes Six Sigma from TQM (Total Quality Management) and earlier quality movements is its reliance on measurable data. Jiju Antony, in "Pros and cons of Six Sigma: an academic perspective", describes this difference like this:
Six Sigma emphasises the importance of decision making based on facts and data rather than assumptions and hunches. Six Sigma forces people to put measurements in place. Measurement must be considered as a part of the culture change.
Surveys are a key tool for transforming hypotheses and hunches about customer attitudes and outlooks into numbers and metrics. As a result, surveys are useful throughout Six Sigma work. - Early in its own deployment of Six Sigma, Caterpillar conducted a Six Sigma Supplier Survey with its partners to understand how they had deployed Six Sigma and what lessons they had learned (see Lean Six Sigma: Combining Six Sigma Quality with Lean Production Speed by Michael George).
- When General Electric began its own use of Six Sigma, each GE division conducted detailed customer surveys, asking customers to rate GE products and services on CTQ (Critical To Quality) issues and to rate best-in-class performance. This evolved into a quarterly customer-satisfaction process for many divisions, with low-scoring items in the quarterly updates becoming candidates for subsequent Six Sigma projects (see Managing Six Sigma: A Practical Guide to Understanding, Assessing, and Implementing the Strategy That Yields Bottom-Line Success by Forrest Breyfogle III, James Cupello and Becki Meadow).
- Voice-of-the-Customer research is often conducted as an input to QFD (Quality Function Deployment), with QFD transforming customer needs into engineering and quality assurance methods for developing new, high-quality products and services.
- One Six Sigma approach to web design involves an ongoing study of web-site effectiveness, which surveys visitors about their goals at the site and tracks the success rate of achieving those goals over time. Regular incremental improvements to the web site are evaluated by their effect on improving goal completion rates.
- Another organization uses an employee survey to identify bottlenecks and excessive bureaucracy that reduce employee productivity, to highlight and prioritize areas for internal process improvement.
- The book Managing Six Sigma is noteworthy among Six Sigma books because it actually practices what it preaches and includes within itself a readership satisfaction survey! The authors assert the results of this survey will help them prepare the next edition of the book.
How have you used surveys in your Six Sigma projects?
Posted by Jeffrey Henning on Thu, May 14, 2009
 Most organizations get to the point where they are subjecting customers to too many surveys through the bottom-up adoption of survey tools. This frequently happens in an organization where no market-research function exists or where MR concentrates primarily on strategic or outsourced projects.
With MR uninvolved, surveying starts as an individual activity, where each individual may have selected a different survey software application to use. Over time, these individuals begin working together, and survey research within the department turns into a collaborative effort. The requirements become increasingly complex as users' knowledge of surveying increases.
Each department or division often evolves through four stages in its use of survey systems:
- Survey activity using simple survey tools is widely distributed.
- De facto survey-domain experts emerge through experience with these tools and through peer recognition. Only rarely do these experts have any formal training in market research.
- Survey development, deployment and analysis changes from a solitary effort to a collaborative approach within the department and occasionally across departments.
- The need quickly arises for more sophisticated survey systems, greater collaborative capability and wider information sharing.
Missing from this evolution, unfortunately, is a market-research department mentoring the users of these survey tools. While enterprise feedback management systems are intended to provide cross-departmental standardization, they are adapted as frequently for standardization within a department. Departments often have all the same problems as the enterprise does when it comes to survey research, only smaller in scale.
Posted by Jeffrey Henning on Mon, May 04, 2009
Ironically, large organizations seeking to better research customer experience don't even understand their customer's experience with their research.
Let's look at the feedback process from the perspective of the typical customer of a large organization: the customer is surrounded on all sides by survey requests.
In this example, survey invitations come from the training department, the customer service department, the marketing department and the sales department. Each department has selected its own survey tool with no central coordination of how frequently customers are invited to take part in research. In fact, one Vovici client had subjected customers to 450 different survey initiatives in the prior year. The result?
- Customers stopped taking requests for surveys seriously and responded less often
- As response rates dropped, individuals invited even more customers to take surveys
- Survey content often overlapped with that of other surveys
- Significant time and energy was invested in gathering survey results, but those results were not even shared within departments, let alone between departments
- Information was gathered inconsistently by different authors, limiting:
- the ability to study changes in customer attitudes over time
- the ability to compare results across departments and product lines
- Wasted investment in redundant tools and surveys
- No one had a true, holistic picture of the customer experience.
Any organization serious about understanding its customer experience needs to start by analyzing and standardizing its own research into that experience. A feedback assessment is a great place to start on the path to enterprise feedback management.
Posted by Jeffrey Henning on Mon, Apr 20, 2009
 Heidi Lo and Brad Bortner of Forrester have written a new independent report, "Role Insights: Market Researchers Struggle For Strategic Relevance":
Forrester surveyed 40 market researchers from business-to-business (B2B) and business-to-consumer (B2C) companies with annual revenues of more than $1 billion and annual market research budgets ranging from $30,000 to $8 million.... Results from Forrester's Q2 2008 Global Market Research Organizing For Influence Online Survey show that many market researchers have not been able to position their market research groups as strategic partners within their organizations. Market research leaders report finding themselves sidelined because they are often viewed more as service bureaus focused on tactical, not strategic, questions and research. To pull out of this rut and get on the road to strategic relevance, it is imperative that market research groups create processes to prove value, enhance their own operational efficiency, and improve alignment with their internal clients.
Table of Contents
- Most Market Research Groups Are Not Strategic Partners Yet
- Challenges In Funding And Operations Are Key Obstacles
- Many Market Research Groups Are Lean Extensions Of Marketing - Minus The Budget
- Recommendations
- Get Organized To Start Attaining Strategic Relevance
Forrester has prepared a sobering assessment of the state of market research within corporations. Given the strategic importance of customer research, it is surprising how project oriented most departments are. Yet MR departments are only addressing major projects-the minor projects are being addressed by line staff across the organization. Missing from Forrester's analysis is a discussion of the disintermediation of market researchers due to the widespread use of survey software, and the power of enterprise feedback management to help. Check out my post Rethinking the Role of the Market Research Department for more on this possible path for internal MR.
What do you see as the way forward for structuring market research within organizations?
Error sending email
Email sent successfully
|