Free EBook!
 We've compiled much of the blog into a free, 73-page ebook, Survey Software Success. The book outlines seven best practices for conducting online surveys. Download your free copy here.
|
Survey Research & Enterprise Feedback Management
|
 RSS Feed
Posted by Vovici Marketing on Wed, Jul 01, 2009
 DULLES, Va.--(BUSINESS WIRE)--Vovici, the leading provider of enterprise feedback management (EFM)
solutions and survey
software, today announced that its board of directors has appointed
Greg Stock as the company’s Chairman and Chief Executive Officer (CEO),
effective July 1. A proven technology leader, Stock brings more than 20
years of technology industry and marketing experience to Vovici.
"As a CEO, I know how important it is to understand the collective voice
of customers," said Stock. "Vovici's solutions enable organizations to
take customer feedback and turn it into customer loyalty. Vovici has
already delivered high-impact feedback solutions to more than half the
Fortune 500. I am excited to join the team and build on this foundation
of success."
An accomplished executive, Stock joins Vovici having most recently
served as president and CEO of Mirage Networks, a provider of network
security solutions based in Austin, Texas. During his tenure, Stock led
the company from start-up to market leader with more than 600 customers
across 40 countries. Mirage was successfully acquired by Trustwave in
February, 2009.
Prior to Mirage Networks, Stock was a key player in successful DC-area
technology companies. As Vice President, Marketing, Stock drove Vastera,
a provider of global trade management solutions, to the market leading
position and was instrumental in the company’s successful initial public
offering in September, 2000. Previous to this, Stock helped Manugistics,
the supply chain management leader, grow to more than $200M in revenues
and initiated the company’s expansion into China and Australia.
Vovici’s investors include Austin Ventures and Mayfield Fund. “Greg is
an outstanding choice to lead Vovici,” said Chris Pacitti, General
Partner at Austin Ventures and Vovici board member. “A tenacious leader
with an inspiring approach that energizes people to achieve, Greg has a
demonstrated track record for establishing market leadership and global
expansion initiatives. Much of Greg’s past experience is in leading
sales and marketing organizations, positioning him well to lead a
company such as Vovici that is focused on helping its clients create customer
loyalty solutions.”
Stock is succeeding Dean Wiltse who has served as Vovici’s Chairman and
CEO since 2005. Wiltse oversaw the merger of Perseus and Websurveyor in
2006, re-branded the company as Vovici, and converted the business into
a software-as-a-service (SaaS) enterprise. “The Board would like to
thank Dean for his achievements to date and know he is looking forward
to spending more time with his family in Arizona,” said David Lack,
Partner at Austin Ventures.
“We are delighted to welcome Greg to Vovici,” said Dean Wiltse. ”Vovici
has established itself as the SaaS leader in enterprise
feedback management. I look forward to working with Greg as the
company enters its next phase of growth, delivering EFM solutions to
clients so that they can better capture and analyze their customer
feedback.”
Posted by Jeffrey Henning on Mon, Jun 29, 2009
 Far and away, the most common objection we hear from organizations considering extending their survey panel into an online community is that community members will hear each other’s negative feedback. When you survey customers, employees or other key constituencies, only the survey administrator sees all the negative comments. When you move to an online community, anyone who logs in can see the negative comments. This is more an opportunity than an issue. In fact, the absolute value of negative feedback is positive. Here’s why: - Criticisms provide authenticity. Imagine, for a moment, that you hosted an online community and never received any bad reviews or comments. Since every organization and its product and services has shortcomings, an online community where those shortcomings were never discussed would seem unauthentic. It would seem like a sham or a marketing exercise, rather than a true community.
- Negative feedback is scarcer than positive feedback. That scarcity makes it worth more. Online communities excel at generating thousands of positive ideas—more ideas than your organization can implement. Negative feedback is rarer than organizations realize and often clusters around key areas that your organization has ignored or handled poorly; negative feedback gives you a chance to prioritize these issues and focus on improving them.
- Negative feedback is actionable. Negative feedback gives you an opportunity to respond. If a customer tells you ten features they want in the next edition of your product or service, there is nothing you can do for them today but to let them know you are listening and building a list. If they complain about a product damaged in shipping, or a mistake on an invoice, or confusion around a feature, you can immediately help them resolve the issue. Quick resolution of these issues demonstrates that your organization listens and cares.
- Better to manage negative feedback on your turf. The fact of the matter is that people are commenting on your organization all over the World Wide Web: on Twitter, on Facebook, on LinkedIn and on the next big social networking site that we haven’t heard of yet. Brand monitoring solutions can catch some of this commentary, but not all of it: many social networks only show designated friends each other’s comments. Right now on Facebook there’s a long discussion going on about why your organization is awful to do business with, and you will never see or be able to respond to that discussion. When the criticism happens on your site, you can instantly read and respond.
For further reading:
Posted by Jeffrey Henning on Fri, Jun 26, 2009
 The follow-up or transactional survey concerns itself with getting customer feedback regarding a specific transaction, such as a purchase, a call to a contact center, a request for service or a product return. Such surveys can be conducted for multiple reasons. They are a great way to perform quality control to determine the level of service being provided and can be used to determine inconsistencies in providing service. Follow-up surveys can identify dissatisfied customers so that service recovery can be attempted and can measure the effectiveness of service staff. Here are some of the more common mistakes I’ve seen when organizations conduct transactional surveys: - Asking respondents to specify details about the transaction rather than using data integration behind the scenes to record that information
- Allowing service staff to select or influence potential respondents, for instance, by transferring some calls but not others to an IVR system—this results in skewed results that typically overstate satisfaction
- Failing to include any survey alerts or email triggers to enable service recovery to be attempted
- Failing to invite the recipient more than once to take the survey, which can result in bias
- Compensating employees on survey results in a way that encourages gaming the system and unethical behavior
- Not sharing survey results with service staff
- Having the questionnaire take more time to complete than the transaction itself—here’s a retail example and an auto club example
- Inviting participants on a monthly basis rather than weekly or daily—respondents are typically unable to answer in detail after more than a week has passed
- Conversely, inviting participants to take the survey before the incident is resolved
- Failing to implement touch-frequency rules where respondents are not invited too often; for instance, not invited more than once in a 30-day period
- Failing to implement a survey unsubscribe process so that customers can opt out of recieving surveys altogether
What mistakes have you seen in transactional surveys you've taken?
Posted by Jeffrey Henning on Thu, Jun 25, 2009
 I hate that every gas station near me has disabled the ability for me to set the fuel dispenser to fill the tank and then walk away, safe in the knowledge the automatic cut-off will kick in. Now I have to stand at the pump manually holding the switch on the fuel dispenser until the tank is full. No doubt this is a safety innovation in case the auto cut-off failed. In a similar way, many users of our survey software dislike the fact that the first time they publish a new survey, it goes into test mode. This is a not-so-subtle reminder that they should double-check the survey before inviting participants to respond. Did I say double-check? How about triple-check and quadruple-check? For very important surveys, you should: - Self-Test – Run through the survey, answering it yourself, multiple times.
- Pre-Test – Invite coworkers or friendly outsiders to take the survey.
- Pilot Test – Invite 10% of the targeted list to take the survey.
- Publish – Invite the world to take the survey.
Self TestHere are some of the things that I self-test or review before publishing a survey: - Question flow: Do questions proceed in a logical manner from topic to topic?
- Question wording: Is each question worded clearly and unambiguously and is it free of typos and grammatical errors? The preceding question itself would make a horrible survey question: whenever one question asks for an answer on multiple, different topics, it should be split into multiple questions.
- Question types: Do the question types match the wording? Can respondents answer both "yes" and "no", or do the choices not correspond to the question? This frequently happens when a series of questions has the same choices or scale, where one or more questions is force-fitted into a method that doesn’t work.
- Scale consistency: Are related questions using the same scale? I just edited a questionnaire this week that had two different satisfaction scales, because one was written off the top of the survey author’s head and the other was copied from the question library. Do different scales arrange items consistently from best to worst or worst to best? I once had to ignore a question in analysis because it used a 1-5 rating scale with 1=best, 5=worst, in reverse of all earlier questions; the open-ended responses made it clear that some customers used the scale as written, and others used the scale as expected based on the earlier scales. (Yet another reason to avoid numbers in favor of labels in rating questions.)
- Answer validation: Is the survey configured to enforce the validation described in each question? For instance, making sure an email address is in the format jane@example.com, that that a fill-in-the-blank question is limited to numbers or that a choose-many question has a limit to the number of choices that can be selected
- Required answers: Are required answers used sparingly but appropriately, especially for critical questions and for questions that drive skip patterns? For closed-ended questions that are required, is there an appropriate choice in each case, such as “Don’t know”, “Can’t remember” or “Not applicable”?
- Skip patterns: Do the skips and conditional branches take respondents where we intended? Editing a question can sometimes delete or invalidate skip logic.
- Errors of omission: What questions did you leave out that you should have included?
Pre-TestThat last question is particularly hard to answer in a self-test. When I am feeling very unsure of a study, or the results are strategic rather than tactical, I will pre-test it on coworkers or, even better, on a small sample of the target audience (no more than 50). I will end the survey with some questions about the survey itself, to identify areas or survey structures that were confusing or ambiguous. Pilot TestSometimes after a pre-test, I will pilot-test the survey to 10% of the participant list, in a “shakedown cruise” of what one favorite client describes as the “final draft but not the final final draft” of the questionnaire. This gives even more opportunities to catch errors before the survey goes live to the full list. PublishOk, now you can publish the survey and invite one billion people to complete it. You still missed something—trust me. Most likely something related to one of your last-minute changes. But you’ve dramatically lowered your odds of missing something major. Take it from hard-won experience: If your CEO or the CEO of your client cares about this survey, you definitely want to make sure you self-test, pre-test and pilot-test before you publish. And that’s why Vovici surveys go into test mode first. Now fill up the tank and go on a test drive before that road trip.
Posted by Jeffrey Henning on Tue, Jun 23, 2009
Often when people talk about “ customer satisfaction”, they really are interested in “customer loyalty”. While CSAT has been a commonly used customer-satisfaction question for years, different organizations often use quite different loyalty measures, depending on their industry and past experience.
When considering loyalty metrics, Forrester Research is a useful firm to start with, because Forrester doesn’t provide one overriding loyalty index but instead looks at three questions to determine types of loyalty behavior:
- Willingness to consider the provider for another purchase
- Likelihood to recommend the provider to a friend or colleague
- Reluctance to switch business away from the provider
The actual question text for each is proprietary to Forrester and has not been published. The following are generic versions of these questions.
- Willingness to repurchase: How likely are you to repurchase from us? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
- Likelihood to recommend: How likely is it that you would recommend us to a friend or colleague? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
- Reluctance to switch: How reluctant are you to switch from us to another provider for these products or services? Not at all reluctant, Slightly reluctant, Somewhat reluctant, Moderately reluctant, Very reluctant
For each measure, Forrester looks at the percent that answer 4 or 5 (“Moderately *” or “Very *” in the scales above). Bruce Temkin has published some Forrester loyalty results using these measures on his blog.
To speed up the questionnaire for respondents, you can use the same scale as the prior two questions by inverting the last question:
3. How likely are you to switch from us to another provider for these products or services? Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely
If you do that, then for consistency of analysis with the other two measures, reverse the coding of this one question so that “Very likely” is 1 and “Not at all likely” is 5.
These measures are unusual in that Forester elected not to roll them up into a multi-item scale but kept them separate. Next time around I’ll look at a loyalty index developed by a Vovici partner.
Posted by Jeffrey Henning on Mon, Jun 22, 2009
 So in the recent redesign of this blog we added a Posts by Popularity sidebar. I’m glad for the addition, as that it always one of my favorite pages on news sites. That said, it reminds me of grade school.
I was the youngest in class, small for my age, and was always picked last in gym class by the Bombardment captains. I recall the one time I got to be captain, and I picked all the unathletic kids first. Yeah, we lost. The game was still fun.
In that spirit, here are the unpopular posts of this blog.
- Rethinking the Role of the MR Dept. – In these times of change, MR departments need to mentor and assist fellow employees who are doing their own surveys using survey software. This can be a jarring change from how such departments operate today.
- Market Research Blogs – Yes, I realize blogs are passé now and Twitter is where the action is: this article lists the blogs and Twitter accounts of a number of MR professionals that I read.
- MROC – Market-research online communities are popular, even if calling them MROCs isn’t. This is an abstract of Brad Bortner’s white paper on “Web 2.0 and MR” and includes my suggestion of one way to position MROCs.
- Unsubscribe Survey Questions – Letting subscribers unsubscribe from email is necessary but unpopular. Since you have to do it, modify one of these templates to make sure that you are soliciting feedback to understand why people are unsubscribing.
- Recommended Survey Length – I know, short surveys are all the rage today. But sometimes a long survey is appropriate for the research task at hand, especially for surveys of major accounts.
- Face-to-Face Interviews – Yes, they’re unpopular because they’re expensive, but face-to-face interviews provide great insight and great experiences. If all you’ve ever done is write questionnaires to administer online, you’re missing the understanding of survey research that comes from seeing the respondent react to the questions you wrote, the wording you used and the order you put the questions in.
Let the bombardment of comments begin.
Posted by Jeffrey Henning on Sat, Jun 20, 2009
 Email triggers highlight two key aspects of enterprise feedback management: 1) transforming surveys from projects and integrating them into processes and 2) distributing survey data to employees. Typically, a web survey of customers is set up with triggers or alerts that are fired off to staff each time a respondent gives a low rating to customer satisfaction or loyalty questions. For instance, through an automated process, each call to a contact center generates a follow-up email survey within 24 hours of the issue being marked as resolved. This survey is short and—behind the scenes, thanks to CRM integration—includes data about the transaction being rated. If the customer rates the service poorly on one of several key measures, an email is triggered: this notification of a poor rating is sent to a contact center manager, and includes within it the customer’s answers to the survey as well as data about the customer, product line and call-center transaction. By receiving this notification moments after the customer has completed the survey, the manager is able to begin customer recovery with a call or email. The actual manager notified might vary depending on other fields contained within the survey, such as location of the contact center or location of the customer. Of course, the survey response is compiled and aggregated for reporting purposes, as with a traditional survey. However, thanks to the user of email triggers, measuring satisfaction has been transformed into intervening to improve that satisfaction.
Posted by Jeffrey Henning on Fri, Jun 19, 2009
If you are simply developing a list of choices to a choose-one question, and those choices have no relative relationship to one another, than you are not developing a rating scale. When you report on these choices, you will simply report on the frequency with which each choice was selected and highlight the most frequently selected choices. Use these choose-one best practices to come up with the appropriate choice list. For a rating scale, on the other hand, you want each label to represent a standard interval from one another. You plan on reporting on the arithmetic mean of the answers, not just choice frequencies, and you may try to discover correlations between the numeric rating and other variables in the survey. How to label the scale depends in part on whether the scale is unipolar (ranging from 0% to 100% of a property) or bipolar (where the zero point is in the middle and the end points are opposites, such as “completely dissatisfied” and “completely satisfied”). If you are developing a unipolar rating scale, use a five-point numeric scale such as 0 to 4 or 1 to 5, choosing a label for each point. A common approach to unipolar scales follows this wording: - Not at all cromulent
- Slightly cromulent
- Somewhat cromulent
- Moderately cromulent
- Extremely cromulent
For a bipolar rating scale, use a seven-point scale ranging from -3 to 3, choosing a label for each point. Bipolar rating scales are easier to write, as the wording should be in parallel for positive and negative items with the same absolute value. For instance, for measuring satisfaction, a good bipolar scale is: - Completely dissatisfied
- Mostly dissatisfied
- Somewhat dissatisfied
- Neither satisfied nor dissatisfied
- Somewhat satisfied
- Mostly satisfied
- Completely satisfied
Purists typically insist that the midpoint take the form “Neither satisfied nor dissatisfied” but others prefer to label the midpoint “Neutral” for succinctness. If you want to use a different word or phrase for each label, take care that the words are approximately equally apart. For instance, Jon Krosnick and Leandre Fabrigar in “ Designing rating scales for effective measurement in surveys” summarize the results of four studies into scale values for labels assessing liking. Clearly, the scale “Very Poor, Poor, Fair, Good, Excellent” does a good (but not excellent!) job of spacing out each label. To develop an original scale such as this requires pre-testing and is probably inappropriate for most business researchers to attempt. In those cases where no other rating scale will do, instead use a five-point unipolar scale with just the endpoints labeled or a seven-point bipolar scale with the endpoints and midpoint labeled. While not ideal, and against best practices, you are less likely to go wrong using such an approach than simply making up a scale of your own.
Posted by Jeffrey Henning on Wed, Jun 17, 2009
 While I believe that the scales with the highest reliability and validity are 5-point unipolar scales and 7-point bipolar scales, and I prefer using fully labeled scales, that is for gathering information from respondents. Presenting information to users of your survey research is a different matter. Simply presenting scales designed for accurate data collection may not facilitate ease of understanding of the results.For instance, since 7-point scales are rare in business use, readers of your report may have difficulty understanding the results if you use a 7-point scale. If you are using a mix of 5- and 7-point scales when reporting results, you are certain to confuse some of your readers. If you are doing a cross-survey analysis where similar questions in different surveys used different scales (see Standardize Your Customer Satisfaction Questions & Rating Scales), you will definitely want to standardize your scales. For presenting data, I typically prefer to map scales to a 0-10 scale. I find that the business professionals I present to intuitively understand this scale. The broader range of values also makes it easier for readers to see differences in the results. For instance, I would typically not present the following, even though this is an accurate representation of the gathered results. Rating of Attributes on a 1-5 Scale
from 1 = Not at all important to 5 = Extremely important | Functionality of product | 4.7 | | Product learning curve | 4.6 | | Quality of technical support | 4.4 | | Ability to grow with product line | 3.9 | | Price of product | 3.6 | | Availability of free trial | 3.5 | | Helpful sales representative | | |