Employee Engagement vs. Employee Satisfaction</h1> <article> <h1>Employee Engagement vs. Employee Satisfaction</h1> <p>Jeffrey Henning — Fri, 20 Nov 2009 14:35:00 GMT</p> <P>Searches on "employee satisfaction" have dropped by half since 2004, while "employee engagement" has come from nowhere to surpass "employee satisfaction" searches in 2009.</P> <P align=center></P> <P>Whenever I talk about <A href="http://blog.vovici.com/blog/bid/22875/Employee-Engagement-Definition" mce_href="http://blog.vovici.com/blog/bid/22875/Employee-Engagement-Definition">employee engagement</A>, I hear very different reactions:</P> <OL> <LI>"<EM>Employee engagement</EM> is just the latest buzzword for HR staff to embrace. It offers nothing materially different from employee-satisfaction research. It's just a way for consultants to sell last year's fashions at marked-up prices."</LI> <LI>"<EM>Employee engagement</EM> represents an important breakthrough that links employee satisfaction to business outcomes. All employee-satisfaction research should become employee-engagement research."</LI></OL> <P>One of the techniques I was taught for moderating focus groups is to assume that what the speaker says is true, from their perspective, and therefore to try to understand that perspective. So let's assume for the sake of argument that both of the above statements are true. </P> <P>How might they both be true? Employee engagement makes <EM>explicit</EM> best practices for employee satisfaction that many researchers have followed <EM>implicitly</EM>. </P> <UL class=unIndentedList> <LI><STRONG><EM>Professional satisfaction research projects always focused on engagement</EM></STRONG>. <A href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx" mce_href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx">Employee-satisfaction</A> research has traditionally been a measure of employees' emotional and rational satisfaction with their job and employer. Thorough researchers have always analyzed employee satisfaction as a key driver of employee retention, customer loyalty and other positive business outcomes.</LI> <LI><STRONG><EM>Many satisfaction projects just measure satisfaction, rather than act on it</EM></STRONG>. Far too many organizations do employee-satisfaction surveys simply because they feel they need to. They make tactical changes based on the feedback, but that's it. They aren't linking <A href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty" mce_href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty">employee satisfaction to customer satisfaction</A>; they aren't linking employee engagement to quality.</LI></UL> <P>So, if you've done employee-satisfaction research strategically, employee engagement to you is just a new label for an old practice. But if you've done employee-satisfaction research tactically in the past, employee engagement represents a new best practice for you to aspire to in your research.</P> <P>This evolution is analogous to how <A href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx" mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx">customer satisfaction</A> research has evolved into customer loyalty research. Good customer satisfaction studies always looked at loyalty, but now that connection is explicit in the new label for the domain.</P> <P>So for those of you with buzzword backlash, embrace the term <EM>employee engagement</EM>: point out the practices you've done in the past that differentiate such research from tactical employee-satisfaction studies, and help us all to do a better job conducting research that will help us satisfy and engage employees to create desired organizational outcomes.</P> </article> <article> <h1>SharePoint as Survey Software</h1> <p>Justin Corrado — Thu, 19 Nov 2009 16:21:00 GMT</p> <P>We often have prospects looking to migrate to a <A href="http://www.vovici.com/" mce_href="http://www.vovici.com">survey software</A> application from a home-grown system. Recently, we've been seeing more organizations looking to move from Microsoft SharePoint, an enterprise content management platform.</P> <P>What clients liked about SharePoint when it comes to building online surveys:</P> <UL class=unIndentedList> <LI>The WYSIWYG form editor makes it easy to build basic surveys.</LI> <LI>It offers complete flexibility for programmers to use HTML, JavaScript, AJAX techniques, Flash and other standard web development approaches.</LI> <LI>It stores data directly in the repository that the administrator selects (e.g., SQL Server or Oracle).</LI> <LI>Survey results are readily exported as CSV (Comma-Separated Value) or XLS files.</LI></UL>Here are the features that caused SharePoint users to move to a dedicated feedback platform:<BR> <UL class=unIndentedList> <LI>It is difficult for business users to create surveys of moderate or significant complexity with SharePoint.</LI> <LI>The lack of integrated email support meant that invitations, <A href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" mce_href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">reminders</A> and thank-you emails had to be created outside the system.</LI> <LI>The lack of <A href="http://blog.vovici.com/blog/bid/23636/Panel-Management-Explained" mce_href="http://blog.vovici.com/blog/bid/23636/Panel-Management-Explained">panel management</A> meant administrators couldn't target respondents and track past survey activity.</LI> <LI>It is difficult to personalize surveys based on known data.</LI> <LI>SharePoint lacks survey-specific web services, making integration more tedious.</LI> <LI>Some users reported security concerns.</LI> <LI>Until recently, there was no support for basic <A href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys" mce_href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys">skip patterns</A> (available with MOSS 2007) or advanced branching.</LI> <LI>It has no real-time reports or dashboards and can't export results to SAV or PPT files.</LI></UL>SharePoint is a great interim solution for organizations first adopting web surveys. It excels at short, uncomplicated and static surveys, but organizations outgrow it when they need advanced questionnaires, intermediate MR functionality, integrated email invitations, panel management or enterprise feedback management. </article> <article> <h1>Panel Management Explained</h1> <p>Jeffrey Henning — Wed, 18 Nov 2009 09:55:00 GMT</p> <p>A panel is a group of people with relevant backgrounds who agree to participate in surveys. Businesses can organize a panel for each group of key stakeholders: customers, employees, resellers, partners, prospects, etc.</p> <p>Because panelists agree in advance to participate in surveys and feedback efforts, they become almost a guaranteed source of information for the sponsoring organization. Customers typically participate in the panel because they value their relationship with the sponsor, and they appreciate the additional information, influence and early access that comes from participating in the panel. For general market panels, panelists instead often look to merchandise and cash rewards, though even for many general panels people participate because they want to make their views heard.</p> <p>How does a panel work? You invite an individual to participate in a panel and communicate the ground rules:</p> <ul class="unIndentedList"> <li>What they will be surveyed about</li> <li>How you'll use the information</li> <li>How frequently they'll be asked to participate in surveys</li> <li>Why it's important that they participate</li> <li>How they can opt-out if they change their mind</li> <li>What's in it for them</li></ul> <p>Once you gain their permission, you invite the panelist to complete a registration survey, which will gather detailed demographic or firmographic information. This information can be used to target individual surveys to panel subsegments and also provides for rich opportunities in cross-tabulating survey results.</p> <p><a href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" title="Vovici: Online Communities, and how they diffre from Survey Panels">In contrast to online communities, in a panel</a>, members communicate only with the sponsoring researcher, through the medium of the surveys they are sent. In online communities, members can engage in discussions with one another through an online portal.</p> <p>Want to learn how to set up your own customer panel? Download a complimentary copy of my white paper, <i><a href="http://www.vovici.com/forms/promos/signup-WP-blog20091118.aspx" target="_new" mce_href="http://www.vovici.com/forms/promos/signup-WP-blog20091118.aspx">Customers as Confidants: Customer Panel Management Made Easy</a></i>.</p><p style="padding: 4px; font-family: Arial,Helvetica,sans-serif; font-size: 11px; color: rgb(231, 231, 231); background-color: rgb(99, 97, 99); width: 420px; text-align: left;"> <b>Vovici Survey Software and Web Survey:</b> <a href="http://www.vovici.com/forms/signup-call.aspx?cta=blogEntry" style="text-decoration: underline;"><font color="#ffffff">Request a Call</font></a> l <a href="http://www.vovici.com/forms/signup-demo.aspx?cta=blogEntry" style="text-decoration: underline;"><font color="#ffffff">Request a Demo</font></a></p> </article> <article> <h1>Customer Experience Management Now a Core Differentiator</h1> <p>Brian Koma — Mon, 16 Nov 2009 13:14:00 GMT</p> <p>Interest in Customer Experience rose dramatically across organizations we surveyed as part of our Vovici <a href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices">CE IQ study</a>, increasing from 47% of respondents stating that Customer Experience Management (CEM) was very important to them in 2008 to 63% in 2009. Clearly the global economic contraction focused many organizations more tightly on understanding how customers are interacting with them, and this has put a dramatic new emphasis on CEM.</p> <p>In fact, much to our surprise, more than half, or 55%, of respondents say that customer experience is a core differentiator that distinguishes their business from others in their market. Customer experience has arrived, and is now as important as traditional differentiators such as quality, service and staff capabilities. Despite the global economic contraction, price was not frequently cited as a core differentiator (18%).</p> <p></p> <p>When asked to assess the current benefits of CEM, customer satisfaction and loyalty were the most important, cited by 52% of respondents. Next was providing greater positive word of mouth (50%) and excellence in customer service (42%).</p> <p></p> <p>Leading the areas where organizations could do better, providing greater consistency of experience at every customer touch point was mentioned by only 36% of respondents. This is an area for improvement, as excellence in Customer Experience Management should drive such behavior. Top-line and bottom-line benefits from CEM are currently experienced by only about a third of respondents (35% and 29% respectively).</p> <p>With organizations acknowledging how important CEM is to distinguish themselves from competitors, it is more important than ever to focus on those <a href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices">CEM best practices that build customer loyalty</a>.</p> <p><i><b>Note</b>: The survey of over 200 organizations around the world is based on a convenience sample designed to identify the impact of CE best practices on loyalty, in order to prioritize those best practices. These results are illustrative and are not representative of any wider population of organizations. For more on the methodology, see <a href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings">Customer Experience Study Findings</a>.</i></p><p style="padding: 12px; font-family: Arial,Helvetica,sans-serif; font-size: 12px; color: rgb(231, 231, 231); background-color: rgb(99, 97, 99); width: 600px; text-align: center;"> <a href="http://www.vovici.com/forms/signup-call.aspx?cta=blogEntry" style="padding: 6px; text-decoration: underline; background-color: rgb(231, 231, 231);"><font color="#cc092f">Request a Call</font></a> <b>Vovici Survey Software and Web Survey</b> <a href="http://www.vovici.com/forms/signup-demo.aspx?cta=blogEntry" style="padding: 6px; text-decoration: underline; background-color: rgb(231, 231, 231);"><font color="#cc092f">Request a Demo</font></a> </p> </article> <article> <h1>Customer Loyalty Questions</h1> <p>Jeffrey Henning — Fri, 13 Nov 2009 09:11:00 GMT</p> <P><A href="http://www.amazon.com/Value-Profit-Chain-Employees-Customers/dp/0743225694" target=_new mce_href="http://www.amazon.com/Value-Profit-Chain-Employees-Customers/dp/0743225694"></A>When asked to measure loyalty, what questions should you ask? </P> <P>For instance, in <EM>The Value Profit Chain</EM>, Earl Sasser Jr. and his co-authors provide this list:</P> <UL class=unIndentedList> <LI>How likely are you to repurchase?</LI> <LI>How likely are you to tell someone else about your experience?</LI> <LI>How many times have you purchased (all products or services of this type) in the past 6 months (or some other period of time reflecting purchase frequency for product or service category)?</LI> <LI>How many times have you purchased (this product or service) from (our company) in the past 6 months?</LI> <LI>How many others have you told about your experience (with this product or service) in the past 6 months?</LI> <LI>How many of those that you have told in the past 6 months have, to your knowledge, also purchased (the product or service)?</LI> <LI>How many times have you offered constructive criticism or suggestions for product or service improvements over the past 6 months?</LI></UL>To this list can be added:<BR> <UL class=unIndentedList> <LI>How likely are you to repurchase if the price increases 10%?</LI> <LI>How likely are you to repurchase if a like competitor has a price that is 10% lower?</LI> <LI>How vital is the competitive advantage this product provides?</LI> <LI>What percentage of your spending on this product category is spent with us?</LI> <LI>When shopping for this product category, how often do you purchase from us instead of from other brands?</LI> <LI>How likely are you to continue purchasing the same products/services from us?</LI> <LI>How likely are you to purchase different products/services from us?</LI> <LI>How likely are you to increase the frequency of purchasing from us?</LI> <LI>How likely are you to switch to a different provider?</LI> <LI>How reluctant are you to switch your business to a competitor?</LI></UL> <P>You should measure the questions that you can <A href="http://blog.vovici.com/blog/bid/22720/Customer-Satisfaction-ROI-Analysis" mce_href="http://blog.vovici.com/blog/bid/22720/Customer-Satisfaction-ROI-Analysis">link to business outcomes</A>. This takes time and experimentation to find out what works best for your organization. Settling on one question prematurely because it works for others in a few industries is the wrong approach. For you, one set of questions might work better for one product line than another, which might need a completely different set of loyalty questions.</P> <P>Here are some sets of loyalty questions you should test:</P> <UL class=unIndentedList> <LI><A href="http://blog.vovici.com/blog/bid/18715/Forrester-Loyalty-Metrics">Forrester Loyalty Metrics</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/19953/Advocacy-Loyalty-Index-ALI-and-Purchasing-Loyalty-Index-PLI">Advocacy Loyalty Index & Purchasing Loyalty Index</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/20625/TNS-Customer-Loyalty-Index">TNS Customer Loyalty Index</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/21986/Measuring-Affective-Calculative-Commitment">Affective Commitment Index & Calculative Commitment Index</A> </LI></UL> </article> <article> <h1>Survey Compensation for Employees Gone Awry</h1> <p>Jeffrey Henning — Thu, 12 Nov 2009 12:28:00 GMT</p> <P>I'm adamantly against financial compensation for staff based on customer-service survey results. Invariably, it leads to customer-service agents gaming the system; this happens far more than executives are willing to acknowledge. For instance, a K-mart cashier posted a sign at the register saying, "For a chance to win a $2,500 gift card, log onto KmartFeedback.com and rate our customer service 9 or 10. Thanks for shopping at your 1 Penn Plaza Kmart!" Of course, people have a chance to win no matter what rating they give.</P> <P align=center><BR><EM>(photo: </EM><A href="http://www.reddit.com/user/bcurrie"><EM>bcurrie</EM></A><EM>)</EM></P> <P>A post on this example at <A href="http://consumerist.com/5366685/kmart-doles-out-helpful-customer-service-survey-suggestions" mce_href="http://consumerist.com/5366685/kmart-doles-out-helpful-customer-service-survey-suggestions">The Consumerist</A> led to a rash of comments giving examples for other major brands:</P> <UL> <LI><STRONG>CVS</STRONG> - "Yesterday I went to CVS for a prescription. When I checked out, the cashier looked at my receipt and stated that I had been selected to participate in a survey. Yeah, yeah, yeah, I know - this happens all the time, but she then showed me a mini candy bar and said, ‘you get a free Take5 candy bar so that will remind you to give us all 5's on the survey!' I thought it was pretty smart but I still didn't do the survey..."</LI> <LI><STRONG>Enterprise</STRONG> - "I was asked to rate an Enterprise rental by the customer rep, on a scale of 1-10 (or whatever it was). I said 9 because I figure one should save 10 for over-the-top service; this particular rental went smoothly, but was nothing special. He kept after me, wanting to know what was wrong, why I didn't give them a 10, until I got the point that to them ‘10' means ‘normal service'."</LI> <LI><STRONG>Home Depot</STRONG> - "The Home Depots around me [hand you a flyer] telling you the almost exact same thing."</LI> <LI><STRONG>Nissan</STRONG> - "This reminds me of whenever I buy a car. The Nissan salesperson reminds me that I will receive a survey in the mail, and that I'm supposed to return that survey with all 5's (the best possible mark), or else their dealership will be dinged or knocked down some points or whatever. I don't generally return surveys anyway, and I certainly don't when someone tells me what my answers should be anyway."</LI> <LI><STRONG>Old Navy</STRONG> - "An Old Navy store stapled a piece of paper to my receipt with a similar suggestion - something like ‘rate us a 10 and get 10% off'. Of course you get 10% off for filling out the survey no matter what, so I filled out the survey honestly, with perhaps a slight negative influence from the suggestion."</LI> <LI><STRONG>Sears</STRONG> - "Did the same thing when I worked at Sears but it said all 10s and we were told to tell them all 10s or we'd fail. It would be nice if they used the surveys to better themselves."</LI> <LI><STRONG>Target</STRONG> - "I have had plenty of cashiers at Target say that sort of thing too."</LI> <LI><STRONG>Toyota</STRONG> - "A Toyota dealer actually told me to bring in the survey so we could 'fill it out together'. In exchange - he would fill my gas tank."</LI></UL>My recommendations: <BR> <UL class=unIndentedList> <LI><STRONG>Don't financially incentivize any staff based on survey results</STRONG>. Doing so will change the results, no matter how strong your corporate culture is (for instance, Enterprise would discipline the representative mentioned above). Compensating only managers will lead to managers asking representatives to ask for higher scores. The most important thing is to gather authentic feedback about current satisfaction.</LI> <LI><STRONG>Don't use survey results as a crutch for measuring employees</STRONG>. It's vital that you share rich, relevant feedback with agents so that they can serve customers better in the future. Their managers need to mentor them not overmeasure them. (See <A href="http://blog.vovici.com/blog/bid/20820/Employee-Customer-Engagement-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/20820/Employee-Customer-Engagement-Best-Practices">Employee-Customer Engagement Best Practices</A>.)</LI> <LI><STRONG>Don't use the Net Promoter Score as a transactional measure</STRONG>. From these and other anecdotes, it is clear that the <A href="http://blog.vovici.com/blog/bid/18260/Customer-Service-Survey-Template-using-NPS" mce_href="http://blog.vovici.com/blog/bid/18260/Customer-Service-Survey-Template-using-NPS">NPS in customer-service settings</A> leads to a collapse in the range and validity of the scale and an obsession with the top score, even for organizations like Enterprise that try to counterbalance this.</LI> <LI><STRONG>Don't use numbers</STRONG>. They are arbitrary; in some pure mathematical realm they are better for averaging than the results of <A href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">fully labeled scales</A> but the lower interrater reliability of numeric scales negates that. What an 8 means on a 10-point scale varies from respondent to respondent. Use labels, as in these <A href="http://blog.vovici.com/blog/bid/18261/Common-Rating-Scales-to-Use-when-Writing-Questions" mce_href="http://blog.vovici.com/blog/bid/18261/Common-Rating-Scales-to-Use-when-Writing-Questions">common rating scales</A>, and make the highest labeled point hard to achieve: <EM>Not at all satisfied, Slightly satisfied, Moderately satisfied, Very satisfied, Completely satisfied</EM>.</LI></UL> <P>Do you have any examples of being asked to provide artificially high ratings to surveys?</P> </article> <article> <h1>Vovici Forms Foundation of Oracle Voice of the Customer Program</h1> <p>Jeffrey Henning — Wed, 11 Nov 2009 12:42:00 GMT</p> <P>Jeremy Whyte, director of customer feedback and reporting with Oracle Corporation, presented details of Oracle's extensive Voice of the Customer research program to the American Marketing Association in a <A href="http://www.vovici.com/about/research-webcasts.aspx" mce_href="http://www.vovici.com/about/research-webcasts.aspx">research webinar</A> on October 20. Surveys form the foundation of Oracle's <A href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies" mce_href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies">Voice of the Customer</A> program: listening to customer input through Vovici surveys provides "comprehensive feedback across the Oracle ecosystem and customer ownership lifecycle". Oracle conducts hundreds of surveys, grouped into relationship surveys, transactional surveys and targeted surveys.</P> <UL> <LI><STRONG>Relationship surveys</STRONG> with customers, partners and employees are the most strategic surveys, highlighting customer experience and loyalty drivers across cumulative contacts. Why survey employees in a Voice of the Customer program? Because Oracle has identified a correlation between employee satisfaction and customer satisfaction and prioritizes investments to improve employee satisfaction based on how that will drive increased customer loyalty.</LI> <LI><STRONG>Transactional surveys</STRONG> measure the quality of each service response by organization. Short term these <A href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" mce_href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">surveys trigger</A> immediate action on a customer-by-customer basis, and long term these drive operational improvements to improve service quality. <A href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey" mce_href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey">Transactional surveys</A> are conducted for technical support, customer service, consulting services, education services, sales win/losses, implementations and even M&A impact on customers.</LI> <LI><STRONG>Targeted surveys</STRONG> are used primarily for <A href="http://blog.vovici.com/blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR" mce_href="http://blog.vovici.com/blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR">qualitative research</A> to supplement relationship and transactional surveys. Sample applications include <A href="http://blog.vovici.com/blog/bid/21083/Competitive-Market-Research" mce_href="http://blog.vovici.com/blog/bid/21083/Competitive-Market-Research">competitive intelligence</A>, product and service planning, marketing, referencing, user group satisfaction and general market research.</LI></UL> <P>All of these survey results are combined with operational measures and financial outcomes in a <A href="http://blog.vovici.com/blog/bid/22720/Customer-Satisfaction-ROI-Analysis" mce_href="http://blog.vovici.com/blog/bid/22720/Customer-Satisfaction-ROI-Analysis">customer satisfaction linkage analysis</A>. As a foundational platform, taken together these surveys empower Oracle's wider Voice of the Customer program, which reaches beyond surveys for other additional types of feedback.</P> <P>How effective has this been? Oracle has improved customer satisfaction each fiscal year for five years running and its customers now demonstrate the greatest propensity to recommend that they ever have. If you talk to a large Oracle client, you will hear firsthand how they have seen Oracle adapt and improve to serve them better. To achieve similar results for your own organization, make surveys the foundation of your efforts.</P> </article> <article> <h1>Insights from the Gartner CRM Summit</h1> <p>Jeffrey Henning — Tue, 10 Nov 2009 15:21:00 GMT</p> <P><A href="http://www.gartner.com/" target=_new mce_href="http://www.gartner.com"></A>This year's North American <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner CRM Summit</A> highlighted important changes in technology and innovation, many driven by the rise of social media and social networks.</P> <P>If you divide the CRM market into <A href="http://blog.vovici.com/blog/bid/21579/CRM-Trichotomy-Operational-CRM-Analytical-CRM-Social-CRM" mce_href="http://blog.vovici.com/blog/bid/21579/CRM-Trichotomy-Operational-CRM-Analytical-CRM-Social-CRM">Operational CRM, Analytical CRM and Social CRM</A>, as Gartner does, then Social CRM has the smallest share of current spending (10%) but the highest share of growth in spending (60%). In fact, Gartner analyst Michael Maoz said that "Networks matter more to your success than any other initiative."</P> <P>When Jim Davies gave his <A href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying" mce_href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying">"State of EFM" address</A>, he too emphasized the rise in social technologies, emphasizing the integration of social feedback within enterprise feedback management, noting that market-research online communities are now a part of many EFM platforms, including Vovici. For Jim, the leaders of the EFM market remain Vovici and Confirmit, and he expects continued consolidation in the industry (Vovici has purchased three companies: Perseus, WebSurveyor and Surveyo, as well as Raosoft web-survey assets).</P> <P>Not all social technologies encourage innovation, though: research director Gareth Herschel reported that Gartner clients had seen marginal-to-no ROI from investments in <A href="http://blog.vovici.com/blog/bid/21872/Blog-Analysis-as-Market-Research" mce_href="http://blog.vovici.com/blog/bid/21872/Blog-Analysis-as-Market-Research">blog content analysis</A>, and Gartner instead advocates that its customers learn the language of the Voice of the Customer by asking more open-ended questions in surveys and then text mining the responses. As you innovate, you need to monitor your progress, and Herschel emphasized the need to select 5-9 <A href="http://blog.vovici.com/blog/bid/21646/Unlocking-Key-Performance-Indicators" mce_href="http://blog.vovici.com/blog/bid/21646/Unlocking-Key-Performance-Indicators">key performance indicators</A> from the hundreds of performance indicators your organization follows. In a separate presentation, Herschel said that organizations need to engage in <A href="http://blog.vovici.com/blog/bid/21734/Meta-Analysis-Who-Analyzes-the-Analysts" mce_href="http://blog.vovici.com/blog/bid/21734/Meta-Analysis-Who-Analyzes-the-Analysts">meta-analysis, analyzing their analysis process</A>, to make sure that they improve the quality of decisions that they make in the future.</P> <P>Kathy Harris, a Gartner distinguished analyst, provided a good summary of the conference. She asserted that "<A href="http://blog.vovici.com/blog/bid/21676/Innovation-Your-New-Core-Competency" mce_href="http://blog.vovici.com/blog/bid/21676/Innovation-Your-New-Core-Competency">CRM is the most fruitful innovation opportunity</A> for your company and your customers"; make sure to use EFM (which Gartner sees as a subset of CRM) as you "embrace your customers in co-creation."</P> </article> <article> <h1>Social Media Market Research: A Study of the Tropicana Repackaging</h1> <p>Jeffrey Henning — Mon, 09 Nov 2009 11:29:00 GMT</p> <P><A href="http://www.facebook.com/group.php?gid=47302346159" target=_new mce_href="http://www.facebook.com/group.php?gid=47302346159"></A>At the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">2009 MRA First Outlook Conference</A>, Owen Shapiro of LJS Associates and Tom Malkin of GeeYee, Inc. presented "The Impact of Social Media on Market Research". In order to better understand the strengths and weaknesses of social media research, LJS conducted phone and social media research into the Tropicana unrepackaging incident.</P> <P>Earlier this year, <A href="http://www.nytimes.com/2009/02/23/business/media/23adcol.html?_r=3&pagewanted=1&ref=business" mce_href="http://www.nytimes.com/2009/02/23/business/media/23adcol.html?_r=3&pagewanted=1&ref=business">Tropicana rolled out repackaging</A> that many shoppers complained looked like a generic brand and made the product much harder to find on store shelves. The resulting pushback led to Tropicana to revert to its earlier packaging.</P> <P>To study this, LJS surveyed 1,000 U.S. consumers by phone and online and then weighted the results to be demographically representative. Fully 20% of consumers surveyed had noticed the new Tropicana packaging; of these, 32% liked the packaging, 32% were neutral and 27% were negative. Only 1% of consumers had posted online about the packaging.</P> <P>For social media research, GeeYee scraped 1,900 posts from over one million web pages analyzed; each post typically covered three or four topics. Here is the categorization of the most frequently mentioned topics:</P> <UL class=unIndentedList> <LI>Price - Primarily discussions of coupons and sales</LI> <LI>Premium OJ - Discussions of comparatively higher cost</LI> <LI>Ingredients - For instance, the shift to Brazilian rather than Florida oranges</LI> <LI>Confusion in store - Fully 34% of posters discussed not being able to find the product in stores</LI> <LI>New package design - Discussions of this accelerated once the packaging was withdrawn</LI> <LI>Reversion to "classic" packaging - Posters commented about Tropicana's announcement or upon seeing the old packaging restored</LI></UL>Comparing surveys to social media is like comparing apples to orange juice (sorry):<BR> <UL class=unIndentedList> <LI><STRONG>Social media's impact is disproportionate to the level of activity</STRONG>. The multiplier effect for each posting produces an impact often missed by traditional marketing metrics. Lost in a typical survey is how vocal those who disagree might end up becoming.</LI> <LI><STRONG>The data is dirty</STRONG>. When it comes to social media, you can't distinguish between grassroots and "Astroturf" (sponsored marketing campaigns design to mimic grassroots efforts), between legitimate blogs and "splogs" (spam blogs), between memes being relayed because they resonate or because they reciprocate (blog-post exchanges).</LI> <LI><STRONG>Brands with unique names are easier to research</STRONG>. It is easier to extract quotes about the iPhone than Coach. Researching Tropicana required removing references to the Tropicana casinos in Las Vegas and Atlantic City, a level of cleaning not needed in coding verbatim survey responses.</LI> <LI><STRONG>Topics are not always organized to answer our questions</STRONG>. Since posters can discuss anything, you need to develop a much broader coding sheet than for open-ended survey questions, which are tightly focused. Sometimes social media may not be talking about your question at all.</LI> <LI><STRONG>Social media is not representative, only directional</STRONG>. You can try to quantify it to better understand the direction but it will still be qualitative.</LI></UL>Owen concluded that social media is best used early in the market research process: to develop hypotheses to test with surveys or to use up front as a qualitative tool to more fully understand issues and to see what you might have overlooked. </article> <article> <h1>General Mills Moving Qualitative Research Online</h1> <p>Jeffrey Henning — Fri, 06 Nov 2009 14:10:00 GMT</p> <P>Ned Winsborough, manager of consumer networks at General Mills, presented "Accelerating Innovation with Social Networks" at the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">MRA First Outlook Conference</A>. "We have a mandate at General Mills to move as much of our qualitative research online as possible in the coming months and years. We have been experimenting with this for a year, but we created our consumer networks team this summer and are now scaling it." (<EM>Consumer networks</EM> is the term that General Mills uses for MROCs.)</P> <P>General Mills has done 22 community projects since last spring. Why online communities? "Online consumer communities meet the needs of consumers, brand teams and agencies with busy lives. They allow you to innovate with consumers better, faster, and cheaper." With communities, General Mills is able to engage in iterative building of concepts: "We listen, we build; we listen, we tweak. This can be done very quickly, with a lot of flexibility to the method." Community research allows for faster speed to market. For one project, General Mills did six months of work in six weeks. Compared to other qualitative methods, communities are less expensive. "There is a fixed cost for setting up the communities, which can be very significant, but the incremental cost of doing extra weeks, extra moderation, is very low."</P> <P>As a result of General Mills' 22 projects, they have made changes to their approach to community research:</P> <UL class=unIndentedList> <LI><STRONG>Focus on Discovery</STRONG> - The General Mills innovation model uses three steps: Discover, Build, Launch. The communities are great for Discovery but less suited for the Build phase. In the Discovery phase, community research always works, according to Ned, whether the project is big or small, whether the tolerance for risk is high or low. In the Build phase, small projects can be supported with community research but larger projects require traditional quantitative research. For future community research, "we are focusing on Discovery."</LI> <LI><STRONG>Smaller Communities </STRONG>- Early communities were larger (for example, 225 participants), but that produced too much information to quickly and easily analyze. "Now we work with communities of 30 to 50 people (more if we have subgroups we want to investigate). With fewer members, we really get to know them as individuals, and we can probe better."</LI> <LI><STRONG>Shorter Duration Communities </STRONG>- General Mills has moved from a permanent online community to project-based communities that last for six to eight weeks. "This is a different model than creating one ongoing community. We have some experience with that type of community: we had done that in the past but found it wasn't cost effective." The ongoing moderation activities can be significant, yet "it is rare that we have things that we need to do every week."</LI> <LI><STRONG>Larger Incentives </STRONG>- Members to an early community were offered $50 for six weeks participation and a chance to win some modest prizes. Current incentives tend to run $40-50 per week.</LI> <LI><STRONG>Geographically Centered </STRONG>- For one of its first project communities, General Mills invited seven local participants to come to their facility for shelf tests and project packaging tests. Now, General Mills "uses focus group facilities to recruit members, so that we can do selective face to face research."</LI></UL>Ned has heard everything from "traditional research is dead" to skepticism about the value of online community research. "The truth is in the middle," he said. "It has a place, and we need to approach it like any other new technology. What questions can it answer? What objectives can it meet? What objectives can't it meet? Where can it fit in an array of methods? It certainly doesn't obsolete core quantitative methods but it has powerful potential to transform qualitative research as we know it." </article> <article> <h1>MROC Case Study from ABC Studios</h1> <p>Jeffrey Henning — Thu, 05 Nov 2009 21:57:00 GMT</p> <P>Karen Manne, VP of research with Disney, presented "Journey inside the ABC Studios Advisory Panel" at the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">MRA First Outlook Conference</A>. <A href="http://en.wikipedia.org/wiki/ABC_Studios" target=_new mce_href="http://en.wikipedia.org/wiki/ABC_Studios">ABC Studios</A> is the production company for the Disney television group, producing shows such as <EM>Brothers & Sisters, Desperate Housewives </EM>and <EM>Grey's Anatomy</EM>, as well as <EM>Castle</EM>, <EM>FlashForward</EM>, <EM>Legend of the Seeker </EM>and <EM>Lost</EM>, among others. "We started building this community three years ago," said Karen. "ASAP (the ABC Studios Advisory Panel) was the first online community at Disney and the first for program planning in the TV industry." </P> <P>The community currently has 1,900 members, who are each heavy viewers of at least two ABC Studios shows and who are opinion leaders: people who are passionate about television and regularly talk to their families and friends about the shows they watch. Membership fluctuates, as members who don't log in for at least three months are purged occasionally; a purge six months ago reduced membership to 1,400. Unfortunately, the panel is not gender balanced: 86% of the members are female; Karen recently went to ComicCon to unsuccessfully recruit more men for the panel, but most men only watch one of the ABC Studios shows.</P> <P>Members are not given monetary incentives at all, but participate because they want to have a hand in shaping TV programming. They are sometimes given digital access to TV shows and also see sneak peeks of shows.</P> <P>For a time, members could refer friends to the community, but Karen has stopped that practice. Too many referrals were skewing some of the research, as members invited others with identical views on particular characters or aspects of the shows.</P> <P>Here are some anecdotes about the community by show:</P> <UL class=unIndentedList> <LI><EM>Brothers & Sisters </EM>- The producers were interested in viewer opinions of several of the male characters; rather than tip their hand into which characters they were most interested in, they did a general study of all the male characters on the show.</LI> <LI><EM>Castle </EM>- As an entertainment company, ABC Studios is able to provide unusual rewards to its members. Three heavy contributors to the site were invited to a book signing of <EM>Heat Wave</EM>, a real book marketed as if it were written by the fictional Richard Castle (played by <A href="http://en.wikipedia.org/wiki/Nathan_Fillion" target=_new mce_href="http://en.wikipedia.org/wiki/Nathan_Fillion">Nathan Fillion</A>, of Firefly). These community members were given the VIP treatment and were photographed meeting Fillion; this was then publicized in the community.</LI> <LI><EM>Grey's Anatomy </EM>- One problem Karen has experienced is the occasional leak of sensitive information from the panel. A poll about the character Lexie was released to the public, for instance. As a result, for some sensitive polls, respondents are no longer shown the results. ("Polls are a favorite of members, since they're a quick and easy way to provide feedback.")</LI> <LI><EM>Scrubs </EM>- ABC bought the rights to this NBC show and is relaunching it with an altered premise this season. The season premiere was uploaded for community members, who were encouraged to watch the whole episode and provide feedback on the significant changes to the story. Initial viewer reaction was positive.</LI> <LI><EM>Ugly Betty</EM> - After three years of working at the fashion magazine, Betty is finally getting a makeover, and ASAP members reviewed seven possible new looks. Through leaks from the community, this lead to fevered coverage in blogs (<A href="http://perezhilton.com/2009-07-31-de-uglify" mce_href="http://perezhilton.com/2009-07-31-de-uglify">Perez Hilton: De Uglifying Betty</A>) and the entertainment press, and finally, even, the Wall Street Journal ("<A href="http://online.wsj.com/article/SB10001424052748704107204574469733432844964.html" mce_href="http://online.wsj.com/article/SB10001424052748704107204574469733432844964.html ">Making Ugly Betty Prettier</A>: To gauge viewer reaction, ABC turns to online focus groups to test its star").</LI></UL>For ABC Studios, the benefits of ASAP are many:<BR> <UL class=unIndentedList> <LI>Provides easy access to consumers</LI> <LI>Yields quick feedback on insights and attitudes</LI> <LI>"Spontaneity allows for flexibility"</LI> <LI>Targeted research</LI> <LI>Viral marketing </LI></UL> <P>Many research projects are quite small and targeted, leading to shorter, more focused questionnaires. Karen has done literally a 1000 projects in the community. </P> <P>Karen said, "I love my community but it is not all puppies and rainbows - it takes a lot of work." Some of the cons:</P> <UL class=unIndentedList> <LI>Unable to verify that members aren't reporters or competitors</LI> <LI>Requires ongoing investment of time and money to recruit new members</LI> <LI>Leaks of sensitive material to the entertainment blogs and press</LI> <LI>Piracy of episodes posted within the community</LI> <LI>Qualitative data is voluminous and time consuming to analyze</LI> <LI>Busy work - during off-production times (such as the summer) need to have community activities to keep members engaged for when they are really needed</LI> <LI>Victim of success - get pushed for rapid turnaround because executives realize the community enables it</LI></UL>For all its cons, Karen said the benefits outweigh the challenges. "Online communities are the hot new ticket in market research." </article> <article> <h1>From Bedrock to MROC: Member Activities beyond Discussions</h1> <p>Jeffrey Henning — Thu, 05 Nov 2009 18:58:00 GMT</p> <P>Jane Mount, PRC, an executive vice president with Digital Research, presented "From Bedrock to MROC: An Evolution in Qualitative Research Practices" at the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">2009 MRA First Outlook Conference</A>. Jane began by describing how quantitative methods over the past sixty years have evolved from door-to-door techniques, to direct mail, to telephone and to online research, while qualitative research has remained with focus groups during this time period. Even today online focus groups represent less than 10% of the qualitative market. MROCs, however, represent a substantial shift in qualitative research: "a shift from asking questions to get reactive consumer feedback, to listening to dialogue to get proactive consumer insight."</P> <P>Jane provided an excellent introduction to the topic of MROCs, covering familiar ground. (If you're new to MROCs, see my past posts on <A href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community" mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community">MROC = Market Research Online Community</A>, <A href="http://blog.vovici.com/blog/bid/17983/Focus-Groups-vs-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/17983/Focus-Groups-vs-Online-Communities">Focus Groups vs. Online Communities</A> and <A href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">Social Networks vs. Online Communities vs. Panels</A>.)</P> <P>Jane presented the MROC market as a continuum ranging from full-service suppliers such as her firm, Digital Research, on one side to technology-only suppliers such as Vovici on the other side. [She showed a couple of her competitors, and a couple Vovici competitors, all of whom I have happily omitted from my recreation of her slide!]</P> <P></P> <P>Frequently researchers think of MROCs as simply "listening posts", a place to facilitate discussions and eavesdrop on conversations, but - as a full-service supplier - Jane talked about the many other research activities that MROC members can do:</P> <UL class=unIndentedList> <LI><STRONG>Static ethnography</STRONG> - Upload photos from their personal life.</LI> <LI><STRONG>Representational images</STRONG> - Upload clip art or a photo that represents a topic. For a sensitive topic like body image, have them submit these privately. For a fun topic like perceptions of their in-laws, have them upload the image for all to see and comment on (one member uploaded a picture of a cactus with the caption "They're prickly" to describe their in-laws, inspiring a comment thread from others who agreed).</LI> <LI><STRONG>Idea banks </STRONG>- Submit ideas to a shared database where they can rate them.</LI> <LI><STRONG>Insight games </STRONG>- Play word-association exercises and MadLibs-style sentence completion games.</LI> <LI><STRONG>Cartoon captions </STRONG>- Write a caption for a cartoon.</LI> <LI><STRONG>Personal diaries </STRONG>- Record daily activities, providing a richer narrative than possible through a one-time survey; for instance, revealing how members struggle with dieting on a day-to-day basis.</LI> <LI><STRONG>Collages </STRONG>- Assemble collages that represent the topic being researched.</LI> <LI><STRONG>Fun polls & quizzes -</STRONG> Answer entertaining questions like "If the economy was a candy bar, which of these candy bars would it be?"</LI> <LI><STRONG>Team activities</STRONG> - Do planned exercises with others. The research team segments users upon registration and then plans team activities where each team represents a different segmentation. For instance, DRI did a traditional quantitative study that produced six segments of consumer buying behavior, then invited those respondents into the community and recorded their segmentation.</LI></UL> <P>Jane said that issues suitable for research with MROCs include "ideation; testing social media strategy; trend spotting; early stage evaluation of branding, packaging, ads; exploring attitudes and behaviors; directional insights when time is critical; and testing suitable language for a target." To her mind, MROCs are a very cost-effective method for qualitative research that is gaining in popularity because they are fast, provide ongoing insight generation, are highly creative, and their tech-intensiveness matches with respondent lifestyles today.</P> <P>MROCs are now part of "a modern Stone Age family" of qualitative tools.</P> <P align=right><EM>Photo credit: </EM><EM>© 2008 <A href="http://www.flickr.com/photos/matthigh/" target=_new mce_href="http://www.flickr.com/photos/matthigh/">Mike Hill</A></EM></P> </article> <article> <h1>Multicultural Market Research: Eight Make-or-Break Rules</h1> <p>Jeffrey Henning — Thu, 05 Nov 2009 12:43:00 GMT</p> <A href="http://www.amazon.com/Multicultural-Intelligence-Make-Break-Orientation/dp/0980174562/" target=_new mce_href="http://www.amazon.com/Multicultural-Intelligence-Make-Break-Orientation/dp/0980174562/"></A>David R. Morse, president of New American Dimensions, discussed his new book <EM>Multicultural Intelligence: Eight Make-or-Break Rules for Marketing to Race, Ethnicity, and Sexual Orientation</EM> at the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">MRA First Outlook Conference</A>. He argued that all U.S. researchers need to develop competency in multicultural research, given shifts such as these in American demographics: <BR> <UL class=unIndentedList> <LI>The percentage of foreign-born population in the United States has increased to 14%, a level not seen since 1910.</LI> <LI>Hispanic immigration is at an all-time high, and Hispanics will make up 20% of Americans by 2050. </LI> <LI>By 2044, white Americans will be a minority according to projections from the U.S. Census department. </LI></UL> <P>Here are David's eight rules:</P> <OL> <LI><STRONG>Boost your multicultural competency</STRONG>. The <A href="http://www.census.gov/" target=_new mce_href="http://www.census.gov">U.S. Census</A> is very detailed and useful for developing a detailed demographic understanding (though it lacks demographics by sexual orientation). When doing focus groups or qualitative research, make sure to have a moderator or interviewer of the race or sexual orientation being studied.</LI> <LI><STRONG>Divide and conquer</STRONG>. Don't accept the stereotypes ("Hispanics are brand loyal") but segment this population to truly understand how it relates to your market. The <A href="http://www.bls.census.gov/cps" target=_new mce_href="http://www.bls.census.gov/cps">Current Population Survey</A> is one of the few resources that lets you segment population by generation. Foreign born (or first-generation Americans) are very different from second-generation Americans, who generally speak English as well as their parents' native language; second-generation Americans are different again from the third-generation Americans, who typically speak only English.</LI> <LI><STRONG>Don't trust the experts</STRONG>. The accepted wisdom is often full of persistent and incorrect "truths" and urban legends, especially relating to translation. Do your homework.</LI> <LI><STRONG>Don't let the joke be on you</STRONG>. Tread carefully when doing multicultural marketing with humor; what is funny differs dramatically by culture.</LI> <LI><STRONG>Don't get lost in the translation</STRONG>. Given the prevalence of translation errors, make certain to backtranslate the questionnaire. David once was surprised to see in a survey that 100% of Spanish-speaking Hispanics disagreed that "the Internet is color blind". Rechecking the Spanish translation, he found that it has been translated to mean "the Internet has red-blue-green color blindness".</LI> <LI><STRONG>Push their buttons</STRONG>. Find the cultural cues that people resonate with; use your research to determine what those are for your market. </LI> <LI><STRONG>Market on a <EM>wink</EM> and a prayer</STRONG>. In a mainstream ad, insert a subtle cultural reference. The mainstream will miss it, but the targeted culture will appreciate it.</LI> <LI><STRONG>Make up; don't cover up</STRONG>. Watch the watchdogs. Many segments have advocacy and anti-discrimination groups; if your marketing runs afoul of them, stop the campaign at once and apologize.</LI></OL> <P>America is not one culture, and savvy researchers realize this. "We are not a color-blind society," said David. "We live in different worlds. We talk differently. We listen to different kinds of music. We worship differently." Follow David's eight rules to improve your organization's marketing and market research.</P> </article> <article> <h1>Market Research at Microsoft: Evolution of the MR Department</h1> <p>Jeffrey Henning — Wed, 04 Nov 2009 20:35:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">MRA First Outlook Conference</A> in San Diego, Reed Cundiff, senior director of central market research for Microsoft (and formerly an analyst with the Yankee Group), discussed how market research at the company has evolved and is evolving.</P> <P>Prior to the creation of the Central Market Research Insights team, researchers had existed alone or in pairs in many different departments across the company. Job descriptions varied significantly; researchers had no career path within Microsoft.<BR>Four years ago, the CMRI had just eight staff. It peaked at about 102 people in June, before being brought down by layoff to 97 staff. The upside of the downside, as it were, is that it has further accelerated the centralization of market research: departments that in the past were funding their own research are now turning to the central group instead. "We see that a lot of ad hoc research budgets have been cut; that is good for us, as there were many projects done outside of our research group." As a result, the organization is eliminating redundant and superfluous research expenditures.</P> <P>The vision for the central research group is "to be a driving force behind Microsoft's business and product strategy" with the mission of delivering "strategic, fact-based insights that drive Microsoft's most essential business decisions."</P> <P>The organization spends $80M to $110M annually on external research: the technology sector's largest research budget, according to Reed. Projects range from doing a market opportunity analysis for a v1 health care product to conducting a customer satisfaction survey with 100,000 respondents across 86 countries. CMRI devotes 3.5 FTEs to its research vendor management program, where they develop the preferred vendor list and do biannual reviews of the vendors (and ask the vendors to review the research managers they interact with). The result has been better use of outside vendors and consistent improvement. Before the new process, Microsoft would "run a pilot with a vendor, fall in love with that vendor, bury that vendor, then never do business with that vendor again!"</P> <P>CMRI's strategy has three key components: to "deliver integrated insights" (primary research with market analysis), to "be a trusted advisor" and to "display business acumen". As a result, what Microsoft wants and expects from its internal researchers is changing [worth a blog post of its own!]: researchers need to be more consultative and need to specialize in a few focus areas.</P> <P>In its research on research, the CMRI has adopted <A href="http://blog.vovici.com/blog/bid/18262/Six-Sigma-Survey-Projects" mce_href="http://blog.vovici.com/blog/bid/18262/Six-Sigma-Survey-Projects">Six Sigma</A>. "In the past fiscal year, we went through the Six Sigma process and we are reducing the number of defects study by study by study. We averaged 12 defects per final report in a six-month period to 3 the next period down to 2 most recently."</P> <P>"We need to seize the opportunity. We are moving through a lot of challenges but to drive a fact-based culture, the timing has never been better for us." In his concluding remarks, Reed said, "We spend millions of dollars that affects hundreds of millions of dollars in marketing spend that affects billions of dollars in revenue. We have to get it right."<BR></P> </article> <article> <h1>Market Research Regulation in 2010</h1> <p>Jeffrey Henning — Wed, 04 Nov 2009 17:38:00 GMT</p> At the <A href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009" mce_href="http://blog.vovici.com/blog/bid/23227/MRA-First-Outlook-Conference-2009">MRA First Outlook Conference</A>, Howard Feinberg (<A href="http://twitter.com/hfeinberg" target=_new mce_href="http://twitter.com/hfeinberg">@hfienberg</A>), director of government affairs of the Market Research Association, presented "Regulation Over the Horizon: Emerging Research Technologies & Modes and the Legal & Privacy Hurdles". Howard covered six broad trends that have implications for the future of market-research regulation.<BR> <UL> <LI><STRONG>Online behavioral tracking </STRONG>- FTC proposes self-regulatory standards, which means "you need to regulate yourselves, or we will do it for you". Facebook recently settled a lawsuit about Beacon, which was considered to be in violation of established, offline privacy laws. Public opinion about behavioral tracking is mixed: users prefer free web content, paid for by ads, to subscription websites and are aware that information about their web browsing is collected, but are disturbed by the "creepiness" factor when they are unsure about what information is collected or how it is disseminated. In a recent survey, 92% of respondents felt that there should be a law requiring websites and ad servers to delete all information about them upon request.</LI> <LI><STRONG>Location and behavioral data</STRONG> - A little appreciated downside of our connectivity is that we've given up locational privacy, as we are tracked by our cellphones and GPS devices in our cars (if you know someone's home and work address you can identify their GPS data stream); sites like Google Latitude and Loopt provide information about where opt-in users are in real time. Locational data is not just about where you go, since those locations demonstrate associations - "political, religious, amicable and amorous, to name only a few," according to a New York State Court of Appeals ruling in May. Even with GPS off, phones can triangulate location from cell towers. Other developments that provide the MR industry great opportunities for observational research from data aggregation:</LI> <UL> <LI>Car insurance companies in California were recently given approval to charge rates based on travel patterns and mileage.</LI> <LI>Feinberg joked about RFID tagging of research participants, pointing out that RFID-tagged passports are now used at U.S. border crossings.</LI> <LI>A smart grid for power management of networked appliances and HVAC systems could provide insight into intimate consumer usage of appliances.</LI> <LI>UK billboards photograph license-plate numbers to look up the make and model of the vehicle, showing an ad for the exact motor oil required by that vehicle.<BR></LI></UL> <LI><STRONG>Social media</STRONG> - Scraping social media is limited by concerns about data quality, which prompts questions about who these people really are. From an ethical standpoint, are users aware that you are watching, listening, reading, analyzing their output? Teens and tweens are often naïve about the privacy of their data. New laws are regulating social networks, without defining social networks; this could have implications for researchers, if online focus groups and online research communities become classified and regulated as social networks. [See <A href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">Social Networks vs. Online Communities vs. Panels</A> for my definition.]</LI> <LI><STRONG>Respondent authentication</STRONG> -One survey found that web users were open to authentication by trusted vendors. No U.S. law currently governs digital fingerprinting, which is used by many panels for respondent authentication. In Europe, an IP address may constitute protected PII (Personally Identifiable Information).</LI> <LI><STRONG>Cloud computing</STRONG> - The provision of data through servers accessible over the Internet raises risks for data security; Howard advised researchers to make certain to download and backup data stored on such systems. How do international data transfer laws relate to cloud computing? This is yet to be decided.</LI> <LI><STRONG>Ethical and legal initiatives</STRONG> - For self-regulation of the industry, researchers are urged to provide transparency and consumer control over data, with limits on data retention. A standard practice, widely adopted, is get consumer consent whenever a privacy policy is changed. An additional practice is to get consent for using "sensitive" data. Best practices for location-based services include providing notice, requesting consent and implementing safeguards. For research authentication, notice, consent and efficiency. U.S. researchers should seek to follow the FTC's <A href="http://www.ftc.gov/reports/privacy3/fairinfo.shtm" target=_new mce_href="http://www.ftc.gov/reports/privacy3/fairinfo.shtm">Fair Information Practice Principles</A>.</LI></UL>Clearly, researchers prefer self-regulation to external regulation and Howard encourages researchers to work with him to shape best practices and influence legislation. </article> <article> <h1>MRA First Outlook Conference 2009</h1> <p>Jeffrey Henning — Wed, 04 Nov 2009 17:35:00 GMT</p> <P>Recaps from a few of the sessions of the <A href="http://www.mra-net.org/" target=_new mce_href="http://www.mra-net.org">Marketing Research Association's</A> First Outlook Conference in San Diego:</P> <UL> <LI><A href="http://blog.vovici.com/blog/bid/23228/Market-Research-Regulation-in-2010" mce_href="http://blog.vovici.com/blog/bid/23228/Market-Research-Regulation-in-2010">Market Research Regulation in 2010</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/23231/Market-Research-at-Microsoft-Evolution-of-the-MR-Department">Market Research at Microsoft: Evolution of the MR Department</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/23244/Multicultural-Market-Research-Eight-Make-or-Break-Rules" mce_href="http://blog.vovici.com/blog/bid/23244/Multicultural-Market-Research-Eight-Make-or-Break-Rules">Multicultural Market Research: Eight Make or Break Rules</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/23263/From-Bedrock-to-MROC-Member-Activities-beyond-Discussions" mce_href="http://blog.vovici.com/blog/bid/23263/From-Bedrock-to-MROC-Member-Activities-beyond-Discussions">From Bedrock to MROC: Member Activities beyond Discussions</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/23269/MROC-Case-Study-from-ABC-Studios">MROC Case Study from ABC Studios</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/23313/General-Mills-Moving-Qualitative-Research-Online">General Mills Moving Qualitative Research Online</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/23361/Social-Media-Market-Research-A-Study-of-the-Tropicana-Repackaging">Social Media Market Research: A Study of the Tropicana Repackaging</A> </LI></UL> </article> <article> <h1>Replying to the Voice of the Customer: A Twitter Experiment</h1> <p>Jeffrey Henning — Tue, 03 Nov 2009 15:47:00 GMT</p> <P>Six weeks ago, as an experiment, I set up a new Twitter account (my main Twitter account is <A href="http://twitter.com/jhenning" mce_href="http://twitter.com/jhenning">@jhenning</A>) to tweet my personal experiences with products, services and establishments. Since about a third of my tweets would be about local establishments, I sought out and followed about 100 other Twitter users near me; about 20 followed me back.</P> <P>That done, I then tried to make at least one comment each business day. Each tweet reflected an authentic experience: some were positive comments, some negative, some mixed. I wrote about 30 local, regional and national brands. </P> <P>My expectation was to do an analysis of brand response by scale of brand and by type of tweet (positive, negative, mixed). Unfortunately, only one brand - a regional brand - ever replied to me. So this makes for a rather boring statistical analysis! </P> <P>The results shocked me - I rarely tweet about personal brand experiences from @jhenning but the one time I did, the retailer responded to me right away. I had expected a fifth to a third of the brands to respond to me and had hypothesized that regional brands would have the greatest participation rate, as they are big enough to monitor social media and small enough to be early adopters of new technology.</P> <P>So, if your organization is out there listening on Twitter, it is time to speak up as well. The inaugural survey of the <A href="http://blog.vovici.com/blog/bid/22890/Social-Media-Trends-around-the-World" mce_href="http://blog.vovici.com/blog/bid/22890/Social-Media-Trends-around-the-World">Global Web Index</A> (a syndicated research offering from TrendStream) reported that 22% of its 16,000 panelists said that their perception of a brand is improved if the organization responds to comments in online communities and forums. Sometimes listening to the <A href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition" mce_href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition">voice of the customer</A> isn't enough, sometimes <A href="http://blog.vovici.com/blog/bid/19075/VOC-ROI-The-Return-on-Investment-of-Voice-of-the-Customer" mce_href="http://blog.vovici.com/blog/bid/19075/VOC-ROI-The-Return-on-Investment-of-Voice-of-the-Customer">acting on the voice of the customer</A> isn't enough: sometimes you have to reply to the voice of the customer.</P> </article> <article> <h1>Perceived Questionnaire Length</h1> <p>Jeffrey Henning — Mon, 02 Nov 2009 15:48:00 GMT</p> <P>Back in 2002, Mirta Galešic of the University of Zagreb wrote an interesting paper that examined respondents' perception of questionnaire length, "<A href="http://mrav.ffzg.hr/mirta/Galesic_handout_GOR2002.pdf" target=_new mce_href="http://mrav.ffzg.hr/mirta/Galesic_handout_GOR2002.pdf">Effects of questionnaire length on response rates</A>: Review of findings and guidelines for future research". If objects in a side-view mirror are closer than they appear, then questionnaires appear to respondents to be longer than they actually are.</P> <P>Galešic analyzed the relationship between objective and subjective questionnaire length. For objective length, she used the number of questions actually answered (to keep it simple, she treated each question as a question, regardless of its length or type). For subjective length, respondents were asked to rate the questionnaire they had just completed as «too short», «optimal», «somewhat too long» or «absolutely too long» (actual labels were in Croatian, as was the entire questionnaire). Not a single one of the 2,059 respondents answered «too short»!</P> <P>Galešic writes: </P> <BLOCKQUOTE> <P>Across all three questionnaire types there was an overall significant, but very small positive correlation between number of questions the respondents answered and their perception of questionnaire length (r=0.11, p<.01). Perceived length was more strongly correlated to the level of interest for the questionnaire topic (r=-.26, p<.01). The less interesting the questionnaire topic was, the longer the questionnaire was perceived to be. Level of interest for the questionnaire topic was not correlated to the number of questions answered (r=.03, p>.05).<BR>Interestingly, however, respondents who had less interest in the topic judged the questionnaire equally long no matter how many questions were answered (the average number of questions answered ranged from 15 to 21 for each of the three surveys).</P></BLOCKQUOTE> <P> </P> <P>In the past I've provided <A href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" mce_href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">six tips for shortening questionnaires</A>. Thanks to this research, here's a seventh: <STRONG>Make the survey interesting to the respondent, and you will shorten the perceived length of the questionnaire</STRONG>.<BR></P> </article> <article> <h1>Survey Trick or Treat: 7 Goblins, Ghosts and Gremlins to Avoid</h1> <p>Brian Koma — Fri, 30 Oct 2009 09:37:00 GMT</p> <P>Are you spooked by poor survey response rates? Do you get a cold chill when your boss asks what can be done to make your surveys better? Are there skeletons in your survey closet you'd rather avoid? Are you bedeviled by survey bias, poor response rates and bad reporting? By understanding the problems that haunt most surveys, you can enhance the quality of your efforts, dramatically improve the value of your data and ensure high participation rates. </P> <P>See if you recognize any of these goblins, ghosts and gremlins:</P> <OL> <LI><STRONG>Zombie Surveys</STRONG> - In the movie "Shaun of the Dead," zombies move forward relentlessly under their own power, but have no thinking behind them. Zombie surveys are typified by survey projects that occur year after year because "we've always done it that way." Since <A href="http://blog.vovici.com/blog/bid/18186/Good-Surveys-start-with-Good-Goals" mce_href="http://blog.vovici.com/blog/bid/18186/Good-Surveys-start-with-Good-Goals">good surveys start with good goals</A>, getting rid of zombie surveys means asking two critical questions: Do I really need to conduct this survey to get this data? What action am I going to take with the data I gather? If you can't answer these questions, you've got a zombie to kill.</LI> <LI><STRONG>Frankenstein Surveys</STRONG> - Dr. Frankenstein bolted a monster together out of unrelated parts, and many organizations create surveys the same way. Too often individual departments are asked to contribute questions to a survey, resulting in an out-of-control monster. Avoiding Frankenstein surveys requires you to relentlessly narrow the scope of your survey and focus only on the data you need to make business decisions. Don't stitch together too many survey questions, but use a scalpel to <A href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" mce_href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">cut out as many questions as possible</A>.</LI> <LI><STRONG>Jekyll & Hyde Surveys</STRONG> - Dr. Jekyll looked perfectly normal, but within minutes of drinking his potion he turned evil. Jekyll & Hyde surveys look normal at the beginning, but quickly turn bad by injecting biased questions or by skewing response scales to summon a pre-ordained result. Never ask questions in such a way that respondents can determine where you stand on any topic. You can avoid drinking Dr. Hyde's potion by striving to <A href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions" mce_href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions">write objective questions</A>.</LI> <LI><STRONG>Response Rate Ouija Board </STRONG>- Conjuring up a high response rate requires more than just a dark room and the right incantations. The right ingredients for great response rates are:</LI> <UL> <LI>1 <A href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers" mce_href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers">high quality list of survey invitees</A></LI> <LI>1 <A href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations" mce_href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations">tasty survey invitation</A></LI> <LI>6 <A href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">key points for the survey invitation text</A></LI> <LI>a spoonful of <A href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam" mce_href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam">tricks to avoid spam filters and blacklisting</A></LI> <LI>a generous helping of treats and incentives to overcome non-response bias </LI></UL> <LI><STRONG>Vampire Invitations </STRONG>- Bats can reach their destination even in complete darkness, but some of them turn into bloodthirsty vampires. Getting invitations out to potential respondents can involve a similar transformation that can limit your ability to field the survey if you're using <A href="http://blog.vovici.com/blog/bid/22595/E-mail-List-Rental-Guidelines" mce_href="http://blog.vovici.com/blog/bid/22595/E-mail-List-Rental-Guidelines">non-permission based lists</A>, or more importantly, not complying with the <A href="http://blog.vovici.com/blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam" mce_href="http://blog.vovici.com/blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam">CAN-SPAM Act</A> of 2003. Repelling Vampire Invitations means your e-mail must be viewed as a friendly spirit by following these simple guidelines:</LI> <UL> <LI>E-mail text contains the physical street address of the sender</LI> <LI>Subject line is accurate and does not mislead the invitee</LI> <LI>"From" line contains the name of the company or representative</LI> <LI>Content includes a valid opt-out or unsubscribe link</LI> <LI>Email list has been reduced by removing names on your suppression list</LI></UL> <LI><STRONG>Headless Horseman Reporting</STRONG> - If you're developing mindless reports that get buried and ignored, you're a victim of Headless Horseman Reporting. Since insights are the reason you conducted a survey, you've got to concentrate on survey reports that people will be dying to read. Overcoming this particularly pernicious gremlin means that you mustn't feel compelled to just report on data in the order it was gathered. Call out the most important elements no matter where they were collected in the survey. Make sure your report addresses the <A href="http://blog.vovici.com/blog/bid/22175/You-Can-t-Be-Brilliant-Alone-How-to-Achieve-Influence-Without-Authority-through-Effective-Collaboration" mce_href="http://blog.vovici.com/blog/bid/22175/You-Can-t-Be-Brilliant-Alone-How-to-Achieve-Influence-Without-Authority-through-Effective-Collaboration">Essential Question</A> that inspired the research in the first place.</LI> <LI><STRONG>Silence of the Lambs </STRONG>- Hannibal Lecter may have done despicable things, but he wasn't shy about talking about them ("I ate his liver with some fava beans and a nice Chianti"). You too must talk up your work. Consider developing summary reports, web seminars or blog posts about your survey data, or ultimately, create an online community to discuss results and show people that you're listening. For the greatest return on your survey investment, <A href="http://blog.vovici.com/blog/bid/20820/Employee-Customer-Engagement-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/20820/Employee-Customer-Engagement-Best-Practices">engage employees with VOC data</A>, then share your results with customers to <A href="http://blog.vovici.com/blog/bid/18174/Closing-the-Feedback-Loop-Sharing-Results-with-Online-Community-Members-Respondents" mce_href="http://blog.vovici.com/blog/bid/18174/Closing-the-Feedback-Loop-Sharing-Results-with-Online-Community-Members-Respondents">close the feedback loop</A> and open up greater participation with your next research.</LI></OL> <P>Exercise these best practices to exorcise the goblins, ghosts and gremlins from your research projects!</P> </article> <article> <h1>The Future Consumer: Co-creating the 2020 Kitchen</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 16:29:00 GMT</p> <P>Darren Lewis and Koen van der Wal of MetrixLab discussed co-creation at the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">2009 ESOMAR Online Research conference</A>. Three emerging themes for product innovation are democratic innovation, Web 2.0 possibilities and active consumers.<BR>Why Co-creation? Consumers make more informed decisions, thanks to unparalleled access to comparative information. Lego provides a great example of co-creation: consumers can build and upload their own ideas for Lego kits and receive a royalty if their idea becomes a product.</P> <P><STRONG>Online Qualitative</STRONG>: Research has been more successful at using the Internet for quantitative research than qualitative research. E-groups and bulletin boards require new skills in research moderation; many qualitative researchers argue that much of the value of the focus group dynamic is lost using such methods.</P> <P><STRONG>Our Co-Creative Methods</STRONG>: MetrixLab begins with a self-administered depth interview with the results unavailable to other participants, prompting greater candor. The MetrixLab co-creative process engages customers, unleashes creativity and discovers opportunities. Interaction is limited to seeing other participants' contributed ideas. The phases (often space a week or more apart):</P> <OL> <LI>Problem analysis and individual idea generation</LI> <LI>Idea sharing and enrichment</LI> <LI>Evaluation by customer</LI> <LI>Evaluation by professional</LI> <LI>Selection</LI></OL> <P><STRONG>Kitchen 2020 Case Study</STRONG>: One hundred and fifty participants in the UK and Netherlands worked to develop the future kitchen. A video introduction to the project provided a more personal description of the project. Participants could create a "moodboard", a bulletin board with images and colors to capture their thinking. Tools channel different creative techniques: pictures and associations, document current frustrations, explore future scenarios, ask "what if?" Participants also wrote descriptions of their moodboards. Themes were the Clean Kitchen, the Green Kitchen, the Flexible Kitchen, the Connected Kitchen and the Automatic Kitchen. The Green Kitchen might incorporate herb gardens and hydroponics, for instance. </P> <P><STRONG>Recommendations: </STRONG></P> <UL> <LI><STRONG><EM>Do's</EM></STRONG></LI> <UL> <LI>Use social media to recruit (e.g., cigar lovers were recruited from Facebook)</LI> <LI>Involve a diverse group of people</LI> <LI>Inspire and motivate people to join your project</LI> <LI>Make the process fun, personal and engaging</LI> <LI>Share the results and give feedback to customers<BR></LI></UL> <LI><STRONG><EM>Don't's</EM></STRONG></LI> <UL> <LI>Don't leave everything open and unstructured</LI> <LI>Don't present a challenge that is too broad or too vague</LI> <LI>Don't forget to involve the client</LI> <LI>Don't judge ideas too fast<BR></LI></UL></UL> </article> <article> <h1>Best Practices in Mobile Research</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 15:32:00 GMT</p> <P>Chris Ferneyhough and Sonia Bishop of Vision Critical discussed best practices for fielding online surveys to mobile audiences in the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research 2009</A> conference. Chris began by pointing out that mobile Internet adoption outpaces desktop Internet adoption and forecasts that eventually usage of the mobile web will be 10 times usage of the traditional web.</P> <P>Vision Critical asked respondents to an online survey if they had received their email invitation on their phone: 1.9% had in the US, 1.2% in UK and 3.8% in Canada. Clearly, respondents are already completing online surveys on mobile devices, even though authors have often not taken this into account. Chris mentioned that a panel registration survey for a smart phone vendor didn't actually work for that smart phone, because of the registration form's reliance on JavaScript, which was off by default on the phone: the open-ended "Other (please specify)" box was locked and disabled because it required JavaScript to enable it once the corresponding radio button was clicked. Since client-side scripting is disabled on many phones, the data your survey collected may be wrong.</P> <P>Researchers need to recognize the fact that online surveys are being completed on mobile devices and need to be optimized for that medium. The wide variety of smartphones at GSMArena.com reveals hundreds of different models with dramatically different market share in different countries. Colors and fonts are implemented differently by different phones and may not be implemented at all on a smart phone: a certain color may render some text unreadable.</P> <P>Many respondents are unfamiliar with their web browser or alphanumeric entry mode on their phone's keypad. Many respondents are concerned about data costs: some have unlimited data plans, others have pay-as-you-go plans. Many have low data connection speeds.</P> <P>To do further research on these topics, Vision Critical studied respondents who are smartphone users and are willing to complete questionnaires on their phone. The sample was balanced and weighted on gender and age, with 500 Canadian respondents, 118 US respondents and 107 UK respondents. The survey covered attitudes towards the national economy. No significant statistical differences were found for closed-ended questions on mobile devices vs. desktop devices. For open-ended questions, of course, desktop users were more verbose. </P> <P>The likelihood to participate in future surveys on mobile phones was greatest for iPhone users (47% were likely to), compared to only 34% of Blackberry users and 23% of all other smartphone users.</P> <P>Sonia presented back-end mobile research best practices:</P> <UL class=unIndentedList> <LI>Maximize use of the available space</LI> <LI>Profile your panel for smartphones, whose email addresses may vary for the phone vs. the desktop</LI> <LI>Identify devices and models supported by your data collection software</LI> <LI>Manage the process of deploying surveys to mobile panelists</LI></UL>And questionnaire design best practices:<BR> <UL class=unIndentedList> <LI>Use simpler question types</LI> <LI>Avoid Flash effects and JavaScript validation</LI> <LI>Write more concise questions and answer lists to minimize scrolling</LI> <LI>Put the Next button "above the fold"</LI> <LI>Limit survey length to 10-15 questions (unless heavily incentivized)</LI> <LI>Keep it simple: avoid color, grids, images, etc.</LI> <LI>Develop for the lowest-common denominator devices</LI></UL>Clearly, survey authors fielding online surveys need to take mobile users into account when developing surveys. </article> <article> <h1>Mobile Interviewing: The Next Frontier of Data Collection</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 15:01:00 GMT</p> <P>Steve Lavine of Toluna discussed mobile interviewing at the 2009 <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research</A> conference, beginning by acknowledging that growth has been slower than expected. While online surveys provide real-time data delivery, such surveys are completed at home or the office: mobile surveys, on the other hand, can now put the survey right at the point of sale. </P> <P>Mobile interviewing works well when you need immediacy or have to reach younger generations or otherwise hard-to-reach audiences. Mobile interviewing works well for diary studies, where panelists can record impulse purchases as they are being made, eliminating reliance on memory and increasing reporting rates; SMS can be used to send reminders to complete the diary. Camera phones can be used in ethnographic research, as respondents submit audio clips, digital images and even short videos about, for instance, the use of a product. Mobile interviewing can also be used for pharmacological testing.</P> <P>Handheld devices have long been used for intercept surveys. Mobile tools leverage affordable hardware and work well in developing countries and can provide access to real-time information such as current quota levels. </P> <P>Good target demographics for mobile interviewing include teens and Generation Y, who are reluctant participants in most other modes of research, but love to text, chat and surf. They love the challenge of the camera phone and eagerly send in images, audio clips and videos; this is true of any income level of teenagers. Much telecommunications research obviously makes sense to do on the mobile phone.<BR>The mobile web provides a rich, extended survey experience with digital images, but many cell phones do not support the web. SMS surveys are discontinuous exchanges of information and are used for simple, text-only polling (a few choose-one or fill-in-the-blank questions), but are accessible to far more people in the U.S. and worldwide than mobile-web surveys. An emerging trend is the downloadable application, which offers rich survey experiences but only to a small installed base; maintaining an access panel of any size or representation is difficult; this works better for a small, well-known group such as employees or partners. IVR surveys fielded to cell phones require the creation and maintenance of an opt-in list for permission contact panelists at the mobile number; invitations are often sent by SMS and the phone IVR survey can be longer, from 5 to 12 minutes in length.</P> <P>Mobile interviewing provides more rapid response time (2.6 hours) than the response rate of any non-mobile method (5+ hours), often 50% faster. Mobile IVR response time is 3.2 hours vs. 5.1 hours for mobile web surveys. This is a significant benefit for entertainment research. For traditional online surveys, SMS invitations provide a 3.9 hour response time, compared to 6.4 hours for email invitations. Respondents invite by text more closely match the population's age distribution.</P> <P>In the future, more short-code surveys will be advertised point of sale and upon exiting the store. Expect to see proximity-trigged surveys using GPS, RFID, cell-tower triangulation and other methods, often with panel registration. </P> </article> <article> <h1>How Online Research Communities Work for Consumers Invited to Participate</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 12:39:00 GMT</p> <P>Ray Poynter of The Future Place presented "It works for us but does it work for them? How online research communities work for consumers invited to participate" at the<A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009"> ESOMAR Online Research 2009</A> conference. Researchers and community-platform vendors assume that "communities provide the authentic Voice of the Customer" and "participants love communities". The benefits we assume for participants are that communities make it easier for participants to get their views across, more empowered, with greater convenience, feel the effort is more worthwhile and more enjoyable.</P> <P>With Lou Rubie of Mars Food and Steven Cierpicki & Daniel Alexander of Colmar Brunton, Poynter did research in Australia, with 1,082 online panel interviews, four focus groups and engaged members of online research communities with discussions, polls and live chat. One hundred percent of respondents to the online survey had done a survey online (whew!), 50% had done a telephone survey (in fact, some had done 10+ telephone surveys), 16% face-to-face interviews, 21% had participated in a focus group (higher than a study of the general population due to the use of panels to recruit participants) and 17% of panel members had participated in online communities (though panelists had broader definitions of communities than MROCs).</P> <TABLE border=1 cellSpacing=0 cellPadding=0> <TBODY> <TR> <TD vAlign=top width=106> <P mce_keep="true"> </P></TD> <TD vAlign=top width=106> <P>Online</P></TD> <TD vAlign=top width=106> <P>Telephone</P></TD> <TD vAlign=top width=106> <P>Face-to-Face Interviews</P></TD> <TD vAlign=top width=106> <P>Focus Groups</P></TD> <TD vAlign=top width=106> <P>Communities</P></TD></TR> <TR> <TD vAlign=top width=106> <P>Always "enjoy participating"</P></TD> <TD vAlign=top width=106> <P>37%</P></TD> <TD vAlign=top width=106> <P>12%</P></TD> <TD vAlign=top width=106> <P>33%</P></TD> <TD vAlign=top width=106> <P>51%</P></TD> <TD vAlign=top width=106> <P>34%</P></TD></TR> <TR> <TD vAlign=top width=106> <P>Always "get my views across"</P></TD> <TD vAlign=top width=106> <P>26%</P></TD> <TD vAlign=top width=106> <P>22%</P></TD> <TD vAlign=top width=106> <P>39%</P></TD> <TD vAlign=top width=106> <P>47%</P></TD> <TD vAlign=top width=106> <P>35%</P></TD></TR> <TR> <TD vAlign=top width=106> <P>Always "convenient to participate"</P></TD> <TD vAlign=top width=106> <P>44%</P></TD> <TD vAlign=top width=106> <P>12%</P></TD> <TD vAlign=top width=106> <P>29%</P></TD> <TD vAlign=top width=106> <P>34%</P></TD> <TD vAlign=top width=106> <P>39%</P></TD></TR> <TR> <TD vAlign=top width=106> <P>Always feel the "return is worth the effort"</P></TD> <TD vAlign=top width=106> <P>20%</P></TD> <TD vAlign=top width=106> <P>10%</P></TD> <TD vAlign=top width=106> <P>28%</P></TD> <TD vAlign=top width=106> <P>42%</P></TD> <TD vAlign=top width=106> <P>30%</P></TD></TR> <TR> <TD vAlign=top width=106> <P>Always able to be "completely honest"</P></TD> <TD vAlign=top width=106> <P>71%</P></TD> <TD vAlign=top width=106> <P>49%</P></TD> <TD vAlign=top width=106> <P>54%</P></TD> <TD vAlign=top width=106> <P>59%</P></TD> <TD vAlign=top width=106> <P>57%</P></TD></TR></TBODY></TABLE> <P>Why do only a third of the sample always enjoy participating in communities? From the qualitative research, comments were:<BR> <UL class=unIndentedList> <LI>"I am putting in my own views but not hearing back from the client"</LI> <LI>One community was too slow to change with too little content</LI> <LI>Another community had too much content, with a participant's comments quickly scrolling off the page</LI> <LI>The belief that other people participating weren't being honest</LI> <LI>People disagreeing with the participant</LI></UL> <P>Communities are less convenient than expected. Why aren't they more convenient? For short-term communities, researchers ask participants to log in four times a day; for other communities, they are asked to upload photos, which is "fun and rewarding but not necessarily convenient!" Researchers give participants a sense of duty: "remember they have a small chance to win a tiny prize!"</P> <P>Honesty in online surveys comes from the lack of direct observation. In communities, people are conditionally responsive based on what others are saying; it is important to give community members other channels were they can make themselves heard.</P> <P>Respondents were asked if they were likely to take part in future ORCs (Online Research Communities): of those who had never participated, 96% were interested; of those who participated, 80% were interested in participating in another community.</P> <P>Caveats: Australia is different from the U.S. and U.K, let alone other countries, so these results may not apply to your market. The results might change in six months to two years, as research communities are evolving so quickly.</P> <P>For a final report card, Poynter concluded, "Good start but must do better!" Communities are exciting, with wonderful potential "but we are resting on our laurels, we need to do a better job." His five key recommendations:</P> <OL> <LI>Ask participants how the experience was for them</LI> <LI>Benchmark against "best in class"; don't just compare to other communities but to other experiences and other methodologies.</LI> <LI>Make communities as enjoyable as focus groups, even though this is a goal that we may never reach.</LI> <LI>Make communities as convenient as online surveys: be more flexible and tailor communities around member preferences.</LI> <LI>Make communities as fulfilling as having a real person there: consider doing an offline event where the community gather together or provide visits to the factory to get a tour on how the products are made.</LI></OL> </article> <article> <h1>ISO 20252 Standard</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 10:35:00 GMT</p> <P>Bill Blyth, Chair of the ISO TC 225 and Global Methods Director of TNS Global described the ISO 20252 standard at the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR 2009 Online Research conference</A>.</P> <P>ISO (International Standards Organization) efforts on survey research grew from European trade associations with concerns about the quality of data collection, dating back to the 1970s for standards for face-to-face interviews in the UK and Netherlands. ISO 20252 covers quality at all stages of the survey process: "As some restaurants in London serving offal say, 'It's nose to tail eating!'"</P> <P>ISO 20252 addresses the quality triangle of Design, Process, User with Fitness in the center. It specifies procedures and documentation and sets minimum levels of validation for key elements. It applies to subcontractors. </P> <P>While ISO 20252 covers all methods, including online research, the Access Panel Standard - ISO 26362 - published in 2008 provides an alternative to 20252 for specialist panel providers. The 20252 standard is being rewritten to be technology neutral so that it will not need to be rewritten "every time a new gizmo" comes out.</P> <P>The ISO 20252 has replaced local standards in Australia, Netherlands, Spain and the UK and will do so in France and Sweden in 2010. Italy and Mexico have 20252-inspired local versions. In total, over 250 companies around the world are certified ISO 20252 compliant; the standard does require a compliance audit by a third party. </P> <P>ISO is encouraging audited compliance to be a contractual/procurement requirement in RFPs. ISO contends that the ISO 20252 saves adopting organizations money by facilitating getting projects right the first time, requiring less rework.</P> <P>Since online research is the easiest method of research to conduct globally, global standards are needed to ensure quality. ISO 20252 provides a framework and common language for online quality and is a global standard suitable for any size organization in any country.</P> </article> <article> <h1>ARF Online Research Quality Council</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 10:13:00 GMT</p> <P>Joel Rubinson, the Chief Research Officer of the ARF, discussed ARF quality initiatives. The ARF ORQC (Online Research Quality Council) was assembled to address the issue of panel quality. The committees developed "Foundations of Quality", a research-on-research project in October and November 2008 conducting 100,000 surveys across 17 U.S. panels, supplemented by phone and mail research. </P> <P>The research revealed that all seventeen panels could retest their results reliably, but results varied significantly from panel to panel. Switching panels or using a panel undergoing major changes could change the results. The purchase intent for a concept correlates to panelist longevity (panels vary significantly by longevity), but no weighting scheme removed this variation when doing multi-panel sourcing. Rebutting assumptions, taking multiple surveys per month (3 to 10) was actually good for respondent engagement, putting the professional in the term professional respondent, which is usually a disparaging terms. Most panelists were not in it for the money, but those who were provided lower quality answers.</P> <P>An industry-solutions committee started translating the insights from this research into an action plan, the QeP. The QeP (Quality Enhancement Process) v 1.0 offers templates, definitions, metrics and declarations to bring structure to conversations between buyers and sellers about online panel quality. The QeP is designed as a process to help buyers and sellers meet the shared goal of providing valid and consistent data. QeP is designed to encourage innovation that improves data validity across the online research ecosystem, which includes marketing and research departments on the client side, management consultants, suppliers and subcontractors to those suppliers. </P> <P>The QeP looks at standards of quality at the panel level, sample/study level and the survey/response-quality level. </P> <UL class=unIndentedList> <LI>At the panel level, panel providers should document recruitment methods, incentive systems, panelist profile, privacy policy, survey QA standards, etc. When panels merge, buyers should be wary of doing trend analysis with results from a predecessor panel. </LI> <LI>At the sample/study level, document a consistent sampling plan, promote sample consistency and eliminate duplicate survey taking in any form (deduping households, not just respondents). </LI> <LI>At the survey/response-quality level, control for panelist longevity.</LI></UL> <P>The next step for ARF is to pilot test the QeF to ensure that the templates are clear, the documents are workable and the reports are useful and comprehensive. These results will drive future evolution of the QeP.</P> <P><A title=preso name=preso></A><STRONG>Update</STRONG> - Joel Rubinson uploaded his presentation to SlideSlide:</P> <DIV style="TEXT-ALIGN: left; WIDTH: 425px" id=__ss_2368722><A style="MARGIN: 12px 0px 3px; DISPLAY: block; FONT: 14px Helvetica,Arial,Sans-serif; TEXT-DECORATION: underline" title="Online Research Arf Quality Enhancement Process Esomar Presentation" href="http://www.slideshare.net/joelrubinson/online-research-arf-quality-enhancement-process-esomar-presentation">Online Research Arf Quality Enhancement Process Esomar Presentation</A> <PARAM NAME="movie" VALUE="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=onlineresearcharfqualityenhancementprocessesomarpresentation-091028140452-phpapp01&stripped_title=online-research-arf-quality-enhancement-process-esomar-presentation"><PARAM NAME="allowFullScreen" VALUE="true"><PARAM NAME="allowScriptAccess" VALUE="always"> <p></p></embed></OBJECT> <DIV style="FONT-FAMILY: tahoma,arial; HEIGHT: 26px; FONT-SIZE: 11px; PADDING-TOP: 2px">View more <A style="TEXT-DECORATION: underline" href="http://www.slideshare.net/">presentations</A> from <A style="TEXT-DECORATION: underline" href="http://www.slideshare.net/joelrubinson">joel rubinson</A>.</DIV></DIV> </article> <article> <h1>ESOMAR 26 Q and Internet Guidelines</h1> <p>Jeffrey Henning — Wed, 28 Oct 2009 09:36:00 GMT</p> <P>Kees de Jong, the CEO of SSI, presented an overview of ESOMAR codes and guidelines at the 2009 ESOMAR Online Research conference.</P> <P><A href="http://www.esomar.org/" target=_new mce_href="http://www.esomar.org/">ESOMAR</A> is less well known in the United States then the rest of the world; ESOMAR is an international organization promoting better market research, with 5000 members in 100 countries agreeing to abide by its code. The United States found online access panels to be of acceptable quality for years, only refocusing on quality after Dedeker's criticisms of quality at the IIR in 2006. This led to a dramatic surge in interest in quality, with many initiatives from many of the industry associations.</P> <P>The <A href="http://www.esomar.org/index.php/codes-guidelines.html" target=_new mce_href="http://www.esomar.org/index.php/codes-guidelines.html">ESOMAR Code</A> outlines eight key fundamental principles for professional researchers. The Code dates back to 1948 and first addressed Internet research in 1997, with the Internet guidelines having been revised three times since then, most recently in the upcoming 2010 guidelines. The E25/E26 provided a list of questions to help inexperienced buyers of online panel compare and select providers. Providers then asked ESOMAR to come up with the right answers, but the goal was to stimulate debate not provide authoritative answers, which was healthy, as new research changed assumptions: for instance, NOPVO discovered that surveys with low response rate (5%) were not materially different from surveys with high response rate (30+%).</P> <P>The 2010 guidelines look at four focal points: ethical (e.g., confidentiality and anonymity), regulatory (e.g., PII), methodology (e.g., panel vs. river), and technology issues (e.g., digital fingerprinting). The three overriding principles: researchers must diligently maintain distinction between marketing and research; don't bring the industry in disrepute; and the foremost principle is treating the respondent with respect, building a relationship built on trust, respect and reciprocity. </P> <P>What does the future hold? A new focus on respondent experience and perhaps an ESOMAR Experience Conference!</P> </article> <article> <h1>Web 2.0: Transformational Technology or Pretty Gradients & Hype?</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 17:41:00 GMT</p> <P>Steve August, the CEO of Revolution, presented a <A href="http://en.wikipedia.org/wiki/Pecha_Kucha" mce_href="http://en.wikipedia.org/wiki/Pecha_Kucha">Pecha Kucha</A> session at the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research</A> 2009 conference. He began by defining Web 2.0 as the convergence of software and socialness. As Paul Moore put it, "Web 2.0 is made of 600 million unwanted opinions in real time." Web 2.0 is made of broadband Internet, social software, digital media and wireless devices. Web 2.0 has a transformational impact for MR. Initially, MR took traditional research methodologies online. The mission of market research is "to understand people to answer business questions" but life is 99% researcher free - moments of decision, of purchase, of consumption take place out of sight. It would be expensive and intrusive to follow people 24/7, but Web 2.0 gives us this: connection + engagement + richness + immediacy. Previously our access to people was limited, a few hours in a focus group, a few minutes in a survey, but with Web 2.0 we have unlimited and sustainable access to people and their emotions. Marry these and it unleashes changes in qualitative research. Web 2.0 MR is climbing up the slope of enlightenment of the Gartner hype cycle, capturing moments of time to be everywhere at once, transforming market research.</P> <P>Tom Ewing is the Social Media Knowledge Leader of Kantar Operations who presented the next Pecha Kucha session. "When a profession has been created as a result of some scarcity, the professionals are often the last ones to see it," Clay Shirky wrote, in <EM>Here Comes Everybody</EM>. Market researchers solicit and create information, collate and validate information, ensure it is representative, analyze and deliver information, deriving the meaning from it. All of these were scarcities that people were willing to pay for, but now there are... </P> <UL> <LI>More and better DIY research tools.</LI> <LI>Weaker barriers to sharing opinions.</LI> <LI>Larger and richer data trails that people leave behind. They leak more data, share more data, leaving more to be analyzed.</LI></UL> <P>Why ask information when it is already out there? Why ask other to test your new ideas? Our ability to collect data has been outflanked by the Internet. "We are tics on the body of the information hippo," Ewing said. Validating user generated data is vital, and such validation is a narrow skill; data can't analyze itself, which is another scarce skill; data can't present itself, which is another scarce skill. The scarcities are more stable, less threatened by Web 2.0, but do market researchers own those scarcities? We need to become essential to participants, making research a benefit and a thrill to get the exclusive information. We need to make the case for validity and the need for representativeness. We are mastering the new data sources such as social graphs. Then we can start prioritizing the analysis and delivery. That is why we can anticipate a bright future for the market research business. We can recognize the decline of scarcities and react accordingly.</P> <P>Anthony Hamelle, the vice president of opinion and market research at Linkfluence, presented next. Where <A href="http://blog.vovici.com/blog/bid/18085/Web-1-0-vs-Web-2-0" mce_href="http://blog.vovici.com/blog/bid/18085/Web-1-0-vs-Web-2-0">Web 1.0 was static, Web 2.0 is dynamic</A>. People get and share information on the Internet. People now get information filtered by who they know, through friends, peers, acquaintances in the social network. A revolution can mean rolling back to a previous solution: sense of community provides belonging and interdependence, which was the natural state of people before the 20th century. Communities have strong and weak links: tribes of a few thousand, communities up to 300, bands up to 50. Influence from a distance is threatening; more comforting from friends. Communities evolve and exist and die online; the flow of opinions can be seen through communities as they travel through community leaders and members. Web 2.0 is a re-empowerment of communities and individuals within communities. Not every human group is a community; it takes time and belonging; very few actual spontaneous brand-driven communities. Rethink how we view interviewees, respondents, participants co-researchers.</P> <P>Web 2.0 is a return to community.</P> </article> <article> <h1>Online Community Platforms: A Macro-Overview and Case Study</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 15:46:00 GMT</p> <P>At the 2009 ESOMAR Online Research conference, James Kennedy from BrainJuicer presented an overview of his vision of the types of community platforms. </P> <P>"We are witnessing a shift in corporate behavior from where they get ideas for products and services," said Kennedy. At P&G, success rate for new products climbed from 20% in 2000 to 60% in 2008 as P&G embraced consumer-generated ideas, and P&G expects 50% of products to be from consumer-generated ideas in 2010.</P> <P>Community and crowdsourcing are two new sources of innovation that organizations need to determine how to make work. It's important to have a clear understanding of goals, who you are engaging with and how you are working with them.</P>Kennedy presented the following taxonomy of communities [simplified below]: <TABLE border=1 cellSpacing=0 cellPadding=0> <TBODY> <TR> <TD vAlign=top width=213> <P mce_keep="true"> </P></TD> <TD vAlign=top width=213> <P>Open</P></TD> <TD vAlign=top width=213> <P>Closed</P></TD></TR> <TR> <TD vAlign=top width=213> <P>Independent</P></TD> <TD vAlign=top width=213> <P>Independent Open Community</P></TD> <TD vAlign=top width=213> <P>Multi-Client MROC</P></TD></TR> <TR> <TD vAlign=top width=213> <P>Branded</P></TD> <TD vAlign=top width=213> <P>Branded Open Community</P></TD> <TD vAlign=top width=213> <P>Branded MROC</P></TD></TR></TBODY></TABLE> <UL> <LI><STRONG>Branded open communities </STRONG>are a bar; "if you build it, they will come". Some brands have the ability to attract participants to be researched, such as MyStarbucksIdea, which has 75,000 ideas with only 350 ideas implemented. (This represents a lot of new ideas that have been engaged, even though it is only a small percent.)</LI> <LI><STRONG>Independent open communities </STRONG>are like matchmaking for the B2B world, where people post problems and others post solutions. See NineSigma for an example.</LI> <LI><STRONG>Branded MROCs </STRONG>are an arranged marriage, where the communities rely on quite close affinities with the brand.</LI> <LI><STRONG>Multi-client MROCs </STRONG>are what BrainJuicer does with its JuicyBrains innovation community, providing access to 10,000 members across a wide range of categories. The five-phase methodology provides for insight exploration (3 weeks), concept generation (3 weeks), a harvest workshop, a concept clinic (2 weeks) and concept validation. Applications include researching problems, habits, brand associations, test diaries, ideation, concept improvement, claims and positioning, product naming and loyalty ideas.</LI></UL> <P>Using JuicyBrains, Philips Healthcare researched 18 people suffering serious respiratory illness and 5 relatives of such people, asking each to document diagnosis, symptoms and treatment. This study was private and not accessible to other members of the community. Philips Healthcare drew functional insights as well as emotional context, and learnt the importance of better framing how information was presented to patients. Based on the results, Philips is investigating how it can better inform patients about the innovation process.</P> </article> <article> <h1>Social Networks: The Big Players</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 14:17:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research</A> conference, Tom Anderson of Anderson Analytics moderated an interactive panel featuring Daniel Shapero, director of enterprise solutions with LinkedIn, and Sean Bruich, monetization analyst with Facebook.</P> <P>According to Anderson Analytics, the top 10 buzzwords that senior marketing executives are tired of hearing are Web 2.0 (19.4%), social networking (12.2%) and social media (11.3%), all up dramatically in 2009 from 2008, but concede the importance of these media. An estimated 60% of the U.S. online population uses a social network, primarily either for fun or for business, though convergence is happening. Nonusers are time-starved, concerned about privacy or are "social media pessimists". Social media is ubiquitous under the age of 24 but the rate declines by age group, down to 20% of the 65+ demographic. LinkedIn is a bit more male, and Facebook is a bit more female. The average user logs in five times a week and is less active on other types of media than a non-user. </P> <P>The four biggest social networks are Facebook, Myspace, Twitter and LinkedIn. A quote from the absent Myspace representative: "LinkedIn is the office, Facebook the backyard barbeque, and Myspace is the bar." Facebook wants to be the backyard barbeque as well as the church and school. From a myopic MR perspective, Facebook has the potential to be the ideal B2C panel and LinkedIn to be the ideal B2B panel. </P> <P>Ten percent of respondents admitted to creating a fake social network profile, according to Anderson research, but maintaining a fake profile is tedious and time consuming-can social networks be a solution to low online panel quality? The speakers said that social network sites have much deeper understanding of their members than most other websites. Bruich argues that the network has counterincentives to limit people from misleading one another (e.g., unfriending, calling people out) leading to high authenticity. LinkedIn discourages "promiscuous connecting" and wants a true representation of your professional social graph. </P> <P>Bruich says that Facebook leverages its market research tools, "eating its own dog food" to determine product features, provide fast feedback for advertisers and conduct traditional and less traditional ad research. "You can't always rely on what users are asking for but have to overcome barriers they identify" to find new opportunities, said Bruich. LinkedIn uses in-depth interviews as well as its own tools to decide what few selected things to go test. "One of the advantages of social networks is that you can be really precise about the audience you want to test," according to Shapero.</P> <P>For market researchers, Shapero says that LinkedIn has very accurate, very high quality data; LinkedIn knows who people are and that they are who they say they are. That's a problem for today; in the future, leveraging the social graph to understand people in context will be more important. The MR industry is much more excited about listening to ongoing conversations on social networks, looking at comments by detailed demographics such as occupation and managerial role. Facebook is primarily driven by advertising influence tracking today but is doing more looking at social and consumer trends in the future. For instance, Facebook is a great place to study a movie, if users are going to go see it, what they thought; you can predict a movie's opening weekend gross from the level of activity on Facebook. </P> <P>If you're researching CPG, you need to complement text analysis with opportunities to solicit feedback about products that aren't being extensively talked about. Some Facebook users share everything they post, and an API provides access to that, but the vast majority of Facebook conversations are private, shared only with the social network. Facebook does a lot of text mining in house so that no PII (Personally Identifiable Information) is shared. Nothing LinkedIn or Facebook does will be allowed if it breaks the users' trust with their site; LinkedIn in particular is concerned about the survey experience given the viral nature of that experience. Facebook correlates with Rasmussen and Gallup results more closely than other sources.</P> <P>Third-party applications on Facebook can be important sources of information, sharing movies watched, books read, etc. "A survey inside a box on Facebook is still a survey, so people haven't really leveraged the platform yet for feedback," said Bruich. LinkedIn gets asked for surveys of respondents where none of the respondents are in each other's first level social graphs. According to Shapero, LinkedIn finds high survey rates across its user base, though small business owners have lower response rates than software developers, for instance. </P> <P>For both firms, market researchers are an important target market for future business offerings.</P> </article> <article> <h1>Bloggers as Research Partners</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 12:32:00 GMT</p> <P>Josephine Hansom, a social researcher with GfK NOP, presented at the <A href="http://hblog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://hblog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">2009 ESOMAR Online Research</A> conference results of qualitative research with ten bloggers. Key points:</P> <UL> <LI><STRONG>Confessional Society</STRONG>: A confessional culture provides accessible opinions; people share because an audience is now available that is interested. On 9/30/09 blogging about Barack Obama, people were concerned that other priorities were more important than the Olympics bid, people in other countries felt that Obama's trip to Copenhagen provided an unfair advantage over other competitors.</LI> <LI><STRONG>The challenge</STRONG>: Deconstructing online opinion, looking at the blogger instead of the blog. The study performed content analysis on 10 blogs, engaged with the bloggers online then talked with the bloggers offline. The blog is different in four distinct ways: motivation, audience, identity (online and offline identities could diverge) and publishing (attitudes towards security and privacy).</LI> <LI><STRONG>Opinion sharers</STRONG>: Who blogs? Three types of blogging participants identified in this study: </LI> <UL> <LI>The <EM><STRONG>ready-meal blogger </STRONG></EM>documents offline personal interests, has a small known audience but is writing for himself more than the audience, like a diary. He does not identify as a blogger and is security conscious. </LI> <LI>The <STRONG><EM>dinner party blogger</EM></STRONG> is interested in generating interest and interacting with guests (like a dinner party host). She wants to generate and maintain an audience, which has changed her blog from its beginnings as a personal blog. Her online world mirrors the offline world; she has a greater understanding of online publishing than other blogs.</LI> <LI>The <EM><STRONG>lite blogger </STRONG></EM>doesn't see the blog as an everyday activity but uses it when task driven. He is aware of his audience and doesn't share personal data or information that wouldn't be useful.</LI></UL> <LI><STRONG>Going forward</STRONG>: How useful are blogs as data? The online persona does not always reflect the offline persona; remember the bloggers' potential motivations and concerns about privacy and security. Bloggers merit a reciprocal relationship. When using blogs as research data, recognize the impact of the audience and interaction on what the blogger is sharing. Engage by acquiring bloggers as sample, eavesdrop to analyze online statements, and connect in order to understand context of what is being shared, meeting bloggers half way; only then can we authenticate the opinions being shared online.<BR></LI></UL> </article> <article> <h1>How to be Successful in Co-creation Research</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 11:48:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research 2009</A> conference, Volker Bilgram of HYVE AG presented a case study of the Swarovski Enlightened Innovation Research Community, which was designed for the purposes of co-creation, marketing research and community. </P> <UL> <LI><STRONG>Cocreation</STRONG>. Part of the IRC was a co-creation tool to enable users to configure wristwatch designs from 24 components then selecting and freely placing any of 108 gemstones on the watch. More than one thousand designs were submitted in the first eight weeks, ranging from the simplistic to complex designs that took participants 45 minutes to create. Users could also upload designs created with other software. </LI> <LI><STRONG>Marketing research</STRONG>. Designs were presented in galleries and rated by three groups: by users, by a jury of Swarovski designers and by a jury of external designers. Users could submit a short evaluation (award 1-5 points) or a detailed evaluation ("vote and win"). Members were invited to comment on one another's designs. Besides stated preferences, data was mined from the configuration tool for frequency of selection of components and gemstones. In addition, preferred combinations were identified.</LI> <LI><STRONG>Community</STRONG>. The IRC was an open community requiring completion of a short registration form. Users uploaded their own avatars. Banners on design-related websites were used for initial recruitment, which led to buzz to spread interest in the community virally. More than 2000 participants contributed to the contest from 48 countries.</LI></UL> <P>In the post-processing, the winning design was produced as a prototype and presented at a leading trade-show fair in Switzerland. A trend booklet of the MR findings was used for B2B marketing by Swarovski. The community was so successful that Swarovski ran a follow-on community and will continue community research initiatives.</P> </article> <article> <h1>Inspiring Change: Innovative Methods & Integrated Advertising</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 11:01:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research conference</A>, Alison Bryant of Smarty Pants, Katie Bessiere of MTV Networks and Brian Levine of Innerscope Research discussed the use of biometrics to demonstrate that casual gaming is serious business.</P> <P>Sixty two participants, ages 14 to 34, were invited to select a game to play while wearing a wireless biometric vest, while having their pupil dilation and attention measured. The key conclusion was that participants had 95-99% focused attention with games, compared to 80% for TV programming and as low as 30% for TV commercials. People pay attention when playing games.</P> <P>Three conclusions from the testing of video pre-roll advertising: </P> <OL> <LI><STRONG>Focus on action games</STRONG>. The study hypothesized that there were ad-impact differences between action games vs. cognitive games. This hypothesis was validated, as video prerolls have higher recall when placed before an action game; the cognitive game seems to drive out what was learned in the video preroll. As a result, MTV Networks now classifies games on its sites based on whether they are action or cognitive games, in order to serve up video prerolls before action games.</LI> <LI><STRONG>Shorter is better</STRONG>. A 15-second video preroll had double the recall levels of a 30-second preroll, perhaps because the key messaging occurs at the end of video, by which point participants have stopped paying attention in the 30-second spot.</LI> <LI><STRONG>Drive synergy between games and brands</STRONG>. Contextually relevant ads in front of nonbranded games provide better recall (e.g., a Burger King ad before an unbranded burger shop game): people who played a game related to the ad actually enjoyed the game 40% more. MTV now codes games about subject categories to better serve advertisers.</LI></OL> <P>Four conclusions for in-game advertising:</P> <OL> <LI><STRONG>Focus on games with high cognition </STRONG>(e.g., word games, games where players must spot the differences between pictures).</LI> <LI><STRONG>Target areas of focused attention</STRONG>. Fifteen seconds of focus on the message in the game produced 80% recall. In the U.S., the left side of the game produces better recall than the right side. </LI> <LI><STRONG>Integrate with the game brain</STRONG>. Put messaging in areas with higher cognition, such as the loading screen, the menu screen and the rewards screen.</LI> <LI><STRONG>Be central to the action</STRONG>. Banner ads are often ignored so brand a scoreboard instead, brand acknowledgement for completing a level.</LI></OL> <P>MTV Networks now has an advergame development handbook that encodes the findings from this research to produce games that better support MTV advertisers. MTV is demonstrating that casual gaming makes for serious advertising.</P> </article> <article> <h1>Innovative Online Research</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 10:24:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research conference</A>, Ron Riley, an independent marketing research consultant, presented a case study of the U.S. Presidential campaign of Barack Obama. In July 2007, the Obama campaign wanted to develop a framework for smarter policy decisions about whether to go to war and how to evaluate performance in war. Riley was hired to apply online research to assess voter perceptions and refine the doctrine for Obama policy staff. </P> <P>Five innovations used in the research:</P> <OL> <LI>The study used hybrid sampling (Caddell, 1984), using random selection of 76 respondents from a national panel, with 20 to 120 minute interviews (50 minutes on average).</LI> <LI>Reinterviewing identical respondents across multiple points in time (August/September 2007 and November 2007) to understand the evolution of individual perceptions. For instance, Gary Hart's re-entry in December 1987 was seen as presidential but that perception evaporated among the same respondents in six weeks.</LI> <LI>A multimedia Internet platform that required webcams was used to interview respondents at any location with versatility, engaging all three learning styles. Only 12% of adult learners are auditory learners, making the phone less engaging for research; the study showed a wide range of stimuli to engage respondents visually and kinesthetically. </LI> <LI>Smart phones and PDAs to visit URLs and upload video and audio. Low incidence and connectivity</LI> <LI>Finally, voice analysis of involuntary vocal frequencies was used to understand respondent emotions, using Nemesysco technology.</LI></OL> </article> <article> <h1>Social Media Trends around the World</h1> <p>Jeffrey Henning — Tue, 27 Oct 2009 08:55:00 GMT</p> <P>Bonnie Breslauer of Lightspeed Research presented the Global Web Index (GWI) at <A href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009" mce_href="http://blog.vovici.com/blog/bid/22879/ESOMAR-Online-Research-2009">ESOMAR Online Research 2009</A>, which includes research from Tom Smith of TrendStream. </P> <P>The Global Web Index is a syndicated research offering from TrendStream focusing on attitudes regarding usage of social media. TrendStream surveys 16,000 respondents every six months with a long questionnaire (completion took longer than 30 minutes). Lightspeed's global consumer panels were weighted by national representation of key demographics, focusing on 16 markets representing 70% of the Global Web Universe.</P> <P>While email is continuing to grow, social network and community sites are growing faster: 31% growth from August 2008 to 2009 for social networks compared to 20% email growth. The key factors for online usage are cultural, generational and gender. </P> <UL class=unIndentedList> <LI><STRONG>Cultural drivers</STRONG>. Emerging markets (Brazil, Russia, India, China) have the greatest usage of social media and use social media more than ecommerce.</LI> <LI><STRONG>Generational drives</STRONG>. After country, age is the second defining factor. The younger the respondent the more likely to use social media. Everyone uses the Internet to stay in touch with friends, but keeping friends up-to-date has the highest skew towards younger generations. Users who create content are also more likely to be younger, regardless of culture. Younger generations have greater trust in social-media connections and trust the author of a blog they've read recently more than a journalist for a national periodical.</LI> <LI><STRONG>Gender</STRONG>. Women are more socially motivated than men and are more likely to have engaged in social media usage within the prior month.</LI></UL> <P>Social media users fall into three types: passive, light and active. Passive users participate by viewing; light users update content or manage a social-network profile; active users create web sites, blogs or videos. While small in number, active users have a disproportionate influence.</P> <P>The survey touched on some brand use and perceptions. Consumers value open dialogue with a company as opposed to one-way messaging: 22% reported that their perception of a brand is improved if the organization responds to comments in online communities and forums. In an interesting segmentation, owners of Chrysler, GM and Ford automobiles are less likely to use microblogs or upload videos than BMW or Mini Cooper owners.</P> <P>As social media evolves, it is more important than ever to understand trends among consumer usage of social media.</P> </article> <article> <h1>ESOMAR Online Research 2009</h1> <p>Jeffrey Henning — Mon, 26 Oct 2009 22:44:00 GMT</p> <P>Here are a list of recaps of select sessions from the ESOMAR Online Research 2009 conference in Chicago. This conference is the first edition of a relaunch of ESOMAR's online panel conference, as online research is now used in 20% of overall research spending in 10 countries, passing telephone research at 18%.</P> <P>Reg Baker discussed the implications of social media for market research and argued that CAPI and panels were not paradigm shifts but that social media does represent such a shift, requiring a fundamentally different science and technique than traditional market research. </P> <P>Recaps:</P> <UL> <LI><STRONG>Online World</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22890/Social-Media-Trends-around-the-World" mce_href="http://blog.vovici.com/blog/bid/22890/Social-Media-Trends-around-the-World">Social Media Trends around the World</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/22895/Innovative-Online-Research">Innovative Online Research</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22897/nspiring-Change-Innovative-Methods-Integrated-Advertising">Inspiring Change: Innovative Methods and Integrated Advertising</A> </LI></UL> <LI><STRONG>Online Activation</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22901/How-to-be-Successful-in-Co-creation-Research">How to be Successful in Co-creation Research</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22905/Bloggers-as-Research-Partners">Bloggers as Research Partners</A> </LI></UL> <LI><STRONG>Online Platforms</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22912/Social-Networks-The-Big-Players">Social Networks: The Big Players</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22917/Online-Community-Platforms-A-Macro-Overview-and-Case-Study">Online Community Platforms: A Macro-Overview and Case Study</A> </LI></UL> <LI><STRONG>Pecha Kucha</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22922/Web-2-0-Transformational-Technology-or-Pretty-Gradients-Hype">Web 2.0: Transformational Technology or Pretty Gradients & Hype?</A> </LI></UL> <LI><STRONG>Online Quality</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22940/ESOMAR-26-Q-and-Internet-Guidelines">ESOMAR 26 Q and Internet Guidelines</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22942/ARF-Online-Research-Quality-Council">ARF Online Research Quality Council</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22945/ISO-20252-Standard">ISO 20252 Standard</A> </LI></UL> <LI><STRONG>Online Sourcing</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22955/How-Online-Research-Communities-Work-for-Consumers-Invited-to-Participate">How Online Research Communities Work for Consumers Invited to Participate</A> </LI></UL> <LI><STRONG>Mobile Research</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22971/Best-Practices-in-Mobile-Research">Best Practices in Mobile Research</A> </LI> <LI><A href="http://blog.vovici.com/blog/bid/22967/Mobile-Interviewing-The-Next-Frontier-of-Data-Collection">Mobile Interviewing: The Next Frontier of Data Collection</A> </LI></UL> <LI><STRONG>Online Future</STRONG></LI> <UL> <LI><A href="http://blog.vovici.com/blog/bid/22977/The-Future-Consumer-Co-creating-the-2020-Kitchen">The Future Consumer: Co-creating the 2020 Kitchen</A> </LI></UL></UL> <P>Wrapups:</P> <UL> <LI>Wrapups from <EM>Research </EM>magazine:</LI> <UL> <LI><A href="http://www.research-live.com/comment/a-refreshing-lack-of-consensus-on-online-communities/4001285.article">A refreshing lack of consensus on online communities</A></LI> <LI><A href="http://www.research-live.com/news/social-media-research-blows-through-windy-city/4001272.article">Social media research blows through Windy City</A><A href="http://www.research-live.com/comment/researchers-in-chicago-ponder-the-future-of-mrocs/4001279.article"> </LI> <LI>Researchers in Chicago ponder the future of MROCs</A></LI> <LI><A href="http://www.research-live.com/comment/getting-to-grips-with-new-ideas-and-dusting-off-some-old-ones/4001281.article">Getting to grips with new ideas and dusting off some old ones</A></LI></UL> <LI>Wrapup from ESOMAR</LI> <UL> <LI><A href="http://www.esomar.org/index.php/online-research-09-conference-review.html" mce_href="http://www.esomar.org/index.php/online-research-09-conference-review.html">Conference review: Window into 2010</A></LI></UL> <LI>Blogger wrapups</LI> <UL> <LI><A href="http://www.pluggedinco.com/blog/bid/27824/ESOMAR-Online-Research-2009-Takeaways" target=_new mce_href="http://www.pluggedinco.com/blog/bid/27824/ESOMAR-Online-Research-2009-Takeaways">ESOMAR wrapup from Matt Foley</A></LI> <LI><A href="http://zebrabites.com/2009/11/03/the-new-world-of-research/">The 'new' world of research (Katie Harris)</A> </LI></UL></UL> <P>And here's a <A title=wordcloud href="http://www.wordle.net/" name=wordcloud target=_new mce_href="http://www.wordle.net">Wordle.net</A> wordmap of the top 50 words from the 500+ <A href="http://twapperkeeper.com/esoc/?limit=1000" target=_new mce_href="http://twapperkeeper.com/esoc/?limit=1000">#esoc tweets</A> [omitting "esoc" and "RT"]:</P> <P></P> <P><STRONG>P.S.</STRONG> If you want to make a pretty word cloud, use Wordle, but for serious qualitative research, check out the Tribal Toolkit <A href="http://www.tribaltoolkit.com/" target=_new mce_href="http://www.tribaltoolkit.com/">word cloud maker</A> (lets you combine words: e.g., "survey" and "surveys", reported separately above).</P> </article> <article> <h1>Employee Engagement Definition</h1> <p>Jeffrey Henning — Mon, 26 Oct 2009 16:24:00 GMT</p> <P>In "A Historical Perspective of Employee Engagement: An Emerging Definition", Michael Bradley Shuck and Karen K. Wollard study the evolution of the term <EM>employee engagement </EM>and synthesize a possible consensus definition. Why is employee engagement an important concept? Shuck and Wollard write:</P> <BLOCKQUOTE> <P>Employee engagement has generated a great deal of interest in recent years as a widely used term in organizations and consulting firms (Macey & Schneider, 2008) especially as credible evidence points toward an engagement-profit linkage (Czarnowsky, 2008). Employee engagement has been characterized as "a distinct and unique construct that consists of cognitive, emotional, and behavioral components . . . associated with individual role performance" (Saks, 2005, p. 602). Engaged employees often display a deep, positive emotional connection with their work and are likely to display attentiveness and mental absorption in their work (Saks, 2006). Although engaged employees are consistently more productive, profitable, safer, healthier, and less likely to leave their employer (Fleming & Asplund, 2007; Wagner & Harter, 2006), only 30% of the global workforce is estimated to be engaged (Harter, Schmidt, & Hayes, 2002; Saks, 2006). Nonetheless, despite continued evidence of linkages to positive business outcomes, employee engagement is declining (Czarnowsky, 2008).</P></BLOCKQUOTE> <P>The first published use of the term <EM>employee engagement </EM>was in the Academy of Management Journal article "Psychological Conditions of Personal Engagement and Disengagement at Work" (W. Kahn, 1990), but since then many varied and conflicting definitions of employee engagement have made it difficult to compare and contrast research findings. To determine a common definition, Shuck and Wollard reviewed 140 articles that mentioned employee engagement and observed four areas of consistency or inconsistency:</P> <OL> <LI>Engagement is a personal decision, not an organizational decision as implied by some definitions.</LI> <LI>While early definitions treated engagement as an atomic concept, later definitions divided it into three basic concepts: emotional, behavioral and cognitive engagement.</LI> <LI>Employee engagement has "no physical properties, but is manifested and often measured behaviorally". Different definitions look at behavior as the employee's basic job performance or extended effort or the success of the employer.</LI> <LI>Employee engagement is about the behaviors that meet or exceed organizational goals.</LI></OL> <P>Synthesizing the reviewed definitions, Shuck and Wollard define employee engagement as "an emergent and working condition as a positive cognitive, emotional, and behavioral state directed toward organizational outcomes".</P> <P></P> <P>Given the growing importance of linking employee engagement to customer loyalty and business profits, standardizing on a common definition for the term provides an important foundation for future research.</P> </article> <article> <h1>Feedback Management Best Practices for CEM</h1> <p>Brian Koma — Fri, 23 Oct 2009 09:49:00 GMT</p> <P>According to the Vovici <A href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx" mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx">CE IQ</A> study, organizations that have the most loyal customers have approached primary research with a coordinated strategy of <A href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx" mce_href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">feedback management</A>.</P> <P>Standardizing on a common feedback management solution throughout the enterprise provides many benefits for organizations. First, they can use the platform to coordinate activities across different departments and functional areas, helping to eliminate redundant work. Second, a common platform enables the use of "guidelines and guardrails" that empower users with the ability to capture data within pre-approved parameters and branding standards. Finally, it enables easy information sharing among internal stakeholders while eliminating redundant costs for multiple platforms. </P> <P></P> <P>Once the platform is in place, organizations must agree on a set of metrics that will be used to evaluate the customer experience. These metrics can be drawn from one of the many existing CEM metrics that already exist, such as <A href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Net Promoter</A>, <A href="http://blog.vovici.com/blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation" mce_href="http://blog.vovici.com/blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation">ACSI</A>, the <A href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model" mce_href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model">Apostle Model</A>, <A href="http://www.cgaexperience.com/pdfs/customer_heartbeat.pdf" target=_new mce_href="http://www.cgaexperience.com/pdfs/customer_heartbeat.pdf">Customer Heartbeat</A> and many others, or they can utilize a proprietary model. While we can nitpick each measure, and some are better than others for different industries and different organizations, consider this: <STRONG>the most important element of success is to agree on a set of metrics and then apply them consistently throughout the organization</STRONG>.</P> <P>To achieve the highest levels of loyalty, organizations must also coordinate their feedback efforts across all parts of the organization to avoid fatiguing customers with requests for information. One large customer of a Fortune 50 firm brought to the CEO's attention that they were receiving 100 <A href="http://blog.vovici.com/blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" mce_href="http://blog.vovici.com/blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">survey invitations</A> a year! After that Fortune 50 firm became a Vovici client, they realized that they were fielding 450 separate customer feedback initiatives. By eliminating duplicate requests for the same or similar sets of information, organizations can reduce the number of surveys that are conducted while ensuring that <A href="http://blog.vovici.com/blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship" mce_href="http://blog.vovici.com/blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship">response rates</A> will increase to higher levels and avoid the erosion that many organizations are facing due to "feedback fatigue."</P> <P>Finally, organizations with the most loyal customers track the metrics they measure over a period of time to determine whether or not they are moving the needle in the right direction. </P> </article> <article> <h1>Know When to Hold Them, Know When to Fold Them</h1> <p>Jeffrey Henning — Thu, 22 Oct 2009 20:41:00 GMT</p> <P>At the <A href="http://blog.vovici.com/blog/bid/22264/AMA-MRC-2009-Roundup" mce_href="http://blog.vovici.com/blog/bid/22264/AMA-MRC-2009-Roundup">AMA Market Research Conference</A>, Andrew Baird, VP of Marketing for Convenience Retail for BP, shared stories about his years of research in the convenience-store market. One particularly vexing recent decision was whether to consolidate three BP brands or not and-if so-which brand to consolidate to.</P> <P>BP had three brands: <EM>am-pm </EM>on the West Coast, the healthier <EM>Wild Bean Café </EM>in the Northeast, and <EM>bp connect </EM>elsewhere. Complicating the research process, Andrew has found that respondents in this space often shape their answers based on the influence of social norms:<BR></P> <UL class=unIndentedList> <LI>When Andrew was researching the impact of sweepstakes on petrol purchases in New Zealand, one focus-group participant said she would never be interested in sweepstakes; they would not make a difference to where she filled up her car. After the focus group concluded, as she got up to leave, her purse fell open and out dropped dozens of sweepstakes tickets. She had been too embarrassed to admit her enjoyment of such promotions.</LI> <LI>As Andrew studies convenience-store food choices, respondents frequently express an interest in healthy choices for food, but in real life buy the less healthy option: they say they want a salad but buy a hamburger. They "think thin, eat fat". Andrew commented that the recent failure of Fresh & Easy, the Tesco entry into the U.S. convenience retail market, may be because it seems "to have taken the American consumer at face value". Yes, the convenience store needs to offer some healthy food, but that salad serves as a permission driver to get customers to treat themselves.</LI> <LI>Even small decisions go awry. To save costs, one store replaced plastic coffee stirrers with wooden stirrers. Customers thought the wooden stirrers were flimsy and would take three of them on average, instead of a single plastic stirrer, resulting in increased costs rather than the expected savings.</LI></UL> <P>Faced with the decision to consolidate three convenience store brands to one, Andrew didn't trust respondents. After initially concluding their traditional research, they decided the data was inconclusive. "We went back to the research table with convenience store users. We developed five trial stores. I'm a huge proponent of ethnography to understand their actual behavior. In my experience, it is the real moment of truth. After observing these trial stores, we had our answer. We moved from three brands to one: <EM>am/pm</EM>, as heavy C-store users preferred <EM>am/pm</EM>. We still had salads, like <EM>Wild Bean Café </EM>led with, but they were not the big sellers."</P> <P>Summing up, Andrew said, "There is no silver bullet, no magic research recipe. It's data plus consumer behavior plus intuition. You only get one chance to launch a product or service; you rarely get a chance to relaunch. It is critical to get it right the first time. Marketers need and want market researchers at the table."</P> </article> <article> <h1>Customer Satisfaction ROI Analysis</h1> <p>Jeffrey Henning — Wed, 21 Oct 2009 13:53:00 GMT</p> <P>Jeremy Whyte, director of customer feedback and reporting with Oracle Corporation, presented details of Oracle's extensive Voice of the Customer research program in a Vovici research webinar yesterday, "<A href="http://www.marketingpower.com/ResourceLibrary/Pages/Webcasts/Driving_Business_Growth_and_Profit_from_a_Customer_Experience_Management_Program_102009.aspx" target=_new mce_href="http://www.marketingpower.com/ResourceLibrary/Pages/Webcasts/Driving_Business_Growth_and_Profit_from_a_Customer_Experience_Management_Program_102009.aspx">Driving Business Growth and Profit from a Customer Experience Management Program</A>". Oracle has achieved five years of year-over-year growth in <A href="http://www.vovici.com/" mce_href="http://www.vovici.com">customer satisfaction</A>, even as revenues have grown from under $14 billion in 2005 to over $23 billion as 53 acquisitions have been merged into the Oracle product line. To evaluate and improve customer satisfaction, Oracle has developed a linkage analysis that ties operational measures, transactional satisfaction, customer satisfaction/loyalty and financial outcomes into a coherent model that can be used for ROI analysis:</P> <P></P> <P><STRONG>Operational measures </STRONG>often have a direct impact on transactional satisfaction. For instance, Oracle identified a clear negative correlation between the total time required to resolve a service request and the overall satisfaction with that service request: the longer it took to close the ticket, the less satisfied the customer was.</P> <P><STRONG>Transactional satisfaction </STRONG>in turn impacts customer satisfaction. Customers who were more satisfied with service requests were, according to the relationship survey, more satisfied overall with support services and product effectiveness. Such customers also reported higher value received from Oracle and greater loyalty.</P> <P><STRONG>Customer satisfaction </STRONG>and loyalty in their turn impact financial outcomes. Oracle was able to identify how much additional revenue for licenses and maintenance renewals would result from increasing the Executive Customer Loyalty Index by one rating point.</P> <P><STRONG>Financial outcome </STRONG>modeling lets you link customer behavior to the bottom line, validating the benefits of customer loyalty. Properly refined, the model can be used to predict the ROI of improvements, helping prioritizing those initiatives and guiding resource allocation. For instance, determining how much can be invested in improving response time based on the ultimate financial outcome deriving from greater transactional satisfaction.</P> <P>For a company as large as Oracle, this is not a one-size-fits-all linkage model. Segmentation is important, as the model varies by customer segment and by product line. Segmented linkage analyses have increased the sponsorship and engagement of senior leaders with the Voice of the Customer research and has proven itself year in, year out, as customer satisfaction with Oracle has steadily increased.</P> </article> <article> <h1>Survey Alerts Transform Survey Projects into Survey Processes</h1> <p>Jeffrey Henning — Tue, 20 Oct 2009 14:28:00 GMT</p> Adding <A href="http://blog.vovici.com/blog/bid/18667/Email-Trigger-a-Key-Aspect-of-EFM" mce_href="http://blog.vovici.com/blog/bid/18667/Email-Trigger-a-Key-Aspect-of-EFM">email triggers</A> to a traditional questionnaire starts the transformation from a survey project to a survey process. This is especially important for transactional surveys. To add <A href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" mce_href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">survey alerts</A> to an existing survey, you will need to consider the following issues:<BR> <UL class=unIndentedList> <LI><STRONG>Move from a random sample to an attempted census to maximize exposure to negative feedback</STRONG>. By replacing your <A href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sample</A> with an <A href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" mce_href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">attempted census</A> of everyone, you will have the opportunity to intervene with more individuals to attempt to improve their satisfaction. The trade-off to balance against this is that customers are now receiving surveys from you more often; this can be worth it, though, given the new opportunity to right previous customer-service wrongs. You are now going to catch more dissatisfied customers than before. Typically, for such <A href="http://blog.vovici.com/blog/bid/20553/Transactional-Surveys-that-Build-Customer-Loyalty" mce_href="http://blog.vovici.com/blog/bid/20553/Transactional-Surveys-that-Build-Customer-Loyalty">transactional surveys</A>, you will want to automate survey invitations, using input from an internal system such as a help-desk to drive invites. Implement touch-management rules so that people do not receive more than one survey of this type every 60 days.</LI> <LI><STRONG>Adding or expanding the series of reminders</STRONG>. Emailing <A href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" mce_href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">reminder invitations</A> to potential respondents who haven't yet taken the survey can be an annoying nuisance. In fact, we often argue for the need to respect your potential respondent's time and interrupt them as little as possible. Reminders are an interruption, but in the case of survey alerts, you really want to make sure unhappy customers raise their hands so that you can help them. A series of reminders, each spaced 3 to 5 business days apart, helps achieve this.</LI></UL> <BR> <UL class=unIndentedList> <LI><STRONG>Develop the business rules for customer-service responses</STRONG>. Some organizations simply have survey alerts delivered to a single email address, often that of customer service (for customer surveys) or the internal help desk (for employee surveys). Others use multiple questions for triggers, and set up elaborate business logic to notify the most appropriate staff to issues they can assist customers with. </LI> <LI><STRONG>Integrate with case management</STRONG>. Most organizations integrate email triggers into their existing case-management systems, so that survey alerts are prioritized and acted on alongside more traditional cases. Integration can be as basic as using the emails as inputs, to more sophisticated systems integration involving web APIs (Application Programming Interfaces).</LI></UL>You can certainly simply slap on a survey alert to an existing project, but we encourage you to plan a more detailed implementation of email triggers, to move from measuring customer satisfaction to intervening to improve it. </article> <article> <h1>E-mail List Rental Guidelines</h1> <p>Brian Koma — Mon, 19 Oct 2009 14:27:00 GMT</p> <P>E-mail list rental from reputable third-parties is one avenue for obtaining e-mail addresses to invite to <A href="http://www.vovici.com/" mce_href="http://www.vovici.com">online surveys</A>. When using a third-party list, keep these four guidelines in mind:</P> <OL> <LI><STRONG>Reputable providers will only rent the e-mail list to you</STRONG>: they will never sell you a list. If a list owner offers to sell you the full contact names and e-mail addresses of their list, you should strongly avoid using this type of list since it is most likely a non-permission-based list and could put you at risk for violating the <A href="http://blog.vovici.com/blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam" mce_href="http://blog.vovici.com/blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam">CAN-SPAM Act</A>.</LI> <LI><STRONG>Reputable providers will send your content out to their list from their e-mail system</STRONG>. You will not have the ability to review the names and their contact information nor will you have the opportunity to obtain a copy of the list. You'll only obtain the responses to your survey.</LI> <LI><STRONG>Reputable providers will only rent double opt-in lists</STRONG>. Double opt-in lists are created when an individual registers on a website and provides permission to contact them via e-mail. Once they have opted-in to the list, a confirmation e-mail is then sent to the e-mail address provided in their initial registration. Only by clicking on a link confirming that they are providing their permission for communications will they be added to the list. By requiring a second level of confirmation to the registered contact, list owners are assured that no one will be added to the list without the permission of the contact.</LI> <LI><STRONG>You pay for one-time use of the list regardless of the response rate to your survey</STRONG>. When you rent a list, all responsibility for obtaining responses is yours, and you must make sure that you have a compelling invitation and incentive in order to justify the expenditure.</LI></OL> <P>Please keep in mind that most research conducted using rented email lists is <A href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research" mce_href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research">qualitative</A>; because respondents were not selected randomly from a target population, you cannot extrapolate to a wider audience. The exception is when a magazine or website itself uses its own list to measure attitudes among its subscriber base; such surveys are typically projectable to the audience of readers. For some research, especially into low-incidence populations, qualitative studies using email lists are simply the only cost-effective approach.</P> </article> <article> <h1>Voice of the Customer Spoken Here</h1> <p>Jeffrey Henning — Fri, 16 Oct 2009 08:12:00 GMT</p> <P>Whenever someone discusses <A href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition" mce_href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition">Voice of the Customer</A>, they almost always use the phrase "listen to the Voice of the Customer". We need to <EM>speak </EM>Voice of the Customer as well.</P> <P>Too often, the typical questionnaire is written in the language of the employee, not the customer. It is written from the sponsoring organization's perspective, uses industry jargon and makes subtle distinctions that the average customer is oblivious too (see <A href="http://blog.vovici.com/blog/bid/18108/Multiple-Choice-Lists-Only-a-Product-Manager-Could-Love" mce_href="http://blog.vovici.com/blog/bid/18108/Multiple-Choice-Lists-Only-a-Product-Manager-Could-Love">Multiple Choice Lists Only a Product Manager Could Love</A>).</P> <P>All of this is bad news for a survey: the respondent gives less accurate answers, because they don't interpret questions the way they were intended. Respondents' frustration with the survey grows, prompting them to skip questions that aren't required, or - when running into required questions - to abandon the survey altogether.</P> <P>To me, an important goal of <A href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research" mce_href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research">qualitative research</A> is to learn how customers speak: the language they use and the words they select to describe what's important to them. In fact, VOC research can be a Rosetta Stone for questionnaire authors, helping them translate between EmployeeSpeak and CustomerSpeak. For instance, here's a question as it might have been written by an engineer, compared to that same question written for the customer:</P> <P></P> <P>The engineers in the audience are probably already alarmed that we are equating 2MB per MP3 file (e.g., per song). We're not; we're just using useful units. When it comes time to translate those into specs, the engineers will need to transform those requirements into gigabyte requirements using a rule of thumb for average song size. A pain, yes, for the engineers, but the survey data will be more accurate by embracing the customer viewpoint. Many customers - and here I am going to lose the engineers altogether - can't tell you how many KB or MB are in a GB; a gigabyte is too abstract a measurement for them.</P> <P>Speaking of abstract, we did consider using the question "How many <EM>hours</EM> of music do you want to keep on your music player?" but found that "songs" was a more natural unit of measurement than "hours".</P> <P>So, as you curl up to read thousands of verbatim comments or transcripts from your <A href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community" mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community">MROC</A> or focus group, try to absorb the language and word choice, not just the opinions and attitudes. Your surveys will be the better for it.</P> </article> <article> <h1>Research 2.0</h1> <p>Jeffrey Henning — Thu, 15 Oct 2009 14:17:00 GMT</p> <P>Having inherited a speaking opportunity that included the phrase "Research 2.0" (thanks, Marketing!), I found myself searching for a nice image to graphically convey whatever Research 2.0 might be. While I confess to finding the <A href="http://blog.vovici.com/blog/bid/18085/Web-1-0-vs-Web-2-0" mce_href="http://blog.vovici.com/blog/bid/18085/Web-1-0-vs-Web-2-0">Web 1.0 vs. Web 2.0</A> distinction useful, research is a little late to this 2.0 party (Enterprise 2.0, Marketing 2.0, Health 2.0, etc.). We can take solace only in the fact that the CRM 2.0 folks are even bigger dorks than we are:</P> <P></P> <P>Even Wikipedia wants no part of this:</P> <P></P> <P>And this from a site that lists <A href="http://en.wikipedia.org/wiki/List_of_Scooby-Doo,_Where_Are_You!_episodes" target=_new mce_href="http://en.wikipedia.org/wiki/List_of_Scooby-Doo,_Where_Are_You!_episodes">every episode of <EM>Scooby Doo, Where Are You!</EM></A></P> <P>Sadly, I was not able to find a nice graphic depicting Research 2.0. To me, the <EM>2.0 </EM>abbreviation is all about mash-ups and intellectual borrowing and remixing, so this was particularly disappointing. I wanted someone else to have done the heavy lifting using an ugly font and a hideous color scheme so I could "improve" it. Oh, well.</P> <P>Since I intended to talk about social media research and MROCs, my illustration of Research 2.0 focuses on social media research and MROCs. You could probably come up with a slide that makes Research 2.0 all about brain mapping and virtual-worlds ethnography, if you wanted. I wouldn't say anything against you if you did (though the brain map would show that I was thinking bad thoughts about you while playing Second Life).</P> <P>I did eventually warm to the task. I started with this graphic, which I've used in the past to distinguish research in <A href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">panels, communities and social networks</A>:</P> <P> </P> <P>Click <A href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">this link</A> for a full recap, but basically panelists only talk to the researcher, community members can talk to one another as well as the researcher, and social network members talk with other members not in the researcher's network.</P> <P>So that could be one dimension. The second dimension is whether the research is <EM>directed </EM>(for instance, with a questionnaire structuring the dialogue), <EM>moderated </EM>(a discussion guide, letting the respondent shape the discussion somewhat) or <EM>undirected </EM>(e.g., "let's see what people talk about"). Taken together:</P> <P></P> <P>Traditional methods:</P> <UL class=unIndentedList> <LI><STRONG>Survey</STRONG> - directed, one-on-one communications between the researcher and the respondent</LI> <LI><STRONG>Interview</STRONG> - moderated communications between the researcher and the respondent</LI> <LI><STRONG>Focus Group</STRONG> - moderated communications between the research and members of a group</LI> <LI><STRONG>Ethnography</STRONG> - undirected one-on-one between the research and the respondent </LI></UL>Research 2.0 methods:<BR> <UL class=unIndentedList> <LI><STRONG><A href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community" mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community">MROC</A></STRONG> (Marketing Research Online Community) - somewhat between moderated and undirected communications of the members of a group</LI> <LI><STRONG>Social Media Research</STRONG> - analysis of undirected conversations between members of a social network, with many conversations taking place outside of the sight of the researcher.</LI></UL> <P>I hope those of you forced to talk about Research 2.0 might find this to be a useful illustration.</P> <P>Note to Marketing: You put "Research 3.0" in a webinar title, and you're writing that slide deck!<BR></P> </article> <article> <h1>Evaluations of Vovici Research Webinars</h1> <p>Jeffrey Henning — Wed, 14 Oct 2009 15:20:00 GMT</p> <P>I wanted to share the results of our webinar evaluations. After each complimentary webinar, we send attendees an email with a link to a recording of the webinar, a link to download the PowerPoint slides and a link to a short survey.</P> <P>In the survey, I ask for satisfaction, likelihood to recommend this webinar and likelihood to attend another webinar. I also ask my own version of the three <A href="http://blog.vovici.com/blog/bid/18288/Customer-Experience-Management-with-the-Forrester-CxPi" mce_href="http://blog.vovici.com/blog/bid/18288/Customer-Experience-Management-with-the-Forrester-CxPi">Forrester CxPi</A> questions. The results:</P> <UL class=unIndentedList> <LI>76% of respondents are very or completely satisfied. Being a perfectionist, I want to increase the percent that are completely satisfied (currently 22%).</LI> <LI>68% are very or completely likely to recommend the webinar they're rating.</LI> <LI>84% are very or completely likely to attend another Vovici webinar.</LI> <LI>54% report the webinar meets most or all of their needs-barely more than half! This is the area that we need to work on most; though, from reading the verbatim comments, its clear that attendees often have very different expectations.</LI> <LI>88% report that it was very or extremely easy to participate in the webinar.</LI> <LI>68% report that the webinar was very or extremely enjoyable. This measure is important to me because I think attendees will learn more and retain more if the webinar is enjoyable.</LI></UL> <P>If you haven't attended a Vovici <A href="http://www.vovici.com/about/webinars.aspx" mce_href="http://www.vovici.com/about/webinars.aspx">research webinar</A> yet, you can view a recording online. Here are how seven recent webinars rated by satisfaction and usefulness.</P> <P><BR><BR>Why did I pick these two dimensions? Because they had the highest standard deviations of the six measures, meaning they had the widest distribution of values and the greatest contrast. In other words, these two measures would make the most dispersed and interesting chart!</P> <P>Some of the representative positive feedback:</P> <UL class=unIndentedList> <LI>"It was informative and impartial -- not pushing Vovici products particularly. It also is a very timely topic."</LI> <LI>"Even though I have been in market research for nearly 20 years, I thought that this was surprisingly insightful as to the current challenges. Nicely done!"</LI> <LI>"Excellent content, great pace and presentation. Speakers were fluent in how to use the technology for the presentation, which is always a bonus. Following along on Twitter was fun, too."</LI> <LI>"I liked everything about it. Concrete, actionable advice from start to finish."</LI> <LI>"I liked how it was very organized and to the point with all of the information."</LI> <LI>"The content was contemporary, relevant, informative and effectively communicated verbally and in the visuals."</LI> <LI>"The presenters were very knowledge and interesting. Great slides and I appreciated that my questions were answered immediately."</LI></UL>And some of the representative constructive criticism:<BR> <UL class=unIndentedList> <LI>"A bit too long."</LI> <LI>"Content was more basic than expected."</LI> <LI>"Disappointed that there was extremely limited time for Q&A."</LI> <LI>"Both days I had to adjust the volume every time the speaker changed. One spoke softly & the other spoke loudly. That gets annoying."</LI> <LI>"I submitted a question to the presenter and he never answered it. I waited until the end and they said there were no further questions."</LI> <LI>"Some of the content was a repeat of a previous webinar I viewed from Vovici."</LI> <LI>"This presentation was bent towards online surveys."</LI></UL> <P>We take it all to heart and try to improve with each iteration we do of a webinar. When it comes to presentations, it is very hard to please everybody all of the time. You often have two contradictory quotes on one subject. For instance, I'm very proud of the <EM>Surveys & Social Media</EM> presentation, because--in the words of one respondent--it provides "fresh ideas in an area that will be a big part of future qual"; of course, another attendee said, "a lot of the information was very basic". (Ah, well, one man's BASIC is another man's FORTRAN.)</P> <P>Please go here to see our library of <A href="http://www.vovici.com/about/webinars.aspx" mce_href="http://www.vovici.com/about/webinars.aspx">research webinars</A>: view an archived copy or register for an <A href="http://www.vovici.com/about/news-and-press/2009/fallwebinars.aspx" mce_href="http://www.vovici.com/about/news-and-press/2009/fallwebinars.aspx">upcoming webinar</A>!</P> </article> <article> <h1>Vovici Releases Next Generation Community & Panel Management Platform</h1> <p>Roderick Morris — Tue, 13 Oct 2009 15:34:00 GMT</p> <P><A href="http://www.flickr.com/people/72825507@N00" target=_new mce_href="http://www.flickr.com/people/72825507@N00"></A>Yesterday was Columbus Day in the United States-always an odd celebration, as we remember a man who discovered a new world (new to Europeans anyway) and yet insisted to his grave that he had just discovered a shortcut to an old world. Vovici celebrated Columbus Day with the latest release of our flagship product. Shortcut to the old world or unlocking a new world?</P> <P>Our <EM>Niña, Pinta </EM>and <EM>Santa Maria </EM>are our three main modules: Survey Workbench, Community Builder and Feedback Intelligence. For this new release, we've extended the functionality of two of the modules. </P> <UL class=unIndentedList> <LI>For the Community Builder, where organizations conduct qualitative research, we've streamlined the registration process for new members and added support for open membership communities (previous communities were invitation only). And we've made it easier for organizations to move from their existing communities by supporting bulk loading of new members and bulk permission-setting for existing members. We've added or enhanced modules for polling, idea voting and member profiles.</LI> <LI>For the Survey Workbench, where organizations structure quantitative research, our new panelist health scoring tracks the participation of individual panelists and of the panel overall. Of course, the value of panels is converting panelists into respondents: survey authors can now view and report on the delivery, unsubscribe, open, and click-through rates for any survey-invitation or reminder e-mail campaign. For appropriately configured surveys, panelists can see how their answers compare to other responses so far. </LI></UL> <P mce_keep="true">We've made 25 significant additions to the product and many smaller additions, based on the feedback of hundreds of customers. In some ways, these may seem like incremental changes to a core platform: shortcuts to the old world. But Market Research Online Communities (<A href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community" mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community">MROCs</A>) and <A href="http://blog.vovici.com/blog/bid/18028/Panel-Management-101" mce_href="http://blog.vovici.com/blog/bid/18028/Panel-Management-101">survey panels</A> enable a new world of research: use the community for <A href="http://blog.vovici.com/blog/bid/17989/Ideas-Prioritized-Faster-than-Ever" mce_href="http://blog.vovici.com/blog/bid/17989/Ideas-Prioritized-Faster-than-Ever">ideation</A> and general feedback, discovering items of importance to your customers that you never would have thought to survey, then use the panel to quantify how widespread interest in these ideas is, so that you set the correct course through many priorities.</P> <P mce_keep="true">Almost every other feedback platform out there is either about community or panels. Both have their weaknesses when used in isolation. This latest release of the Vovici platform represents the best of both quantitative and qualitative worlds. And the extent of the value of that discovery is only beginning to be known.</P> </article> <article> <h1>Use Multiple Questions per Page of a Web Survey</h1> <p>Jeffrey Henning — Mon, 12 Oct 2009 14:53:00 GMT</p> <P>I'm frequently asked whether it is better to show a single question per page or to show multiple questions per page. My advice has been to group questions together logically on a single page, using different pages to change topics.</P> <P>Since that advice was intuitive rather than empirical, I decided to do an experiment. My control was a 5-page, 9-question questionnaire where some pages had one question and some pages had two or three. My experiment took the same version of the questionnaire but added a page break after the introduction and placed one question per page: this made it twice the length, at 10 pages. It was the exact same questionnaire, differing only in pagination. (For the record, both versions displayed a progress bar indicating how far the respondent was through the survey.)</P> <P>After only 44 survey starts, the abandonment rate had jumped from 5% across 79 starts of my 5-page survey to 25% for its 10-page equivalent. I cut the experiment short due to the significant impact the additional pages had on abandonment.<BR>Broadening my analysis to include some surveys that differed from one another in other ways, I found an 18% abandonment rate for several 10-page surveys with one question per page vs. an abandonment rate of 3% for two surveys with logical groupings of questions.</P> <P></P> <P>Clearly, the need to click the Next button and wait for a new page of the survey to load represents enough additional effort to respondents that it discourages them from completing a survey.</P> <P>Despite the small sample sizes, this is not an experiment I will be repeating anytime soon! When writing your own questionnaires, to maximize completion rate, avoid showing one question per page.</P> </article> <article> <h1>CRM Best Practices for Customer Experience Management</h1> <p>Brian Koma — Fri, 09 Oct 2009 08:06:00 GMT</p> <P>According to the Vovici <A href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx" mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx">CE IQ</A> study, organizations that have the most loyal customers make sophisticated use of their Customer Relationship Management systems.</P> <P>Nothing undermines loyalty of your customers more than when your organization demonstrates that it doesn't know who your customers are, what they have bought or how they have recently interacted with you. Too often survey authors ask customers to provide basic details about themselves rather than crossreference that information from a CRM system.</P> <P>By associating known information about the customer - such as name, company name, product/service history, support interactions and other relevant data - with requests for unknown transactional, attitudinal or experiential data, organizations can engender much higher degrees of customer loyalty. This approach achieves two ends: First, it shows customers that you're treating them as individuals and not as numbers; second, it allows organizations to ask fewer questions but get better data. By no longer needing to ask customers to provide answers to questions which you already have answers to in a CRM system, it's possible to ask fewer questions but get better data by further reducing feedback fatigue. How valuable was this practice? With a 0.57 loyalty correlation, it was second only to having a <A href="http://blog.vovici.com/blog/bid/20534/CE-Best-Practices-Require-Strategic-Commitment" mce_href="http://blog.vovici.com/blog/bid/20534/CE-Best-Practices-Require-Strategic-Commitment">strategic commitment to customer experience management</A>. </P> <P> </P> <P><BR>Sadly, many organizations have not yet standardized on a single CRM system across the organization, and those that have fail to integrate other applications with the CRM system where appropriate, enshrining the CRM system as the central data repository for customer information.</P> <P>Only by appending CRM data to attitudinal information can organizations build a complete 360 degree view of their customer and their customer's experience with the organization. These four practices truly leverage CRM to build customer loyalty.</P> </article> <article> <h1>AMA MRC 2009 Roundup</h1> <p>Jeffrey Henning — Thu, 08 Oct 2009 15:24:00 GMT</p> <P>I'm just back from the 30th annual AMA Marketing Research Conference and had an article published at <A href="http://www.research-live.com/" mce_href="http://www.research-live.com/">Research Live</A> summarizing the events: <A href="http://www.research-live.com/comment/three-tips-for-a-clear-picture-of-your-market-zoom-in-zoom-out-and-focus/4001107.article" mce_href="http://www.research-live.com/comment/three-tips-for-a-clear-picture-of-your-market-zoom-in-zoom-out-and-focus/4001107.article">Three tips for a clear picture of your market--Zoom in, zoom out and focus</A>. As a companion piece to that article, I have sorted and organized the 13 presentations that I was able to document. My apologies to the great presenters I wasn't able to get to. Hats off to the AMA, the Conference Committee and especially conference chairman <A href="http://brainzooming.blogspot.com/" mce_href="http://brainzooming.blogspot.com/">Mike Brown</A> for a phenomenal conference!</P> <P><STRONG>Practical advice for market researchers:</STRONG></P> <UL> <LI>How to conquer the complexities of <A href="http://blog.vovici.com/blog/bid/22237/Conquering-the-Complexities-of-Research-in-Business-Markets" mce_href="http://blog.vovici.com/blog/bid/22237/Conquering-the-Complexities-of-Research-in-Business-Markets">B2B market research</A> (Strategic Spark)</LI> <LI>How to conduct <A href="http://blog.vovici.com/blog/bid/22244/To-Price-or-Not-to-Price-That-Is-the-Question-Predicting-the-Effect-of-Price-Increases-A-Non-Technical-Explanation" mce_href="http://blog.vovici.com/blog/bid/22244/To-Price-or-Not-to-Price-That-Is-the-Question-Predicting-the-Effect-of-Price-Increases-A-Non-Technical-Explanation">pricing research</A> during economic turmoil (Del Monte)</LI> <LI>How to use <A href="http://blog.vovici.com/blog/bid/22189/Social-Media-as-One-Medium-to-Consider-in-Media-Planning" mce_href="http://blog.vovici.com/blog/bid/22189/Social-Media-as-One-Medium-to-Consider-in-Media-Planning">media planning research</A> to calculate how much budget to allocate to social media (Carat.com)</LI> <LI>Top 10 <A href="http://blog.vovici.com/blog/bid/22240/Finding-Out-What-You-Don-t-Know-Top-10-Things-Researchers-Don-t-Realize-About-Research" mce_href="http://blog.vovici.com/blog/bid/22240/Finding-Out-What-You-Don-t-Know-Top-10-Things-Researchers-Don-t-Realize-About-Research">market researcher blindspots</A> (Georgia Pacific)</LI> <LI>Market researchers must <A href="http://blog.vovici.com/blog/bid/22224/Market-Research-Blogs-Social-Networks-and-Community-Building-or-Leveraging-Communities-to-Tap-for-Market-Insights" mce_href="http://blog.vovici.com/blog/bid/22224/Market-Research-Blogs-Social-Networks-and-Community-Building-or-Leveraging-Communities-to-Tap-for-Market-Insights">engage communities not lurk</A> in them (Floor64)</LI> <LI>How to shift research to <A href="http://blog.vovici.com/blog/bid/22164/Questions-about-Panels-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/22164/Questions-about-Panels-Online-Communities">panel management and online communities</A> (Vovici)</LI></UL> <P><STRONG>Working with senior management:</STRONG></P> <UL> <LI>How to achieve <A href="http://blog.vovici.com/blog/bid/22175/You-Can-t-Be-Brilliant-Alone-How-to-Achieve-Influence-Without-Authority-through-Effective-Collaboration" mce_href="http://blog.vovici.com/blog/bid/22175/You-Can-t-Be-Brilliant-Alone-How-to-Achieve-Influence-Without-Authority-through-Effective-Collaboration">influence without authority</A> through effective collaboration with management and clients (American Express)</LI> <LI>How to embrace behavioral research to successfully challenge <A href="http://blog.vovici.com/blog/bid/22193/Why-Your-Management-Doesn-t-Listen-to-You-Like-You-Think-They-Should" mce_href="http://blog.vovici.com/blog/bid/22193/Why-Your-Management-Doesn-t-Listen-to-You-Like-You-Think-They-Should">management's sacred cows</A> (TimeWarner)</LI> <LI>How to align <A href="http://blog.vovici.com/blog/bid/22178/Deviant-Leadership-for-Researchers" mce_href="http://blog.vovici.com/blog/bid/22178/Deviant-Leadership-for-Researchers">market research with corporate strategy</A> and influence that strategy (YRC)</LI></UL> <P><STRONG>Strategic planning:</STRONG></P> <UL> <LI>How to use <A href="http://blog.vovici.com/blog/bid/22192/Using-Human-and-Cultural-Insights-to-Anticipate-Consumer-Desires" mce_href="http://blog.vovici.com/blog/bid/22192/Using-Human-and-Cultural-Insights-to-Anticipate-Consumer-Desires">human and cultural insights</A> to anticipate customer desires (Coca Cola)</LI> <LI><A href="http://blog.vovici.com/blog/bid/22170/Envisioning-the-Future-at-Ford" mce_href="http://blog.vovici.com/blog/bid/22170/Envisioning-the-Future-at-Ford">Ten macro trends</A> to factor into your scenario planning (Ford)</LI> <LI><A href="http://blog.vovici.com/blog/bid/22224/Market-Research-Blogs-Social-Networks-and-Community-Building-or-Leveraging-Communities-to-Tap-for-Market-Insights" mce_href="http://blog.vovici.com/blog/bid/22224/Market-Research-Blogs-Social-Networks-and-Community-Building-or-Leveraging-Communities-to-Tap-for-Market-Insights">Forget strategic planning</A>: anticipate, act and adjust (Citi Group)</LI> <LI>Leverage seven triggers to create products (and careers) that persuade, captivate and <A href="http://blog.vovici.com/blog/bid/22215/Be-Fascinating" mce_href="http://blog.vovici.com/blog/bid/22215/Be-Fascinating">fascinate</A> (Sally Hogshead)</LI></UL> <P>Don't wait for next year's conference: remember, you can engage year round in the <A href="http://amasigs.marketingpower.com/marketingresearch" mce_href="http://amasigs.marketingpower.com/marketingresearch">AMA MR SIG</A> online community!</P> </article> <article> <h1>To Price or Not to Price, That Is the Question: Predicting the Effect of Price Increases - A Non-Technical Explanation</h1> <p>Jeffrey Henning — Wed, 07 Oct 2009 12:05:00 GMT</p> <P>Eliot Roth, Senior Manager of Pet Custom Research for Del Monte, began saying, "From the beginning of time, man has been consumed with one great question: How much do I charge?" Businesses have succeed or failed on this. Like the Chinese curse, we live in interesting times. Del Monte has seen hyperinflation in commodities and then deflation, causing a rethinking of pricing research. The new world challenge is "out of bounds" pricing. The hyperinflation of commodities has pushed us outside the range of past models.</P> <P>The three traditional pricing models are cost-based pricing (which Finance likes), competitive pricing (which Marketing likes) and economic pricing of the supply and demand curve (which economists like). In the real world, multivariate regression can be used to model pricing effects. For elasticity models, start with a good data source (for CPG, IRI and Nielsen are the suppliers, and Del Monte is a Nielsen house) and then control for other market forces (e.g., store size, seasonality, promotions, direct and indirect competition). </P> <P>Key things to keep in mind:</P> <UL> <LI><STRONG>Use relative pricing, not absolute pricing</STRONG>. What happens when everyone raises their price? Increased fuel costs and commodities costs have raised the pricing of entire categories. The effect differs depending on the category: if it is not discretionary, people still buy it (gasoline doubled but consumption dropped only 5%), but other categories see shifts to substitutes. </LI> <LI><STRONG>Relative pricing needs to consider item elasticity and category elasticity</STRONG>. We did the math of the price increase separately: 3% price increase * 1.5 elasticity (4.5%) for item + 4% category component * 0.5 elasticity for 6.5% elasticity. This was the most predictive model for us.</LI> <LI><STRONG>Use interaction levels to calculate category price increases</STRONG>. The relative category is defined by evaluating the item's competitors that interact most with it, using a weighted evaluation by interaction.</LI> <LI><STRONG>Avoid over-fitting models</STRONG>. People overfit the model, with too many variables providing better fit statistics (one model had 4,800 factors) at the expense of predictivity.</LI> <LI><STRONG>Do not mix sizes</STRONG>. Mixing different priced (different sized) SKUs introduces noise: for instance, a discount on the larger size can increase the average price.</LI> <LI><STRONG>Transform variables</STRONG>. Avoid overly simplistic models, such as traditional linear models. Data models need to be transformed or normalized to have a better fit and to be more predictive.</LI></UL> </article> <article> <h1>Finding Out What You Don’t Know: Top 10 Things Researchers Don’t Realize About Research</h1> <p>Jeffrey Henning — Wed, 07 Oct 2009 11:23:00 GMT</p> <P>David Weinberger, VP of Insights, Georgia-Pacific, discussed the top 10 items researchers often don't recognize about their work:</P> <P>10) <STRONG>Understand the business</STRONG>. Who is the target customer? How does the business make money? Connect the dots between different functions? Home Depot only learnt a few years ago that 3% of their customers made up 30% of their revenue.</P> <P>9) <STRONG>You don't need to reinvent the wheel</STRONG>. As an industry, we have a tremendous literature representing a wealth of knowledge going back a long time. Motivational research was developed in 1947 and continues to evolve in brand-strategy development; the volume forecasting model was created in 1960 by Fourt and Woodlock. You need to seek out these old and valuable methods.</P> <P>8) <STRONG>Develop repeatable processes and frameworks. </STRONG>It's faster, cheaper, provides norms and cross-study comparisons. For instance, build a standard advertising-effectiveness model for your organization.</P> <P>7) <STRONG>It's 10:00 in the morning. Do you know what your customer is doing?</STRONG> One hundred years ago, running a country grocer or butcher shop, you'd know everything about your customers. Look at ethnography, store and lifestyle shadowing, deep dives and immersion excursions to truly learn customer behavior.</P> <P><A href="http://www.amazon.com/Your-Purse-Archaeology-American-Handbag/dp/1434317064/ref=sr_1_1?ie=UTF8&s=books&qid=1254933150&sr=8-1" mce_href="http://www.amazon.com/Your-Purse-Archaeology-American-Handbag/dp/1434317064/ref=sr_1_1?ie=UTF8&s=books&qid=1254933150&sr=8-1"></A>6) <STRONG>Try new insightful approaches to understand unmet customer needs</STRONG>. Do you know the definition of insanity? Doing the same thing over and over again and expecting a different result. Corollary: "If you do what you did, you will get what you got." Look at deep qualitative research, ethnography and cultural insight. Check out <EM>In Your Purse: Archeology of the American Handbag </EM>for a study of women's handbags, which uncovered billions of dollars in product innovation and marketing opportunities.</P> <P>5) <STRONG>Learn to speak finance</STRONG>. Learn to translate your research results into ROI: for instance, quantify sales lost due to customer problems.</P> <P>4) <STRONG>Can you write a one pager?</STRONG> Tell me the "What" and the "So What". Give your clients the one page with the key findings and the key recommendations. Talking in threes helps people remember it: summarize in threes. Put the insight at the top and the implication at the bottom. </P> <P>3) <STRONG>Develop actionable insights that drive execution</STRONG>. Use insight strategically and holistically to increase sales and profitability. BCG will publish a study of 45 organizations and how the value of the market research is bringing a broad perspective to the organization.</P> <P>2) <STRONG>Focus on leadership and execution</STRONG>. Inform the organization with all the research.</P> <P>1) <STRONG>Don't forget to sharpen your saw</STRONG>. Read, participate and invest the time to improve.</P> </article> <article> <h1>Conquering the Complexities of Research in Business Markets</h1> <p>Jeffrey Henning — Wed, 07 Oct 2009 11:06:00 GMT</p> <P>Barb Murphy, president of <A href="http://www.strategicspark.net/" mce_href="http://www.strategicspark.net/">Strategic Spark</A>, discussed conquering seven objections and complexities of business-market research. Before starting her own firm, Barb had roles within a research supplier, ad agency and then client side. Business markets aren't just B2B; for traditional B2C, business markets include research into the businesses that distribute products to resellers or that resell to consumers. </P> <P>Researchers need to bring clarity to murky situations. Constituencies to research include end users, distribution channels, strategic partners, employees, competitors, industry analysts, media representations and government entities. Supplemented by secondary research, a "dimensional perspective" brings all these viewpoints together to develop scenarios.</P> <P>You need to balance the views of the channel as customer and the end user as customer. When researching a business market, who do you interview? The decisionmaker, the evaluators/recommenders, indirect decision influencers, purchasers? Individually or in combination? For instant, for call center software, you need to interview the call center manager, the IT manager, the CFO and the CEO. Barb transformed what was planned as a survey of 200 call-center managers into a series of dyads and triads. A good qualifying question that casts a wide net within an organization: "Are you involved in evaluating the ‘product' your company purchases?" </P> <P>Despite all its complexity, B2B marketing spend often represents only 20-25% of B2C's spend. Key reasons:</P> <UL> <LI>Fewer customers to target</LI> <LI>Relationship focused B2B culture places lower value on marketing</LI> <LI>Complex to target and message all the roles that influence market decisions</LI></UL> <P>How do you overcome objections to B2B research? "Internal objections are simply great opportunities to educate management and transform your ‘research' into a market insights strategist," said Barb. The seven objections:</P> <OL> <LI><STRONG>We already know that</STRONG>. "Our sales force and service reps talk to customers every day; we know them." To conquer this objection, harness the power of internal experience and insight. Research these employees to learn what they know.</LI> <LI><STRONG>They just won't get it</STRONG>. "Our products are too technical, too complicated, too complex. It's our job to come up with new ideas. Our customers just aren't smart enough." To conquer this objection, institute collaborative insight planning sessions that brins everyone together and have them help identify the types of information they need to make a decision.</LI> <LI><STRONG>It's too complex</STRONG>. "No survey is going to help us understand our business customers-too many entities influence the decision." To conquer this objection, engage experts, influencers and lead users to find out how they might respond to the subject being researched. For instance, for truck emission technologies needed to cope with new regulation, experts and customers did in fact have a handle on a complex subject.</LI> <LI><STRONG>It's too small</STRONG>. "We only have 3,000 business customers vs. 3 million consumers. It's a waste of money to have formal research with such a small group; let's just do a focus group!" To conquer this objection, right-size the approach and provide phased study options that integrate qualitative and quantitative research.</LI> <LI><STRONG>You'll never find them</STRONG>. "Even if we wanted to reach them, we don't know how. We have customer contact information, but we don't know how to find and reach the potential users within our customers." To conquer this objection, if you can find a sample solution, you help crack the "lead generation code". If they can't find out who to research, they are probably having difficulty finding out who to sell to! Take a research role and help solve a much bigger problem. </LI> <LI><STRONG>They'll never tell us that</STRONG>! "You just don't know how small and competitive our industry is. Why would experts, customers and competitors share this information?" This is the hardest objection to conquer. To do so, start early, generate and distribute sound bites throughout the study. "Research isn't just about the beginning and end of a project," said Barb; you can provide guidance throughout the project.</LI> <LI><STRONG>It's way too expensive</STRONG>. "We just can't afford it. It seems so much more expensive than our B2C research." To conquer this objection, you need to educate your clients about the drivers of B2B research cost: less sample to draw from, the time required to network to find the right contacts, declining cooperation rates (often driven by company policy), complexity of research that require detailed industry knowledge, need for higher incentives (contributions to charities works well), expense of hybrid research methodologies, all with comparatively fewer research firms specializing in B2B research.</LI></OL> <P>This was a great discussion of a common type of research that does not receive enough attention within organizations or within the market-research industry itself.</P> </article> <article> <h1>Market Research, Blogs, Social Networks and Community: Building or Leveraging Communities to Tap for Market Insights</h1> <p>Jeffrey Henning — Tue, 06 Oct 2009 13:41:00 GMT</p> <P><A href="http://www.techdirt.com/rtb.php?tid=400" target=_new mce_href="http://www.techdirt.com/rtb.php?tid=400"></A>Mike Masnick, CEO of <A href="http://www.floor64.com/" target=_new mce_href="http://www.floor64.com">Floor64.com</A>, started the blog <A href="http://www.techdirt.com/" target=_new mce_href="http://www.techdirt.com">TechDirt</A> in 1997, which is now rated #75 on Technorati and #53 on BlogPulse. Mike is also the author of <EM>Approaching Infinity</EM>, about the rise of new abundances: abundant access to ideas and people, with people abundantly willing to join communities, abundantly willing to share, in ways that weren't even possible a few years ago.</P> <P>Despite the title of his AMA MRC presentation, Mike said, "Don't say that you are <EM>building </EM>communities, but that you are <EM>enabling </EM>communities." Thanks to the Internet, communities easily assemble to discuss diverse and oftentimes narrow topics. </P> <P>Mike contends that what is different for market researchers about online communities is that researchers must engage the audience. Listening and watching in a community is called <EM>lurking</EM>, which is a bad word in the community world, because it means that you are not participating. Instead, through engagement, you can find out what people really want, not just what they say they want. To truly understand a community, you have to really engage that community. Think of the added level of understanding you have of a friend compared to an acquaintance: engaging a community offers that greater level of detail over simply observing a community. </P> <P>Engagement builds trust and respect and opens up opportunities for creative dialog.</P> </article> <article> <h1>Don't Look Back, Anticipate Instead - Forecasting Changes in the Marketplace</h1> <p>Jeffrey Henning — Tue, 06 Oct 2009 11:40:00 GMT</p> <P>At today's AMA Marketing Research Conference, Ravi Parmeswar, VP Global Consumer and Marketplace Insights, Citi Group, discussed how to anticipate changes.</P> <P>Think of the housing crisis like the tornado grabbing Dorothy and her house and whisking her to a strange new land. What should Dorothy do?</P> <P>Ravi began with a recap of failed predications he had witnessed in his career: the prediction in the 1970s the world would be out of oil by 2010, that microwaves would radically change the kitchen, that the Internet would be the death of brick and motor, and that concerns about health and weight would lead to the death of fast food.</P> <P>Linear extrapolation is dangerous. The outcome is highly non-linear with significant interaction effects. Organizations need to plan for several outcomes. A ball, released on the same terrain with the same initial conditions, will roll down to a very different place, driven by many variables as the ball rolls down. The first blade of grass it hits can change the entire sequence of events. </P> <P>As a result, organizations need to forget the old planning rules and prescribed steps but look instead at multiple scenarios and multiple futures. Anticipate what will happen tomorrow, plan to Act on it then Adjust. </P> <OL> <LI><STRONG>Anticipate: Develop your organization's peripheral vision and sensing</STRONG>. "Most organizations have maverick employees with insights about the periphery, but these individuals are rarely tapped." - George Day & Paul Schoemaker, "Scanning the Periphery". To anticipate our new normal at Citi, we find social networks incredibly helpful.</LI> <LI><STRONG>Act now: Long-term planning is dead</STRONG>. "Strategy - as we knew it - is dead: welcome to the fast strategy game." - Yves Doz & Mikko Kosonen, Fast Strategy: How Strategic Agility Will Help You Stay Ahead of the Game. </LI> <LI><STRONG>Adjust continuously: Tweak the plan or completely revamp the plan?</STRONG> Implement more measuring, with multi-dimensional measures; too often dashboards don't change unless management changes. Agility is paramount. The old way was to have a 5-year plan. The new way is to change course. </LI></OL> <P>The future belongs to those companies with a bias towards action and re-evaluation. Ignore the wicked witches that would distract us. Be our own wizard and create our own future.</P> </article> <article> <h1>Be Fascinating</h1> <p>Jeffrey Henning — Tue, 06 Oct 2009 10:04:00 GMT</p> <P>At the AMA MRC, Sally Hogshead, author of <EM>Radical Careering</EM>, discussed her new book, <EM>Fascinate: Your 7 Triggers to Persuasion and Captivation</EM>, due out next February.</P> <P>Sally began with a recap of the tulip craze. Tulips bloom in the spring for one week, yet people would trade their life savings for a bulb. Economists call this the first economic bubble. In the moment, when people get fascinated, when their perception of value has been driven up, things get out of whack. When people realized that tulip bulbs weren't worth 12 acres of land, the Dutch economy crashed.</P> <P>When marketers bring fascination to their clients' products, it raises the value of those products. Take something that is not innately fascinating and find something fascinating with it.</P> <P>Consumers want to buy things to make them fascinated. Most brands are purchased so that the person can feel more fascinating to others. The more fascinating, the more we can charge. Based on research that Sally did, she found that women will spend more to be fascinating than they spend on food and clothes combined: willing to spend 15% of their budget. If you make your product fascinating, you can access this market.</P> <P>Most marketing messages fail badly. They are not fascinating. Here are seven triggers to persuade and captivate:</P> <OL> <LI>Mystique: Intrigue with unanswered questions.</LI> <LI>Prestige: Social rank and respect.</LI> <LI>Vice: Temptation with "forbidden fruit".</LI> <LI>Power: The ability to command and control.</LI> <LI>Lust: The craving of pleasure.</LI> <LI>Alarm: Threat of negative consequences.</LI> <LI>Trust: Certainty based on experience.</LI></OL> <P>When you map brands to these triggers, you find that Coke is very high on lust and trust, high on vice and alarm, lower on mystique and power. FedEx is very high for alarm and trust and is high on power and prestige. Kmart has low values for most triggers, except for trust.</P> <P>Potential fascination badges:</P> <UL> <LI>Purpose</LI> <LI>Core beliefs</LI> <LI>Heritage and culture</LI> <LI>Products</LI> <LI>Benefits</LI> <LI>Actions</LI></UL> <P>And some gold hallmarks of a fascinating brand:</P> <UL> <LI>Provokes strong, immediate reactions</LI> <LI>Creates advocates</LI> <LI>Becomes "cultural shorthand"</LI> <LI>Incite conversation</LI> <LI>Forces competitors to realign</LI> <LI>Triggers social revolutions</LI></UL> <P>Keep in mind that these may not be the most liked brands. For your own brand, don't be fascinating for the sake of being fascinating. Be fascinating for the sake of communicating the ideas that resonate with your brand.</P> </article> <article> <h1>Why Your Management Doesn’t Listen to You Like You Think They Should</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 19:52:00 GMT</p> <P>Jack Wakshlag, Chief Research Officer of TimeWarner/Turner Broadcasting, discussed the disconnect between management and market researchers. He boiled it down to two primary problems: the elephant in the room and the sacred cow.</P> <H4>The Elephant in the Room</H4> <P>The elephant in the room for the market research industy is the problem or issue that is obvious but ignored: if you want to know how consumers use anything or why, just ask them. What happens when we ask people to think about and tell us what they do or why? They will tell us something. It's just not a very reliable something. "While people are very willing and very good at volunteering information explaining their actions, those explanations ... aren't necessarily true," wrote Malcolm Gladwell in <EM>Blink</EM>. </P> <P>Consumers are not conscious, rational beings; in fact, consumers are often not aware of why they behave as they do. What consumers tell us often reflects their best guesses or what they feel is socially acceptable.</P> <P>Bad research generates and perpetuates this myth of consumer self-awareness. Jack emphasized: "When good researchers fail to challenge bad research, all researchers--including the good researchers--lose credibility. When we report what people say-while behavioral data tells us what people actually do - we lose."</P> <P>There is a fundamental disconnect when researchers provide advice based on what people say when management makes money on what people do.</P> <P>Focus on behavioral metrics for your brand - KPIs to gauge the progress and relative strength of your brands. Management wants to win: they want something to follow that tells them how they are doing.</P> <H4>Sacred Cows</H4> <P>Your managers have certain beliefs: how do you deal with those certain beliefs when your results call them to question? "Don't be afraid to challenge sacred cows - in fact, relish the opportunity!" Of course, make sure to have solid data, be prepared to answer detailed questions and offer explanations for why the sacred cow is, in fact, a false hypothesis. Because it can be contentious, do it at the right time in the right place and the right way.</P> <P>Jack encountered this in the television industry with the "Appointment Viewing" fable - the belief that, because program ratings don't change much from day to day or week to week, it seems to be the same core group of viewers. This myth has many incarnations [reincarnated sacred cows?!]:</P> <UL> <LI>People generally watch the same shows week in and week out</LI> <LI>Appointment viewing is critical to the success of a show</LI> <LI>"Our show has a small but a loyal core audience."</LI></UL> <P>To study this, Turner Research analyzed 44 different shows across many networks and genre types. The study included a survey where programmers and marketing staff gave their perception of how many of 10 episodes the average viewer watched.</P> <P>On average, across the 44 different shows, people watched 2.5 episodes out of 10. Rather than show many charts and graphs, show them one big number. </P> <P>Appointment viewing is a myth, but beliefs in myths don't go down easily. You have to provide good reasons to support your refutation. For this sacred cow, Jack pointed out:</P> <UL> <LI>Only 50% of those who view a given program one day are even watching TV at the same time the following day.</LI> <LI>Everybody has a few shows they hate to miss, but it really is only a few; you can count them on one hand.</LI> <LI>These "can't miss" shows do not account for a majority of most people's viewing: a large proportion of time is spent with other shows.</LI> <LI>Our recollections are wrong: we remember those few shows we intended to watch and made sure we watched, and don't recall those other shows that we happened upon.</LI></UL> <P>So, if you want management to listen to you, you must become a more effective communicator.</P> </article> <article> <h1>Using Human and Cultural Insights to Anticipate Consumer Desires</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 17:29:00 GMT</p> <P>Tom Laforge, Global Director of Human & Cultural Insights, with The Coca Cola Company presented at the AMA Marketing Research Conference. Human insights are universal and unchanging, based on biology, lifestages, psychological motivations and ergonomics. Cultural insights change slowly: lifestyles, values and beliefs, trends, macroforces and worldviews. Coca-Cola has a cultural researcher in almost every country it does business in; for cross-cultural interactions the two researchers of the two countries discuss. Tom prefers to look at macroforces and worldviews, which tie us together. Marketing insights, on the other hand, change more rapidly: category, consumer, customers, channel, shopper, brand and product. 90% of Coca Cola researchers work in this area.</P> <P>Coca-Cola was invented in 1886 by American pharmacist Dr. John Pemberton. The macroforces that have shaped the world since then include population growth, the spread of capitalism, increasing affluence, medical and psychological advances, technology and automation, connectivity and mobility, environment changes and globalization. Given these macrotrends, how will each affect consumers, brands, marketers, companies and our worldview?</P> <P>Consumer goods constantly have improved quality: Fordism, Taylorism, Time Motions Studies, Lemon Laws, Quality Circles, Deming, Kaizen. This has produced an abundance of low cost and high quality stuff, a tremendous difference since Dr. Pemberton's day. Tomorrow's consumers will be more likely to be in buying clubs; iPhone apps will change the purchase experience as individuals scan product barcodes for price and product comparisons. The Walmart Sustainability Product Index measures energy, packaging and environmental impact; consumers want to buy better products for the environment but they don't know how yet. "We have fewer people with bigger houses who have to rent apartments to put their stuff in!" Carrotmob is the opposite of a boycott.</P> <P>For the changing role of brand, yesterday the residual value of brands was quality and safety; today the dominant value of brands is identity construction for consumers; the emerging role is of societal control. You buy a pair of shoes from Toms he gives a pair of shoes to someone who needs one. Toyota sells the Prius as a car that improve the environment for everybody. Coca-Cola's ads continue to talk about spreading happiness and joy. Coca Cola is launching many initiatives to improve society, such as supporting the WWF polar bear fund.</P> <P>Macroforces caused changes in manufacturing, consumers and brands. How should marketers, and market researchers, change? Market researcher or... Storyteller? Creative agent? Steward of the planet? Cultural leader? Emotion provider? Aesthetic designer? </P> <P>Tom Laforge is clearly all of the above!</P> </article> <article> <h1>Social Media as One Medium to Consider in Media Planning</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 16:18:00 GMT</p> <P>At the AMA MRC conference, Mike Hess, Carat.com's EVP of Research, Marketing Science & Consumer Insights, shared data from the Never Ending Friending study and covered social media as a medium and its implications on research.</P> <P><STRONG>Current Climate: Economic and Media </STRONG>- Economy has dropped from a concern of 65% of Americans in March to 40% in August, though it remains the number-one issue, with health care in second place. Despite the recession, households are not downgrading from cable/satellite to broadcast.</P> <P><STRONG>Media Touchpoints</STRONG> - Media touchpoints have risen from six types of media in the 1980s to 26-36 today (e.g., radio, TV, magazine, email marketing, social networks, keyword search). The "first screen" is the TV (live, DVR, video games), the second screen is the Web, the third screen is the mobile device, and the fourth screen is all others (e.g., kiosks). TV remains the 800-pound gorilla of media, despite declining hours among the young. People spend 8.5 hours per day on media: 65+ spend that at least 10 minutes a day on five media touchpoints; 18-24 year olds spend at least 10 minutes on 10 media touchpoints.</P> <P><STRONG>Social Networks</STRONG> - From broadcasters to "mycasters": social networking has gone mainstream, as people choose who to receive information from. In the U.S., 72% of online users participate in social networking; as a result, 56.3% of ad agencies realistically intend to purchase ads on social networking. After declining from $1.18B to $1.14B annual spending in 2009, social-network spending will resume growing next year. Social media is used for self-expression, discovery and interaction. Social networking sites are the preferred free-time activity, cited by 17% of respondents. As a result, social networking is crowding out traditional behavior; social network users watch less TV and read fewer magazines and newspapers. Social networks providing new opportunities for research: instead of doing a focus group, Mike "scraped" online communities for content about tuna, finding extensive discussions on diet/weight management websites (as well as online communities as far afield as a Star Trek fan site).</P> <P><STRONG>Navigating the Media Landscape (BOE)</STRONG> - Media can be bought, owned or earned: bought media is traditional advertising, owned media is packaging and your web site, and earned media is word of mouth, viral sharing, social chatter, press and organic search. Earned media is when your brand is talked about by consumers, when your brand listens to and feeds conversations.</P> <P><STRONG>Maintaining Neutrality in Planning: Tools/Cases</STRONG> - To help an organization improve communication to the Hispanic market segment, Carat conducted a psychographic segmentation to come up with two Hispanic consumer segments in this marketplace: a "Healthy & Financially Fit" segment (also digitally savvy) and a "Sedentary Indulgers" segment. This segmentation was then applied to media demographics to select the best media to target.</P> <P>Given the profusion of media touchpoints, the next decade in advertising will be as much an era of testing as modeling.</P> </article> <article> <h1>You Can't Be Brilliant Alone: How to Achieve Influence Without Authority through Effective Collaboration</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 13:05:00 GMT</p> <P>At today's AMA MRC, Chris Frank, vice president of B2B research in Global Marketplace Insights for American Express, discussed how market researchers can collaborate with internal and external clients. Chris has worked client side and supplier side, bought research and designed research, even moderated focus groups. Chris provided seven principles on how to collaborate:</P> <OL> <LI><STRONG>Be clear on the Essential Question</STRONG>. What is the high-order, indispensible business question for each business unit to drive their business? Don't move forward on a plan until you understand this question. Then, when you present the results, emphasize the essential question and how the results relate.</LI> <LI><STRONG>Lead a hypothesis flip-chart session</STRONG>. Document key hypotheses for the essential questions to find out preconceptions of the sponsors of the research. In the report, summarize the hypotheses, whether they are true or false, and how you came to that answer.</LI> <LI><STRONG>Practice smoke-jumping</STRONG>. "A smoke jumper is a firefighter that parachutes in to combat wildfires." In research, the smoke jumper is rapidly deployed to investigate smoke and understand hot spots from ongoing research, to the point of commissioning a second study even while the first is still in the field. The smoke jumper prevents the fire drill!</LI> <LI><STRONG>Reveal the surprises</STRONG>. What surprised you in the results? Don't lead with the data, but lead with the surprises: summarize that 400-page research tome in one page.</LI> <LI><STRONG>Steer clear of the squiggly-line syndrome</STRONG>. "What?" Ignore the trending and the minor shifts in metrics over time; don't get caught up in the morass. Ask "So What?" What are the ramifications of the data? Then ask "Now what?" What do we do as a result of this?</LI> <LI><STRONG>Avoid the "Alice in Wonderland" Meeting</STRONG>. Don't do research for the sake of research if the results will be ignored; reframe the research to answer the Essential Question.</LI> <LI><STRONG>Make your bottom-line your top line</STRONG>. Put the key findings at the front of the research. Organize for clarity; data supports the story. "Research is like a lighthouse; we shine a bright light on the five big rocks."</LI></OL> <P>As Chris said up front, "Syndication is 90% of a researcher's jobs. We are educators as researchers. We have to tell the story again and again and again."</P> </article> <article> <h1>Deviant Leadership for Researchers</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 12:13:00 GMT</p> <P>At the AMA Marketing Research Conference, Greg Reid, CMO, YRC Worldwide, outlined eight principles for a different kind of leadership: deviant leadership.</P> <P>A leader is anyone who can influence at least one person "to passionately embrace a new idea, follow a new direction or chart a new course". Leaders do not need to be managers with direct reports.</P> <P>Corporate culture works to eliminate deviant employees, to discourage deviant ideas and even to punish deviant behaviors and attitudes. "And of course, as a result, most large companies lose the opportunity to discover the future and get there first," write Ryan Matthews and Watts Wacker in <EM>The Deviants Advantage</EM>. Deviant leaders have to struggle against that culture, since such leaders are "the source of all innovation, new ideas, services, personalities and ultimately new markets."</P> <P>The 8 principles of change for researchers to become deviant leaders:</P> <OL> <LI><STRONG>Plan</STRONG>: Know the company's strategy.</LI> <LI><STRONG>Position [within the organization] doesn't matter</STRONG>. Employees need to consider how they can create change, how they can fulfill their responsibility to their employer to help it succeed.</LI> <LI><STRONG>Eliminate project mentality</STRONG>. Embrace the broader picture of what you are doing not the tactics of a project.</LI> <LI><STRONG>Profit: seek ways to improve the bottom line</STRONG>. Are you in the research business or your organization's business?</LI> <LI><STRONG>Accelerate the Pace of change</STRONG>. Skate to wear the puck is going, don't go where the ball has been. Deviant leaders discover the future and get there first.</LI> <LI><STRONG>Don't succumb to the Pressure</STRONG>. Nothing produces more pressure than an office environment; have meetings outside the office. Exercise, pursue outside interests.</LI> <LI><STRONG>Develop partnerships</STRONG>. Build relationships with resources.</LI> <LI><STRONG>Perfection: Forget it</STRONG>. We're not trying to do more with less; we're trying to do lots more with lots less. Quantify the risks of not having perfect data, but go with the data you have.</LI></OL> </article> <article> <h1>Envisioning the Future at Ford</h1> <p>Jeffrey Henning — Mon, 05 Oct 2009 11:10:00 GMT</p> <P>At this morning's opening of the AMA MRC conference, Sheryl Connelly, with the Global Consumer Trends and Futuring for Ford Motor Company, discussed her role within Ford. Her job is to understand the world. She describes herself as the only person in the company who doesn't concentrate on cars: instead, she looks at demographics, consumer trends and scenario planning. Ford follows a process to provide a broad view and diverse opinions. Ford tracks over 200 trends in a trend database and then rolls that up into ten major tends. She then educates the rest of the company on those trends.</P> <P>Ford's top trends:</P> <UL> <LI><STRONG>Aging population</STRONG>. The population is getting older, driven by: medical advances, active lifestyle, delayed marriage, delayed parenthood and declining fertility. What does the mean for Ford customers? Do you give up your car keys at 80 if you think you will live to be 102? This is significant in some markets (i.e., Japan) and not present in other markets (i.e., Russia).</LI> <LI><STRONG>Changing physiology</STRONG>: Short and long term physical changes that accompany aging, affluence and urbanization. Important among these is the increase in obesity, especially childhood obesity: even infant obesity is up 70% in 20 years. </LI> <LI><STRONG>Consumer is King</STRONG>. Consumers have unlimited choice and options, making them more savvy and more demanding, driven by: globalization, market fragmentation, on demand, online auctions and word of mouth.</LI> <LI><STRONG>Rising Power of Women</STRONG>. Women's power growing around the globe as social, political and economic status rises. Key drivers: improved education, career opportunities, financial independence, and delayed marriage and parenthood. Women control 85% of household financial decisions.</LI> <LI><STRONG>Ethical Consumption</STRONG>. Increasing concern over health, society and environment has consumers integrating ethical and religious beliefs into the purchase process. Key drivers: power of the Internet, rising trust in NGOs, butterfly effect (local scale, global impact), citizen groups and shifting accountability.</LI> <LI><STRONG>Crisis of Confidence</STRONG>. Threats to financial security, health and personal safety has consumers feeling vulnerable. Examples of firms reacting to this: BofA's Keep the Change program has brought in 2M new customers (6M accounts); Hyundai will make your car payments for three months if you lose your job (and now Ford will as well).</LI> <LI><STRONG>Careful Consumption</STRONG>. The consumer "balances practicality with passion" and exercises self-discipline, thoroughness and deliberation. Key drivers include the credit crunch and end of the era of excess.</LI> <LI><STRONG>Safety & Security</STRONG>. Consumers seeking out reassurance of personal safety, security, health and wellness. Key drivers: new threats, mistrust in business and government, media, technological advances, demand for precision and perfection.</LI> <LI><STRONG>Information Addiction</STRONG>. Consumers have become reliant on access to real-time information, giving them greater control, power and success. Key drivers: knowledge as status, infuentials, greater scrutiny, time poverty, just in time lifestyle.</LI> <LI><STRONG>Information Overload</STRONG>. Too much information can be even more problematic than too little information. Key drivers: flogs and blogola, astroturfing (fake grassroots campaigns), Lonelygirl15, advertorials.</LI></UL> <P>So what's next? No one can predict the future. "I want the organization to never be surprised in our brightest dreams or our darkest nightmares." Ford relies heavily on scenario planning to make sure that it is contemplating the actions required by these major trends.</P> </article> <article> <h1>Questions about Panels & Online Communities</h1> <p>Jeffrey Henning — Sun, 04 Oct 2009 23:25:00 GMT</p> <P>Today at the AMA MRC (Marketing Research Conference), I presented a preconference workshop on proprietary panels and online communities. Interestingly, 80% of the attendees were looking at building or expanding proprietary panels and communities of their own customers and prospects. </P> <P>Here are some of the questions that came up in the session itself and one-on-one with attendees (with links to some answers):</P> <UL class=unIndentedList> <LI>How does <A href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" mce_href="http://blog.vovici.com/blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">a panel differ from an online community</A>?</LI> <LI>How do I build a <A href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers" mce_href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers">representative email list of customers</A>?</LI> <LI>If we have a boring product, <A href="http://blog.vovici.com/blog/bid/17886/Communitizing-the-Commodity-Product" mce_href="http://blog.vovici.com/blog/bid/17886/Communitizing-the-Commodity-Product">how can we get people to join an online community</A>?</LI> <LI>As you make the case for your company to build its own community of customers, how do you calculate an <A href="http://blog.vovici.com/blog/bid/18045/Measuring-ROI-for-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/18045/Measuring-ROI-for-Online-Communities">ROI for online communities</A>?</LI> <LI>How prominent should the <A href="http://blog.vovici.com/blog/bid/18020/Microsites-or-Integrated-Sites-for-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/18020/Microsites-or-Integrated-Sites-for-Online-Communities">online community be from your web site</A>?</LI> <LI>Who <A href="http://blog.vovici.com/blog/bid/18019/Idea-Ownership-in-Research-Communities" mce_href="http://blog.vovici.com/blog/bid/18019/Idea-Ownership-in-Research-Communities">owns the ideas submitted to an online community</A>?</LI> <LI>How do I <A href="http://blog.vovici.com/blog/bid/17885/Dealing-with-Detractors-in-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/17885/Dealing-with-Detractors-in-Online-Communities">throw members out of the community</A>, if necessary?</LI> <LI>How do we make sure we don't make <A href="http://blog.vovici.com/blog/bid/18160/Qualitative-and-Quantitative-Research-Fusion-Powered-by-Online-Communities" mce_href="http://blog.vovici.com/blog/bid/18160/Qualitative-and-Quantitative-Research-Fusion-Powered-by-Online-Communities">bad decisions from our community research</A>?</LI></UL> <P>Since it was a three-and-a-half-hour session, I did something new as far as event-evaluation research goes: after the first hour, I gave each attendee a four-question survey, asking what I should change about how I was presenting, what I shouldn't change, how satisfied they were and how enjoyable the session was so far. I took to heart the advice of Jim Davies, principal analyst with Gartner; he had said in his <A href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying" mce_href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying">EFM presentation at the Gartner CRM Summit</A> a few weeks ago that too few organizations gather feedback during the experience itself. Based on the survey results, I made the rest of the session more interactive and covered some topics in more detail than I had originally planned.</P> <P>I loved the additional interaction, which exposed all of us to more points of view. I walked away with six ideas for future blog posts to discuss aspects of panel and online community management that I have neglected so far. I also learnt a widespread fear: that management will look to an online community for all its answers, leaping to conclusions from community research without validating that those insights represent widespread sentiments. We agreed that as market researchers we need to continue to educate our coworkers and management teams about the differences between <A href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research" mce_href="http://blog.vovici.com/blog/bid/17986/Qualitative-vs-Quantitative-Research">quantitative and qualitative research</A>, so that organizations make the right decisions. Using panels and online communities as complements to one another is a key way to accomplish this.</P> </article> <article> <h1>Convenience Sampling: Everyone's a Star on YouTube</h1> <p>Jeffrey Henning — Tue, 29 Sep 2009 09:01:00 GMT</p> <P><EM>Convenience sampling</EM>, also called <EM>opportunity sampling</EM>, involves conducting surveys with anyone at hand. The results of a study conducted using a convenience sample cannot be extrapolated to a target population, because the two key aspects of a probability or <A href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sample</A> were not met: 1) there was not an equal chance that anyone in the target population could participate, and 2) respondents were not externally selected to participate.</P> <P>Most web polls use convenience samples, and YouTube recently published a great blog post that illustrates one of the perils of such sampling: <A href="http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html" target=_new mce_href="http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html">Five Stars Dominate Ratings</A>. YouTube users only bother to rate a video (only registered users may vote) if they think it is worth five stars, as this classic <A href="http://buildingreputation.com/writings/2009/08/ratings_bias_effects.html" target=_new mce_href="http://buildingreputation.com/writings/2009/08/ratings_bias_effects.html">J-curve</A> from their blog illustrates:</P> <P></P> <P>And I thought <A href="http://blog.vovici.com/blog/bid/18058/How-Using-a-5-Point-Scale-Costs-Netflix-Sales" mce_href="http://blog.vovici.com/blog/bid/18058/How-Using-a-5-Point-Scale-Costs-Netflix-Sales">Netflix had problems with <EM>its</EM> 5-point rating scale</A>!</P> <P>To get an idea for how frequently YouTube videos are rated, I picked 15 YouTube videos out by convenience and found that ratings averaged 0.4% of views and ranged from 0% to 2.7% of views depending on the video (for instance, one video with 7,657 views had no ratings and one video with 22,226 views had 608 ratings, or 2.7%). Yes, it's ironic that I'm using a convenience sample in a blog post about the peril of convenience samples: as a result, this analysis may or may not be representative of YouTube videos in general; if it is, then it shows that when respondents select themselves (rather than being externally selected) they rarely participate and do so only to register an extreme, positive opinion.</P> <P>In comments on the YouTube blog and <A href="http://www.techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-realization-its-ratings-are-useless/" target=_new mce_href="http://www.techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-realization-its-ratings-are-useless/">a related TechCrunch post</A>, most of the discussions suggested other scales or metrics to use. I didn't see anyone suggest switching the sampling technique. If YouTube's ratings were up to me, as an experiment I would move it from a convenience sample to a random sample:</P> <OL> <LI>I would disable your ability to rate 95% of videos. If you viewed a video, there would only be a 5% chance you would be able to rate it.</LI> <LI>If you were one of the 5% of users who could see the rating scale for a particular video, I would flash an invitation next to the scale: "You have been randomly chosen to rate this video. How do you rate it?"</LI> <LI>I would store the status of your invitation; if you weren't selected to rate it, you would never be selected to rate it; if you were selected, but hadn't rated it, you'd be given an opportunity next time.</LI> <LI>On your history page, I would include a section for "Videos to Rate", showing those videos that you could rate but haven't yet.</LI></OL> <P>Such a system would now have external selection and would provide an equal probability than any user was selected to participate. With invitations to take the survey reduced to 5% of the time, the novelty value is greater and respondents will now be more likely to provide a rating. I believe this system would produce a wider dispersion of ratings than the current system.</P> <P>If the experiment worked, then I would discard the existing ratings, as they seem to add little or no value.</P> <P>Now, how many stars would you give my idea?!</P> </article> <article> <h1>Measuring Affective & Calculative Commitment</h1> <p>Jeffrey Henning — Mon, 28 Sep 2009 14:48:00 GMT</p> <P>In my recent post, <A href="http://blog.vovici.com/blog/bid/21337/Your-Half-Human-Half-Vulcan-Customers" mce_href="http://blog.vovici.com/blog/bid/21337/Your-Half-Human-Half-Vulcan-Customers">Your Half-Human, Half-Vulcan Customers</A>, I discussed <EM>affective commitment</EM> (the emotional pleasure your customer takes in doing business with you: the Human half) and <EM>calculative commitment</EM> (the cold-blooded evaluation that governs why your customer has a business relationship with you: the Vulcan half). After reading that post, the logical question for you to ask is how to measure your customers' affective and calculative commitment.</P> <BLOCKQUOTE> <P>My past review of the literature hadn't shown much in the way of consensus, though I did find two papers using the following measures:</P> <P>On a scale of 1 to 10, where 1 means "strongly disagree" and 10 means "strongly agree", please indicate your level of agreement or disagreement with each of the following about your telecommunications provider.</P> <P><EM>[Affective Commitment]</EM></P> <P>1. I take pleasure in being a customer of the company.<BR>2. The company is the operator that takes the best care of their customers.<BR>3. There is a presence of reciprocity in my relationship with the company.<BR>4. I have feelings of trust toward the company.</P> <P><EM>[Calculative Commitment]</EM></P> <P>5. It pays off economically to be a customer of the company.<BR>6. I would suffer economically if the relationship were broken.<BR>7. The company has location advantages versus other companies.<BR><BR><EM>Source: </EM>Anders Gustafsson, Michael D. Johnson, & Inger Roos, "<A href="http://www.atypon-link.com/AMA/doi/abs/10.1509/jmkg.2005.69.4.210" target=_new mce_href="http://www.atypon-link.com/AMA/doi/abs/10.1509/jmkg.2005.69.4.210">The Effects of Customer Satisfaction, Relationship Commitment Dimensions, and Triggers on Customer Retention</A>".</P></BLOCKQUOTE> <P>Sadly, there's a lot to dislike about this measurement, as it doesn't reflect modern <A href="http://blog.vovici.com/blog/bid/21102/Rating-Scale-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/21102/Rating-Scale-Best-Practices">rating scale best practices</A>: it uses a 10-point <A href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">bipolar scale</A> rather than a 7-point bipolar scale, it uses <A href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">numbers rather than labels</A>, it's a Likert scale and subject to <A href="http://blog.vovici.com/blog/bid/21978/Agree-Disagree-Scale-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/21978/Agree-Disagree-Scale-Best-Practices">acquiescence response bias</A>, and it has some complex items (e.g., "There is a presence of reciprocity in my relationship with the company"). </P> <P>Despite those weaknesses, I have used it with minor modification in some of my research (changing #2 to "[vendor] is the company that takes the best care of their customers" and changing #7 to "The company has advantages versus other companies"). I'm continuing to experiment: for B2B and more expensive consumer goods (e.g., appliances and cars), calculative commitment is an important predictor of loyalty (i.e., repurchase likelihood); for inexpensive consumer goods, affective commitment is more important for its impact on loyalty.</P> <P>If your customers are "Human" (driven by emotion), consider adding the affective commitment index to your research. If your customers are "Vulcan" (driven by logic), consider adding the calculative commitment index. And if your customers are half-Human, half-Vulcan, use both measures. And never let them try to pinch you on the neck!</P> </article> <article> <h1>Agree-Disagree Scale Best Practices</h1> <p>Jeffrey Henning — Mon, 28 Sep 2009 10:28:00 GMT</p> <P>Nothing is easier for the survey author then to dash off a questionnaire asking respondents to rate a bunch of items on an agree-disagree scale (also known as the Likert scale). For instance:</P> <BLOCKQUOTE> <P>For each of the following statements, please indicate if you: Completely disagree, Disagree, Somewhat disagree, Neither agree nor disagree, Somewhat agree, Agree, Completely agree.</P> <UL class=unIndentedList> <LI>My overall job satisfaction is very high.</LI> <LI>The issue of excessive executive compensation is very important to me personally.</LI> <LI>I rarely feel discouraged with my work.</LI> <LI>I am very likely to seek employment elsewhere in the next six months.</LI></UL></BLOCKQUOTE> <P>You can easily add many other statements to this list for respondents to rate. In fact, I have seen questionnaires with 80 to 100 items, all to be rated on this agreement scale. </P> <P>Unfortunately, in such batteries of questions, respondents exaggerate their actual agreement. Over 100 studies now have demonstrated <EM>acquiescence response bias</EM>, as some respondents will agree to almost any assertion. Saris, Krosnick and Shaeffer identify three reasons for this, in their paper "<A href="http://communication.stanford.edu/faculty/Krosnick/docs/Saris%20Paper%20-%20New%202005.pdf" target=_new mce_href="http://communication.stanford.edu/faculty/Krosnick/docs/Saris%20Paper%20-%20New%202005.pdf">Comparing Questions with Agree/Disagree Response Options to Questions with Construct-Specific Response Options</A>":</P> <OL> <LI>Some respondents are simply agreeable, and indicate agreement out of politeness.</LI> <LI>Other respondents expect that the researchers agree with the listed items and defer to their judgment.</LI> <LI>Most respondents engage in survey satisficing and find that agreeing takes less effort than carefully weighing each optional level of disagreement and agreement.</LI></OL> <P>The standard solution to acquiescence response bias has been to have a balanced battery of items, where each item has a negated counterpart somewhere else in the questionnaire. For instance, averaging the agreement level to "I am generally a satisfied employee" with "I am not generally a satisfied employee" was thought, in theory, to produce a rating that factored out the acquiescence response bias. Saris, Krosnick and Shaeffer put that to the test and found three ways it leads to lower data quality: having to answer twice as many questions, which leads to satisficing; processing negations, which are more complex cognitively; and placing respondents who acquiesce in the middle of the scale.</P> <P>The solution with the highest data quality does lead to more work for the survey author. Each question needs to be asked with what Saris <EM>et al </EM>call "construct-specific response options": in other words, a rating scale that can be used to measure the item in question. Applying this recommendation to the four questions above yields:</P> <UL> <LI>How would you rate your job satisfaction overall? Not at all satisfied, Slightly satisfied, Moderately satisfied, Very satisfied, Completely satisfied?</LI> <LI>How important is the issue of excessive executive compensation to you personally? Not at all important, Slightly important, Moderately important, Very important, Extremely important?</LI> <LI>How often do you feel discouraged with your work? Never, Rarely, Sometimes, Often, Always?</LI> <LI>How likely are you to seek employment elsewhere in the next six months? Not at all likely, Slightly likely, Moderately likely, Very likely, Completely likely?</LI></UL> <P>At first glance, this looks like more work for the respondent, who must read the choice list for each question. While there is more to read, there is less to think about. For the statement, "I am generally a satisfied employee", respondents might have come up with four reasons to disagree:</P> <OL> <LI>They are generally dissatisfied.</LI> <LI>They are neither dissatisfied nor satisfied.</LI> <LI>They are often satisfied, but not often enough to classify it as "generally".</LI> <LI>They are always satisfied, which is more often than "generally".</LI></OL> <P>The agreement/disagreement scale gives the respondent too much to think about for each item and too many potential reasons to disagree. If this particular question were reworded to use a satisfaction scale, respondents are only rating their satisfaction. Less work mentally, and a more accurate answer.<BR>The best practices for agree-disagree scales are therefore simple:</P> <UL> <LI>Avoid them if at all possible, rephrasing each question to use a <A href="http://blog.vovici.com/blog/bid/18261/Common-Rating-Scales-to-Use-when-Writing-Questions" mce_href="http://blog.vovici.com/blog/bid/18261/Common-Rating-Scales-to-Use-when-Writing-Questions">common rating scale</A> where possible, otherwise using a <A href="http://blog.vovici.com/blog/bid/18620/Custom-Scale-Development" mce_href="http://blog.vovici.com/blog/bid/18620/Custom-Scale-Development">custom rating scale</A>.</LI> <LI>When the executives or customer sponsoring the research dictate that you must use an agreement scale, use the seven-item <A href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">bipolar</A> scale: Completely disagree, Disagree, Somewhat disagree, Neither agree nor disagree, Somewhat agree, Agree, Completely agree.</LI></UL> <P>The Likert scale has a long and illustrious history, having been invented in 1932. Sadly, it is now as obsolete as the cathode-ray tube (which RCA first demonstrated could be used as to receive TV transmissions in 1932). Construct-specific response options are the HDTV of the survey world.</P> </article> <article> <h1>How to Maximize the Number of Open-Ended Responses</h1> <p>Jeffrey Henning — Thu, 24 Sep 2009 14:41:00 GMT</p> <P>My standard advice about <A href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically" mce_href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically">structuring questionnaires</A> is, after the <A href="http://blog.vovici.com/blog/bid/18203/Screening-Questions-May-Indicate-Need-for-Better-Survey-Profiles" mce_href="http://blog.vovici.com/blog/bid/18203/Screening-Questions-May-Indicate-Need-for-Better-Survey-Profiles">screener</A>, to begin the questionnaire proper with <A href="http://blog.vovici.com/blog/bid/18095/Open-Ended-Questions" mce_href="http://blog.vovici.com/blog/bid/18095/Open-Ended-Questions">open-ended questions</A>. Since respondents start fresh and then tire over the course of the survey (moving from giving the optimal answer to <A href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" mce_href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">satisficing</A>), you are going to get more feedback when you lead with these open-ended questions. Further, starting with open-ends removes the chance for earlier <A href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" mce_href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">closed-ended questions</A> to bias the answers to open-ends, as they raise other topics to consideration than what might have been top of mind for respondents. </P> <P>Many of my customers push back on this advice, however, because they are concerned that leading with verbatim questions will increase the survey abandonment rate. They prefer to keep their open-ended questions near the end of the survey.</P> <P>I recently had the opportunity to put my advice to the test. I did two versions of a 5-page, 9-question questionnaire. Survey A had the open-ends as the seventh and eighth questions. Survey B moved the same questions to the first and second position; everything else stayed the same. Respondents were not required to answer the open-ended questions; clicking the Next button would simply take them to the next question. </P> <P>The test was fielded in August. Survey A had 70 responses from 290 invites (24% response rate), and Survey B had 79 responses from 379 invites (21% response rate) [had the test been the primary purpose of the survey, I would have kept these distributions equal].</P> <P>As expected, the abandonment rate did increase when leading with open-ended questions, climbing from 1% for Survey A to 6% for Survey B. Yet that trade-off was worth it: 92% of respondents answered at least one of the open-ended questions for Survey B, compared to only 60% of the respondents of Survey A. So in exchange for a modest drop-off in overall response, we gathered fully 50% more open-ended comments!</P> <P>Surprisingly to me, though, the abandonment rate for Survey B did not occur at the beginning of the survey-I fully expected respondents to exit without answering the open-ends, but no respondent did so. Instead what happened was that respondents who answered at least one of the open-ends were the respondents who were more likely to grow tired of the survey and abandon it. In fact, every respondent who skipped answering the two open-ended questions answered all seven subsequent closed-ended questions.</P> <P></P> <P>I had also hypothesized that you would get longer answers when leading with open-ended questions. In this, I was disappointed: the average length of an answer was 13 words in both surveys.</P> <P>As a result of this test, I will continue to advise that open-ended questions are more productive when placed at the beginning of a survey rather than near its end.</P> </article> <article> <h1>Gartner CRM Summit Conference Recap - 2009</h1> <p>Jeffrey Henning — Wed, 23 Sep 2009 22:07:00 GMT</p> <P><A href="http://www.gaylordhotels.com/gaylord-national/" rel=nofollow target=_new mce_href="http://www.gaylordhotels.com/gaylord-national/"></A>Summary of my coverage of the 2009 North American convention:</P> <UL> <LI><A href="http://blog.vovici.com/blog/bid/21579/CRM-Trichotomy-Operational-CRM-Analytical-CRM-Social-CRM" mce_href="http://blog.vovici.com/blog/bid/21579/CRM-Trichotomy-Operational-CRM-Analytical-CRM-Social-CRM">CRM Trichotomy: Operational CRM, Analytical CRM, Social CRM</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying" mce_href="http://blog.vovici.com/blog/bid/21594/EFM-The-Who-When-Why-Where-What-of-Surveying">EFM: The Who, When, Why, Where & What of Surveying</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/21646/Unlocking-Key-Performance-Indicators">Unlocking Key Performance Indicators</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/21676/Innovation-Your-New-Core-Competency">Innovation: Your New Core Competency</A></LI> <LI><A href="http://blog.vovici.com/blog/bid/21734/Meta-Analysis-Who-Analyzes-the-Analysts">Meta-Analysis: Who Analyzes the Analysts</A>?</LI> <LI><A href="http://blog.vovici.com/blog/bid/21872/Blog-Analysis-as-Market-Research">Blog Analysis as Market Research</A></LI></UL> <P>And my <A href="http://blog.vovici.com/blog/bid/17900/Gartner-CRM-Summit-Conference-Recap">Gartner CRM Summit Conference 2008 Recap</A>.</P> </article> <article> <h1>Blog Analysis as Market Research</h1> <p>Jeffrey Henning — Wed, 23 Sep 2009 16:44:00 GMT</p> <P><STRONG>In the Dock: </STRONG>Blogs, charged with the crime of wasting market-research resources.</P> <P><STRONG>The Prosecution</STRONG>: Gartner research director Gareth Herschel</P> <BLOCKQUOTE> <P>In one of last week's keynotes at the <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner CRM conference</A>, <A href="http://blog.vovici.com/blog/bid/21646/Unlocking-Key-Performance-Indicators" mce_href="http://blog.vovici.com/blog/bid/21646/Unlocking-Key-Performance-Indicators">Analytics-to-Action: Key Analyses for Customer-Centric Decisions</A>, Gareth Herschel dismissed the value of blogs as a source of MR insights. He said that Gartner clients had seen marginal-to-no ROI from their investments in blog content analysis. "It's eye candy: management likes to see it." Similarly, Gartner clients have seen no ROI on connecting customers to their social networks. In terms of concrete ROI, Gartner advises its customers to instead learn the language of the <A href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies" mce_href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies">Voice of the Customer</A> by asking more open-ended questions in surveys and then text mining the responses.</P></BLOCKQUOTE> <P><STRONG>The Defense</STRONG>: Janet Eden-Harris, VP of Web Intelligence for J.D. Power</P> <BLOCKQUOTE> <P>In an <A href="http://sherpablog.marketingsherpa.com/research-and-measurement/market-research-via-social-media/" target=_new mce_href="http://sherpablog.marketingsherpa.com/research-and-measurement/market-research-via-social-media/">interview by Adam Sultan</A> of Marketing Sherpa, Janet identified the following applications of blog content analysis for market research:</P> <OL> <LI><STRONG><EM>Brand monitoring</EM></STRONG>, especially of competitors' brands</LI> <LI><STRONG><EM>Trend analysis</EM></STRONG> - tracking studies</LI> <LI><STRONG><EM>Customer profiling</EM></STRONG> - with analysis across the posts of a single author providing a richer look at customers as individuals</LI> <LI><STRONG><EM>Unmet needs identification</EM></STRONG> - reading blogs to learn product and service frustrations, wishes and ideas.</LI></OL></BLOCKQUOTE> <P><STRONG>The jury:</STRONG> Out </P> <P>I will recuse myself, since clearly I like Gartner's answer that surveys are more valuable than blogs for market research.</P> <P>What would you say? Would you be a witness for the defense or for the prosecution?</P> </article> <article> <h1>Twitter-Triggered Surveys</h1> <p>Jeffrey Henning — Tue, 22 Sep 2009 14:46:00 GMT</p> <P>One of the many ways that people use Twitter is to share their good and bad experiences with specific brands. As an example, we analyzed 150 recent tweets about McDonald's and identified four complaints:</P> <P></P> <P>While feedback is great, even negative feedback, a tweet-being limited to 140 characters-doesn't provide actionable context. No doubt a McDonald's executive reading these tweets would have some additional questions for customers about their in-store experience: <UL class=unIndentedList> <LI>Where was the McDonald's that this happened to you?</LI> <LI>When did this happen?</LI> <LI>Did you alert a McDonald's employee at the restaurant to the situation?</LI> <LI>Is there anything else you would like to tell us?</LI></UL> <P>In fact, this is a great opportunity for a transactional survey. A McDonald's social media representative could tweet a reply to each person, along the lines of this invented example: "@VenessaLeal, so sorry to hear about your ice coffee. We'd love additional feedback, to serve you better next time. <A href="http://bit.ly/" mce_href="http://bit.ly/">http://bit.ly/</A>..."</P> <P>This provides a way to transform unstructured feedback into structured feedback, which would be more manageable. Such a transactional survey would provide McDonald's the additional information they need to take action: they would be able to route the feedback to the appropriate store, where management could remind McDonald's employees of the best practices that would resolve the identified issue. McDonald's could incorporate this feedback flow into their other ongoing research initiatives (e.g., mystery shopping and customer satisfaction studies) to identify ongoing opportunities for improving training and service.</P> <P>I wouldn't trust automation to identify which tweets are negative feedback, since sentiment analysis is too inaccurate. I think for now a human needs to read the tweets and respond; someone in your marketing and customer-service departments should be reading tweets that mention your brands already. Nor would I extrapolate from the results of such surveys; they are not representative of the wider customer base that you serve, the vast majority of whom are not on Twitter.<BR></P> <P>As of this writing (September 22, 2009), the <A href="http://twitter.com/McDonalds" target=_new mce_href="http://twitter.com/McDonalds">@McDonalds</A> Twitter account has issued just 1 tweet: "We'll be joining you very soon. Stay tuned." It will be interesting to see in what ways McDonald's uses its Twitter presence, especially for collecting and managing feedback.</P> <P>If you are aware of any organizations that use Twitter to initiate transactional surveys, please share in the comments section below. Thanks!<BR></P> </article> <article> <h1>CE Best Practices Require Strategic Commitment</h1> <p>Brian Koma — Fri, 18 Sep 2009 11:30:00 GMT</p> <div>According to the Vovici <a mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx " href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx ">CE IQ study</a>, organizations that have the most loyal customers have a formal customer experience strategy and tactical plans to execute against it. In fact, out of 24 best practices studied, having a formal customer-experience strategy in place had the highest positive correlation (0.59) to the customer loyalty index.</div><div> </div><div>But it’s not enough to just have a strategy: you must also have planned programs and tactics in place to achieve the strategy through practical, measurable changes (0.44 correlation). Further, to be effective, a strategy and plan must have the support of senior leadership and be communicated throughout the organization (0.42 correlation). </div><div> </div><div>While three quarters of respondents to the study say that they have a CEM strategy, fewer than half have formal CEM programs in place along with compensation programs that align with CEM objectives. Many organizations appear to be paying lip service to the importance of CEM, with only 44% of respondents reporting that senior executives’ compensation is tied to the achievement of CEM goals.</div><div> </div><div> </div><div>Having an executive sponsor and the buy-in of the senior leadership team are indispensable components of success, but they may take time to achieve. Ultimately, this commitment to a CEM strategy and plan must translate into the organization in a way that allows the brand promise to be fulfilled (0.37 correlation).</div><div> </div><div style="text-align: center;"> </div><div> </div><div>Need help developing a customer-experience strategy for your brand? Please watch our <a mce_href="http://www.vovici.com/about/webinars.aspx" href="http://www.vovici.com/about/webinars.aspx">research webinar</a>, “Countdown to a CEM Program”. Remember, out of 24 best practices, putting a formal CE strategy in place is the single most important practice.</div> </article> <article> <h1>Meta-Analysis: Who Analyzes the Analysts?</h1> <p>Jeffrey Henning — Fri, 18 Sep 2009 06:59:00 GMT</p> <DIV>At this year’s <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner NA CRM conference</A>, Gareth Herschel presented “Making Gut Decisions More Intelligent”. One of his key points was that most organizations fail to study and improve their decision-making process. “We have to create a culture of auditing the decisions,” he said: not to punish bad decision makers, but to identify ways to make better decisions in the future.</DIV> <DIV> </DIV> <DIV>As part of this decision audit, Gareth advocates logging for each decision which best practices were followed. For instance:</DIV> <DIV> <UL> <LI>Was the forecast of market demand accurate? </LI> <LI>Was there an opportunity for dissenting voices to be heard? </LI> <LI>Did stakeholders share information? </LI></UL></DIV> <DIV>Then, once enough time has passed to determine whether a decision was a success or failure, plot each best practice on a quadrant. </DIV> <DIV> </DIV> <DIV>This quadrant analysis is adapted from "Flaws in Strategic Decision Making: McKinsey Global Survey Results": For the vertical axis, use the percentage of decisions for which the best practice was followed, with 0% at the bottom and 100% at the top; for the horizontal axis, use the difference between the presence of such practices in successful decisions subtracting out their presence in unsuccessful decisions. </DIV> <DIV> </DIV> <DIV style="TEXT-ALIGN: center"> </DIV> <DIV>The most important quadrant that emerges reveals practices that are “Common in Successes, Rare in Failures”. These are the key decision-making practices that are most important to your industry and organization.</DIV> <DIV> </DIV> <DIV>To parallel Kathy Harris’ advice of “Continually <A href="http://blog.vovici.com/blog/bid/21676/Innovation-Your-New-Core-Competency" mce_href="http://blog.vovici.com/blog/bid/21676/Innovation-Your-New-Core-Competency">innovate how you innovate</A>!”, you should continually analyze how you analyze.</DIV> <DIV><BR></DIV> </article> <article> <h1>Innovation: Your New Core Competency</h1> <p>Jeffrey Henning — Wed, 16 Sep 2009 13:27:00 GMT</p> <DIV>This morning Kathy Harris, Gartner VP & Distinguished Analyst, presented “Innovation: Your New Core Competency” at the North American <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner CRM conference</A>. Kathy began by asserting “CRM is the most fruitful innovation opportunity for your company and your customers.”</DIV> <DIV> </DIV> <DIV>Kathy identified six key drivers shaping innovation in the coming five years:</DIV> <DIV> <OL> <LI><B>Seek out innovation through consumerization and the Web</B>. This is a fertile area as past business models are disrupted. Learn from companies far outside your market space.</LI> <LI><B>Embrace customers as co-creators in innovation</B>. The future of idea generation includes crowdsourcing, idea marketplaces and innovation “jams”. Many customers want you to serve them better in the future: you need to identify these lead customers from the standpoint of idea generation, build communities and councils to “systematize VOC” and spark customer creativity. "If you don't engage your customers in innovation, someone else will," Kathy said.</LI> <LI><B>Similarly, embrace your business partners in innovation</B>. Build outside-in networks as eBay has done with 85,000 developers; co-develop, as Novartis has done on public-sponsored research; and build inside-out networks as P&G has done.</LI> <LI><B>Let social trends pull you into further innovation</B>. Changes in demographics, lifestyles, technology use and refusal all provide opportunities for reconsidering the types of products and services you provide today.</LI> <LI><B>Innovate with your IT spending</B>. In a Gartner survey of over 2000 organizations last year, 67% of IT budgets were spent on running the business, 19% on growing the business and only 14% on transforming the business. You need to set goals to innovate your IT infrastructure to reduce costs, to free up additional budget for growth and transformation.</LI> <LI><B>Don’t forget basic CRM best practices</B>. You can’t build innovative CRM capabilities on a shaky foundation.</LI></OL></DIV> <DIV>Kathy Harris' final recommendation: "Continually innovate how you innovate!" </DIV> </article> <article> <h1>Unlocking Key Performance Indicators</h1> <p>Jeffrey Henning — Tue, 15 Sep 2009 15:18:00 GMT</p> <DIV>In last night’s keynote at the <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner CRM conference</A>, <I>Analytics-to-Action: Key Analyses for Customer-Centric Decisions</I>, research director Gareth Herschel presented a wide-ranging discussion of using analysis to reach conclusions centered on the customer. A number of his points focused on selecting KPIs (Key Performance Indicators).</DIV> <DIV> </DIV> <DIV>Gareth related a conversation with a client, after he had asked her about her KPIs. Turns out she had 137 KPIs for her department. “Well, that makes for an interesting definition of K!” he said. As a practical matter, she concentrated on just 7 of the KPIs. Unfortunately, her boss had a different 7 that were important to him. Moral: you have to measure what matters.</DIV> <DIV> </DIV> <DIV>Gareth’s advice is that no manager have more than 5 to 9 key metrics. Since we figuratively talk about “keeping balls in the air”, he used the analogy of juggling: world records for juggling a few items measure the feat in hours, while world records for juggling 10 or more items measure the record in catches. Physically and figuratively, we can’t juggle too many items.</DIV> <DIV> </DIV> <DIV>Of course, if you are only going to concentrate on a few items, you’d better select them with care. Too often items are tracked because they can be, not because they should be. As Gareth said, “Marketing never saw a piece of data they didn’t want.” Worse, most KPIs are at too simplistic a level and fail to focus on LTV (Life-Time Value).</DIV> <DIV> </DIV> <DIV>Customer churn by itself is an example of a bad performance indicator: “80% of value destruction occurs from 20% of your customers; you want to fire those customers,” Gareth said. Telecommunications firms and utilities especially suffer from this. Instead of just looking at minimizing churn rate, for instance, ask “Who’s at risk of churning? Do we care about them? Why will they churn? What should we do to retain them? Would we succeed? Are they likely to be at risk of churning again?” Segment customers by LTV and current value and seek to minimize the churn rate of those with the highest current and lifetime values. <SPAN style="WHITE-SPACE: pre" class=Apple-tab-span></SPAN></DIV> <DIV> </DIV> <DIV>Remember, you can have hundreds of <I>performance </I>indicators but you have to focus on the <I>key</I> indicators to achieve customer-centered decisions.</DIV> </article> <article> <h1>Upcoming September/October Webinars & Presentations</h1> <p>Jeffrey Henning — Tue, 15 Sep 2009 10:08:00 GMT</p> <div>If you find this blog useful, then I would encourage you to attend one of my upcoming webinars or public presentations. </div><div> </div><div><b>Webinars</b></div><div><ul><li><b><i>Surveys & Social Media: Learn How and Why You Should Incorporate Twitter into Your Survey Research</i></b>, <a mce_href="https://cc.readytalk.com/cc/schedule/display.do?udc=hzt8kh6odq7o" target="_new" rel="nofollow" href="https://cc.readytalk.com/cc/schedule/display.do?udc=hzt8kh6odq7o">Sept. 22, 1 pm ET</a>. Social media sites like Twitter provide an excellent qualitative source of market research of your prospects, customers and competitors. Roderick Morris and I will review the strengths and weaknesses of social media research and discuss how to integrate such research into your traditional survey research projects. </li><li><b><i>It's Greek to Me: Best Practices for Multi-Lingual Surveys</i></b>, <a mce_href="http://www.vovici.com/webinars/20090923.aspx" target="_new" rel="nofollow" href="http://www.vovici.com/webinars/20090923.aspx">Sept. 23, 2 pm ET</a>. Conducting the same online survey research study across respondents who speak different languages poses its own special challenges. From the perils of translation to differences in cultures to browser incompatibilities, there are many traps that await the unwary researcher. Kathryn Korostoff of <a mce_href="http://www.researchrockstar.com" target="_new" href="http://www.researchrockstar.com">Research Rockstar</a> and I will give you a Rosetta Stone to help unlock the mysteries of multi-language surveys.</li><li><b><i>7 Ways to Inspire Customer Loyalty with Voice of the Customer Initiatives</i></b>, <a mce_href="http://www.vovici.com/webinars/20090929.aspx" target="_new" rel="nofollow" href="http://www.vovici.com/webinars/20090929.aspx">Sept. 29, 2 pm</a>. Listening and acting on the Voice of the Customer (VOC) is a proven practice area within Customer Experience Management programs. Brian Koma and I will discuss how Voice of the Customer research has evolved and expanded, the correlation between VOC best practices and loyalty, what steps to follow to implement VOC research and how to spin up the feedback cycle to drive continuous improvement.</li></ul><b>Presentations</b><br><ul><li><b><i>Merging Proprietary Panels and Online Communities to Lower Research Costs and Maximize a Company’s Ability to Stay Linked to its Customers</i></b>, at the <a mce_href="http://www.marketingpower.com/Calendar/Pages/MarketingResearchEvent_2009Conference.aspx" target="_new" href="http://www.marketingpower.com/Calendar/Pages/MarketingResearchEvent_2009Conference.aspx">AMA MR conference, Oct. 4, 1:30 PM PT</a>. Business online communities and proprietary panels have the power to transform customer relationships in a fundamental way, by promoting deeper engagement, mutual respect and a collaborative, co-creative environment. This 3-hour tutorial will give you an overview of proprietary panels and online communities and experience a hands-on, step-by-step approach for how to build and benefit from your own online business community.</li><li><b><i>How to Use Research 2.0 to Reduce Market Research Costs and Improve Effectiveness</i></b>, at the <a mce_href="http://www.marketingpower.com/Calendar/Pages/MarketingResearchEvent_2009Conference.aspx" target="_new" href="http://www.marketingpower.com/Calendar/Pages/MarketingResearchEvent_2009Conference.aspx">AMA MR conference, Oct. 6,4:00 PM PT</a>. Web 2.0 technologies have changed the way respondents want to interact with researchers, who must now embrace these new technologies to improve the effectiveness of their efforts and to achieve dramatic reductions in the cost of obtaining feedback. </li></ul></div><div>Do you need a speaker for a webinar or conference? I am happy to give presentations on a wide variety of topics related to survey research, online communities and feedback management. Please feel free to contact me at +1 703-481-9326 x 550 to discuss your requirements.</div> </article> <article> <h1>EFM: The Who, When, Why, Where & What of Surveying</h1> <p>Jeffrey Henning — Mon, 14 Sep 2009 17:36:00 GMT</p> <DIV>Today at the <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner CRM conference</A>, Jim Davies gave his presentation, "EFM: The Who, When, Why, Where & What of Surveying". Some highlights:</DIV> <DIV> <UL> <LI><B>What? </B>I noticed in Jim’s other presentation this morning, he used “feedback management” and “survey” rather than “EFM”: it’s still an acronym that prompts quizzical looks. Jim described enterprise feedback management systems as the feedback “hubs” of their organizations, the central repository for survey data, across departments and across channels (web, paper, phone, etc.).</LI> <LI><B>Why?</B> Without EFM systems, organizations trip all over themselves, oversurveying customers and missurveying them, because of a lack of attention to process and results. A big failure is the lack of personalization. Jim related the tale of a colleague who bought a Mini Cooper online: he was kept up to date with each stage of the manufacture and shipping of the car, to the point that “the car” emailed him e-postcards from its cruise across the Atlantic (“Playing shuffle board with the other Minis. Looking forward to meeting you!”). It was a great, personalized experience. Until the survey came! Ten pages, 80 questions, beginning by asking which car was ordered, what option package was selected, and so on. Apparently the survey researcher hadn’t played shuffleboard with the car.</LI> <LI><B>Who?</B> Jim mentioned two firms as leaders in enterprise feedback management: ConfirmIt and Vovici. He said that there were 60-70 vendors, though most had less than $2M revenue. He mentioned Allegiance as an up-and-coming vendor for its acquisition of Inquisite; he also mentioned MarketTools. I thought it was notable that, having mentioned consolidation was coming to the industry, he discussed the four firms that have made acquisitions in this market: ConfirmIt, Vovici, MarketTools and Allegiance have each made at least one acquisition (Vovici has purchased three companies: Perseus, WebSurveyor and Surveyo, as well as Raosoft web-survey assets).</LI> <LI><B>When?</B> Begin planning for EFM implementations now! As Jim said, “Your competitors are getting better at this!” Gartner estimates that EFM implementations will grow 15-20% in 2010 over 2009. When planning to implement an EFM system, Jim noted that “EFM functionality is more diverse and complex than many organizations perceive.” </LI> <LI><B>Where?</B> By department and across the organization. In online communities. Set up an online community with member profiles containing information such as the member’s product portfolio, demographics and geographic location. Use forums for ad-hoc feedback, with text mining supporting the qualitative analysis. For the quantitative side, field surveys to not only community members but to customers who aren’t community members as well; let customers select their preferred modality. </LI></UL></DIV> <DIV>So there you have the 5 W’s of EFM.</DIV> </article> <article> <h1>CRM Trichotomy: Operational CRM, Analytical CRM, Social CRM</h1> <p>Jeffrey Henning — Mon, 14 Sep 2009 13:31:00 GMT</p> <DIV>Woven through the presentations I’ve seen this morning at the <A href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009" mce_href="http://blog.vovici.com/blog/bid/22602/Gartner-CRM-Summit-Conference-Recap-2009">Gartner North American CRM Summit</A> is an emphasis on the Gartner trichotomy of CRM into operational CRM, analytical CRM and social CRM. In the morning keynote, Gartner analyst Michael Maoz said that Gartner is seeing spending shift from operational CRM to analytical and social CRM.</DIV> <DIV> </DIV> <DIV>Maoz reported the following ballpark estimates for the growth between the segments:</DIV> <DIV> <TABLE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; BORDER-COLLAPSE: collapse; BORDER-TOP: medium none; BORDER-RIGHT: medium none; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-yfti-tbllook: 1184; mso-padding-alt: 0in 5.4pt 0in 5.4pt" class=MsoTableGrid border=1 cellSpacing=0 cellPadding=0> <TBODY> <TR style="mso-yfti-irow: 0; mso-yfti-firstrow: yes"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1" vAlign=top width=213><?xml:namespace prefix = o /><o:p> </o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1" vAlign=top width=213>Spending<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1" vAlign=top width=213>Growth<o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 1"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=213>Operational CRM<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>60%<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>10%<o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 2"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=213>Analytical CRM<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>30%<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>30%<o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 3; mso-yfti-lastrow: yes"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=213>Social CRM<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>10%<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 159.6pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=213>60%<o:p></o:p></TD></TR></TBODY></TABLE></DIV> <DIV> </DIV> <DIV>Underlying the importance of the social aspect, Maoz said “Networks matter more to your success than any other initiative.” He emphasized that social technologies now (externally to vendors!) provide consumers more information than those same vendors have internally. He gave two examples to illustrate this: </DIV> <DIV> <OL> <LI>He went to the web site of his refrigerator maker to buy a filter, where it treated his search “refrigerator [model #] filter” as an “any keyword” search, giving him dozens of untargeted links, before he was finally able to find the single fridge filter they sold. He decided not to buy it, then went to Google, found three filters on Amazon rather than the one, read user-submitted reviews, found a link to a crowdsourced Howcast video on how to install one of the filters, then went back to Amazon and purchased it. Superior service from the social Web!</LI> <LI>Calling Toyota for a problem with his RAV4. Yes, they had CRM and instantly knew him, knew his car and knew its warranty-replacement history. Unfortunately, the customer service representative didn’t have access to any of the three articles in the <I>New York Times </I>and elsewhere (which included references to internal Toyota engineering documents) that had led Maoz to the conclusion his car qualified for additional warranty work.</LI></OL></DIV> <DIV>Jim Davies, in the “Role of Technology in Improving the Customer Experience”, returned to the CRM trichotomy, using it for a taxonomy of CE initiatives:</DIV> <DIV> </DIV> <DIV> <TABLE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; BORDER-COLLAPSE: collapse; BORDER-TOP: medium none; BORDER-RIGHT: medium none; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-yfti-tbllook: 1184; mso-padding-alt: 0in 5.4pt 0in 5.4pt" class=MsoTableGrid border=1 cellSpacing=0 cellPadding=0> <TBODY> <TR style="mso-yfti-irow: 0; mso-yfti-firstrow: yes"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1" vAlign=top width=160><o:p> </o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1" vAlign=top width=160><B>Before Experience</B><B><o:p></o:p></B></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1" vAlign=top width=160><B>During the Experience</B><B><o:p></o:p></B></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: black 1pt solid; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1" vAlign=top width=160><B>After the Experience</B><o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 1"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=160><B>Operational CRM</B><o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Campaign Management<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Knowledge Management<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Survey<o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 2"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=160><B>Analytical CRM</B><o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Propensity Mode<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Real Time Next Best Action<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Speech Analytics<o:p></o:p></TD></TR> <TR style="mso-yfti-irow: 3; mso-yfti-lastrow: yes"> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1" vAlign=top width=160><B>Social CRM</B><o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Forum<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Customer Self-Service<o:p></o:p></TD> <TD style="BORDER-BOTTOM: black 1pt solid; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 5.4pt; WIDTH: 119.7pt; PADDING-RIGHT: 5.4pt; BORDER-TOP: medium none; BORDER-RIGHT: black 1pt solid; PADDING-TOP: 0in; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; mso-border-bottom-themecolor: text1; mso-border-right-themecolor: text1" vAlign=top width=160>Media Monitoring<o:p></o:p></TD></TR></TBODY></TABLE></DIV> <DIV> </DIV> <DIV>Vendors have too often taken an “after the experience” operational view of Customer Experience, and need to broaden the initiatives they are considering deeper into the analytical and social spheres.</DIV> <DIV> </DIV> <DIV>Good stuff, and I look forward to more insights to come.</DIV> </article> <article> <h1>Twitter Research: Supplementing Surveys with Twitter Analysis</h1> <p>Jeffrey Henning — Thu, 10 Sep 2009 09:52:00 GMT</p> <div>Last night here in the U.S. we had the latest passing of the political football on healthcare, as the President exhorted Congress to pass health-care reform. For CNN, <a mce_href="http://i2.cdn.turner.com/cnn/2009/images/09/09/top13.pdf" href="http://i2.cdn.turner.com/cnn/2009/images/09/09/top13.pdf">Opinion Research Corporation surveyed by telephone</a> 427 adult Americans who watched the Presidential speech. <a mce_href="http://www.pollster.com/blogs/instant_reaction_polls_a_presp.php" href="http://www.pollster.com/blogs/instant_reaction_polls_a_presp.php">Surveying Presidential speeches</a> is a pretty narrow domain, but given its importance, one that has been studied and shown to be full of issues: the most important issue to keep in mind is that such speeches are watched mainly by fans of the President, not by a representative group of Americans. And, in that tradition, this most recent CNN poll is representative only of that group that watched the speech last night, a group that skewed to the left politically: 45% identified themselves as Democrats, 37% as Independents and 18% as Republicans.</div><div> </div><div> </div><div>Some highlights from the poll:</div><div><ul><li>77% of respondents had a somewhat or very positive overall reaction to the speech.</li><li>70% thought the President’s policies will move the country in the right direction, up from 60% of the same panel surveyed in the four days before the speech</li><li>67% favor Obama’s health care plan, up from 53%</li><li>75% think it somewhat or very likely that Congress will pass most of his proposals</li></ul></div><div>Taken by itself those highlights don’t tell us much. Like most quantitative research, the narrative, the story behind the story, is missing.</div><div> </div><div>Tonight I’ll go from watching political football to watching the start of the NFL season. NFL games are broadcast with a play-by-play announcer, who describes the details of the game as it unfolds, and a color commentator or color analyst, who provides background information and anecdotes. Let’s flesh out the play-by-play of the poll with some color commentary from Twitter this morning:</div><div> </div><div><ul><li>77% of respondents had a somewhat or very positive overall reaction to the speech.</li><ul><li>Positive</li></ul><ul><ul><li>“President Obama Rocked My World Last Night Greatest Speech Ever by any President ever in History of USA !”</li><li>“hi good evening! Have u watched obama's speech on health care? It was a good one, as expected. Obama is one eloquent man.”</li><li>“I thought President's Obama's speech on healthcare was much needed.”</li></ul></ul><ul><li>Negative</li></ul><ul><ul><li>“Had to laugh at Obamas speech last night! Bi Partiscian negotiations on the healthcare bill? NOT!!!!!!”</li></ul></ul><ul><ul><li>“Watching Obama give a speech is like watching someone watch a tennis match as his head just bobs back and from teleprompter to teleprompter.”</li></ul></ul><ul><ul><li>“Dennis Miller had it right - if POTUS cannot start a speech on time, do you trust the govt to deliver healthcare on time & under budget?”</li></ul></ul><li>70% thought the President’s policies will move the country in the right direction, up from 60% of the same panel surveyed in the four days before the speech</li><ul><li>Positive</li></ul><ul><ul><li>“Busy @work today. Looking forward to catching up on the Obama health speech later, the fate of 46million uninsured Americans is in the balance”</li></ul></ul><ul><li>Negative</li></ul><ul><ul><li>“scared for our Country. That speech didn't convince me that his plan is best for America...I still don't believe he is best for America”</li></ul></ul><ul><ul><li>“So..big Obama speech. Is there any more agreement? No! I'm really disappointed that America has lost it's MOJO”</li></ul></ul><li>67% favor Obama’s health care plan, up from 53%</li><ul><li>Positive</li></ul><ul><ul><li>“finally listening to yesterday's obama health care speech...health care is ONE SIXTH of our economy??!! lord. we've got to get it together..”</li></ul></ul><ul><li>Negative</li></ul><ul><ul><li>“Good morning! I think his speech accomplished nothing, he didnt speak of specific details, and mandatory healthcare is dumb”</li></ul></ul><li>75% think it somewhat or very likely that Congress will pass most of his proposals</li><ul><li>Positive</li></ul><ul><ul><li>“Its not enuf to be excited by Obama's speech - we must act! If you believe, call or email your Congress reps TODAY & tell them enuf is enuf!”</li></ul></ul><ul><li>Negative</li></ul><ul><ul><li>“Overall I think Obama's speech is not the issue, it's what happens after the speech. Will Congress give us same garbage or real reform?”</li></ul></ul></ul></div><div>A survey supplemented with Twitter commentary provides an interesting mix of quantitative and qualitative information. The Twitter audience is not representative of last night’s speech watchers to any degree; reading the tweets, in fact, you see that some watched last night, some this morning, some plan to watch, some read the transcript and some are simply reacting to news reports with no plans to watch. So it’s a different cross-section of the populace, and one not limited to America.</div><div> </div><div>I downloaded about 400 tweets this morning between 9:17 am and 10:35 am to write this post [BTW, I'm <a mce_href="http://twitter.com/jhenning" target="_new" href="http://twitter.com/jhenning">@jhenning</a> on Twitter]. In the interests of time, I downloaded any tweet that had the word “speech” in it; that seemed more neutral than “President” or “Prez” or “Obama” or “POTUS”. It did mean that some tweets unrelated to the speech crept in. </div><div> </div><div>A quick Wordle word cloud of the results make a few additional points:</div><div></div><div> </div><div>Besides the point-of-fact words “Obama” and “last night”, you see many references to Joe Wilson, the U.S. Representative who must have dreamt he was in the House of Commons last night shouting at the Prime Minister. Words related to that include “Rep” and “outburst” and “LIE”. Clearly Joe Wilson is the unintentional story of the day.</div><div> </div><div>But then, if you live in the U.S., you didn’t need Twitter to tell you that. </div><div> </div><div>In the comments section, please let me know how you are using Twitter to supplement your market research.</div> </article> <article> <h1>August 2009 Posts by Popularity</h1> <p>Jeffrey Henning — Mon, 07 Sep 2009 09:52:00 GMT</p> Our most popular posts last month:<p class="MsoNormal"></p><ol><li><a mce_href="http://blog.vovici.com/blog/bid/20582/The-Seven-Habits-of-Highly-Successful-Surveys" href="http://blog.vovici.com/blog/bid/20582/The-Seven-Habits-of-Highly-Successful-Surveys">The Seven Habits of Highly Successful Surveys</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/21102/Rating-Scale-Best-Practices" href="http://blog.vovici.com/blog/bid/21102/Rating-Scale-Best-Practices">Rating Scale Best Practices</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20296/Apple-Does-No-Market-Research-So-You-Don-t-Have-To-Either" href="http://blog.vovici.com/blog/bid/20296/Apple-Does-No-Market-Research-So-You-Don-t-Have-To-Either">Apple Does "No Market Research", So You Don't Have To Either</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20666/When-DIY-Surveys-Become-DYI-Surveys" href="http://blog.vovici.com/blog/bid/20666/When-DIY-Surveys-Become-DYI-Surveys">When DIY Surveys Become DYI Surveys</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer" href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer">Net Promoter Score is a Misnomer</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First" href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First">Survey Scales Listing Negative Choices First</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas" href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas">Customer Experience Best Practice Areas</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20625/TNS-Customer-Loyalty-Index" href="http://blog.vovici.com/blog/bid/20625/TNS-Customer-Loyalty-Index">TNS Customer Loyalty Index</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings" href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings">Customer Experience Study Findings</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/21083/Competitive-Market-Research" href="http://blog.vovici.com/blog/bid/21083/Competitive-Market-Research">Competitive Market Research</a></li></ol><p> This ranking is based on an average of total page views and page views per day. Total page views unfairly inflates the importance of the posts published the first week of August.</p><p class="MsoNormal"><b><i>See also:</i></b> <a mce_href="http://blog.vovici.com/blog/bid/18701/Unpopular-Posts-from-Voice-of-Vovici" href="http://blog.vovici.com/blog/bid/18701/Unpopular-Posts-from-Voice-of-Vovici">Unpopular Posts from "Voice of Vovici"</a></p> </article> <article> <h1>Sample Quality of Online Panels: Putting Lipstick on the Piggy Bank</h1> <p>Jeffrey Henning — Fri, 04 Sep 2009 09:16:00 GMT</p> <DIV>I’ve ignored many of the initiatives to improve the quality of third-party online panels. To me, these initiatives are laughable. Yes, you should…</DIV> <DIV> <UL> <LI>Seek to identify panelists participating in the same survey multiple times under different names</LI> <LI>Remove respondents who speed through their answers</LI> <LI>Have a broad-based demographic representation so that you do not need to weight individual respondents</LI></UL></DIV> <DIV>But these simply put lipstick on the piggy bank. They make it easier for organizations to continue to put cost before quality and to justify doing research on the cheap with third-party panels. “See? The panel companies are working hard to ensure consistent high quality!”</DIV> <DIV> </DIV> <DIV>Um, a consistent high quality convenience panel is certainly better than a low quality convenience panel. But it’s still a pig. Er, piggy bank: a cheap alternative to a <A href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sample</A>.</DIV> <DIV> </DIV> <DIV>The laws of mathematics have not been repealed: a convenience sample cannot be used to extrapolate to any target audience. A convenience sample is representative of its respondents only. This point keeps getting lost, as I saw last year at the MRA Conference at the presentation <A href="http://blog.vovici.com/blog/bid/17843/What-s-the-Catch-Does-Sample-Sourcing-Matter" mce_href="http://blog.vovici.com/blog/bid/17843/What-s-the-Catch-Does-Sample-Sourcing-Matter">What's the Catch? Does Sample Sourcing Matter</A>:</DIV> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>A pointed question from the audience said that probability sampling was the theoretical basis for the projectability of survey research and asked what the scientific underpinnings were for assuming that Internet research was similarly representative. Melanie [the presenter] answered that replicability is emerging as the standard instead of randomization and that the results from her research were replicable.</BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <DIV>What "irrational exuberance" was to NASDAQ, the third-party online panel is to MR.</DIV> <DIV> </DIV> <DIV>This week, Gary Langer, director of polling at ABC News, writes <A href="http://blogs.abcnews.com/thenumbers/2009/09/study-finds-trouble-for-internet-surveys.html" mce_href="http://blogs.abcnews.com/thenumbers/2009/09/study-finds-trouble-for-internet-surveys.html">in his column</A>:</DIV> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>A new study led by Stanford University researchers raises doubts about the accuracy of one of the most common forms of survey research, polls done among people who sign up to fill in questionnaires via the internet in exchange for cash and gifts.<BR> <BR>In the most extensive such analysis to date, David Yeager and Prof. Jon Krosnick compared seven non-random internet surveys with two surveys based instead on random or so-called probability samples. The non-probability internet surveys were less accurate, and customary adjustments did not uniformly improve them.<BR> <BR>While the random-sample surveys were “consistently highly accurate,” the internet surveys based on self-selected or “opt-in” panels “were always less accurate, on average, than probability sample surveys, and were less consistent in their level of accuracy,” the researchers said. Further, they said, adjusting these samples to known population values had no effect on accuracy (and in one case even worsened it) as often as that process, known as weighting, improved it.<BR><BR></BLOCKQUOTE> <DIV>Most Vovici customers are surveying house lists of customers, employees, resellers and other key constituencies. It’s very easy to do a random survey of employees when you have the email address of every employee and have empaneled the list of employees by synchronizing your HRIS. For surveys of prospects, many organizations are using the web for all lead generation and can easily field random samples of prospects. Unless you’re an e-commerce or SaaS business, though, it is more difficult to build a representative house list of customers that you can then random sample: check out these tips for creating and managing <A href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers" mce_href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers">representative email lists</A> of your customer base.</DIV> <DIV> </DIV> <DIV>Putting in regular processes to build a quality house list is like setting up automatic monthly withdrawals from checking to savings: better than the panel piggy bank as way to save research costs in the long run. Building such a house list is a sound investment towards conducting quality, representative survey research.</DIV> </article> <article> <h1>Co-Creation Best Practices</h1> <p>Brian Koma — Thu, 03 Sep 2009 17:52:00 GMT</p> <div>According to the <a mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx " href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx ">CE IQ</a> study, organizations with the most loyal customers actively engage those customers in co-creation.</div><div><br></div><div>A new breed of customer now exists who demands to be heard and who wants to engage in a two-way dialogue with their vendors about issues that are important to them. As a result, organizations that have the most loyal customers include them in the ideation process around product and process improvement. Very closely related to this practice is having a formal process for obtaining customer input into the planning cycle, instead of relying on ad-hoc methods. Fundamentally, these organizations are engaging customers at every step in the process and, importantly, are providing customers with status updates on the ideas that they have submitted, as well as informing them if their ideas have not been put into an implementation plan. </div><div><br></div><div>In fact, each of the four co-creation best practices we studied are in the top 9 best practices with the highest correlation to customer loyalty out of all 24 practices reviewed.</div><div><br></div><div> </div><div>How can your organization apply these best practices to build customer loyalty? First, create a formal process to engage in co-creation activities with customers, through quantitative surveys and through qualitative means such as social media and online communities. Share with your customers what steps your taking based on customer input, closing the feedback loop so that your customer are motivated to provide information on an ongoing basis.</div> </article> <article> <h1>Champion Model</h1> <p>Jeffrey Henning — Thu, 03 Sep 2009 09:38:00 GMT</p> <div>Inspired by the <a mce_href="http://blog.vovici.com/blog/bid/17878/The-Folk-Apostle-Model" href="http://blog.vovici.com/blog/bid/17878/The-Folk-Apostle-Model">folk Apostle Model</a>, TNS has its own customer-loyalty segmentation, the Champion Model, which is a quadrant analysis contrasting likelihood to recommend and likelihood to repurchase.</div><div> </div><div style="text-align: center;"> </div><div> </div><div><b><i>Note: </i></b>I swapped the axes from the customary presentation that TNS uses to make the segmentation more clearly match up with the Apostle Model.</div><div> </div><div>The four segments of this model:</div><div><ul><li><b>Champion</b> - <i>high repurchase intentions, high likelihood to recommend</i> – true apostles of your brand, who practice what they preach</li><li><b>Moral Supporter</b> – <i>low repurchase intentions, high likelihood to recommend</i> – false prophets of your brand</li><li><b>Rebel</b> - <i>low repurchase intentions, low likelihood to recommend</i> – these apostates won’t be customers long</li><li><b>Captive</b> - <i>high repurchase intentions, low likelihood to recommend</i> – these hostages can’t switch because of corporate policy or financial considerations</li></ul></div><div>The actual questions, taken from the <a mce_href="http://blog.vovici.com/blog/bid/20625/TNS-Customer-Loyalty-Index" href="http://blog.vovici.com/blog/bid/20625/TNS-Customer-Loyalty-Index">TNS CLI</a> template:</div><div><ul><li><b>Likelihood to recommend</b> - "Based on your experience, how likely would you be to recommend [organization/brand] to a friend or colleague looking for [product/service category]? Definitely would, Probably would, Might or might not, Probably would not, Definitely would not"</li><li><b>Likelihood to repurchase</b> – "For your next similar purchase of [product/service category], how likely would to be to buy from [organization/brand] again? Definitely would, Probably would, Might or might not, Probably would not, Definitely would not"</li></ul></div><div>This champion model is most well-known to Microsoft partners, for whom TNS will conduct this customer-loyalty segmentation for free (e.g., at Microsoft’s expense). The segmentation seems to suffer from not segmenting enough. In segmentations made public by two Microsoft partners, each showed samples sizes of two dozen respondents, with 100% champions; the partner average itself currently shows 88.6% of customers are champions.</div><div> </div><div>This model is great in its focus on loyalty behaviors, and it is certainly better than the traditional <a mce_href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model">Apostle Model</a> in aligning its segmentations with actual behavior: the Apostle Model itself segments a group into “Loyalists/Apostles” but doesn’t have any measure of likelihood to recommend, a key behavior that would be expected of a group that is evangelizing a particular vendor. (Though Bob E. Hayes, among others, shows how <a mce_href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question" href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question">CSAT</a> can drive <a mce_href="http://blog.vovici.com/blog/bid/19953/Advocacy-Loyalty-Index-ALI-and-Purchasing-Loyalty-Index-PLI" href="http://blog.vovici.com/blog/bid/19953/Advocacy-Loyalty-Index-ALI-and-Purchasing-Loyalty-Index-PLI">advocacy loyalty</a>.)</div><h4>Best Practices for Using the Champion Model</h4><div>TNS uses a five-point <a mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">bipolar scale</a> instead of a five-point unipolar scale, which will have greater reliability and validity. Accordingly, I suggest using this scale instead: Not at all likely, Slightly likely, Moderately likely, Very likely, Completely likely.</div><div> </div><div>A segmentation that classifies almost 90% of your customers as champions isn’t going to inspire your organization to improve. In truth, you want customers who are <i>completely likely</i> to repurchase and recommend: only classify the “completely likely” as champions. Your customer satisfaction initiatives should be designed around driving that type of behavior. Instead of using a traditional quadrant analysis, I refactor the analysis to be analogous to the Apostle Model:</div><div> </div><div style="text-align: center;"> </div><div> </div>This refactoring lowered one analysis I did from 71% champions to 21% champions! Remember, though, it's not about the score: it's about understanding how the highest level of promoters differ from the lowest so that you can make continuous improvement. </article> <article> <h1>Your Half-Human, Half-Vulcan Customers</h1> <p>Jeffrey Henning — Wed, 02 Sep 2009 16:07:00 GMT</p> <div>Your customers are like Mr. Spock from <i>Star Trek</i>. In popular culture (as opposed to geek culture), Spock has become a synonym for cold-blooded logical thinking. But, as the true geek knows, Spock’s father was from Vulcan and his mother was from Earth. Talk about an identity crisis! Spock is a half-Vulcan. Just like your customers.</div><div> </div><div>Customer loyalty literature expresses the dichotomy this way:</div><div><ul><li><b><i>Affective commitment</i></b> – The emotional pleasure your customer takes in doing business with you, their identification with your organization, their sense of belonging with your brand. We’ll call this <i>the Human impulse</i>.</li><li><b><i>Calculative commitment</i></b> – The cold-blooded logic that dictates why your customer has a business relationship with you: the economic analysis of the benefits vs. the costs of that relationship, the economic suffering if you fail them, and other rational considerations you offer. We’ll call this <i>the Vulcan impulse</i>.</li></ul></div><div>Now much of the drama of the original <i>Star Trek </i>series—and a key reason Spock is central to the recent reimagining of the franchise—grew from the half-human, half-Vulcan nature of Spock. How would he react in a particular situation? Which half would (temporarily) triumph?</div><div> </div><div>And much of the drama of your business involves determining what keeps each of your customers loyal. Will the Human impulse forgive a lapse on your part? Will the Vulcan impulse cause them to quite logically switch to another supplier that introduces a more economical alternative? Which half will win, and why?</div><div> </div><div><a mce_href="http://www.econ.kuleuven.be/robrecht.vangoole" href="http://www.econ.kuleuven.be/robrecht.vangoole">Robrecht Van Goolen</a> and <a mce_href="http://www.econ.kuleuven.be/katia.campo" href="http://www.econ.kuleuven.be/katia.campo">Katia Campo</a> investigate this issue in their research report, “Does Commitment Really Provide Protection against Critical Incidents and Competitive Switching Incentives? The Differential Impact of Affective and Calculative Commitment” [<a mce_href="https://lirias.kuleuven.be/bitstream/123456789/198353/1/9+sept.+08+-+Does+Commitment+Really+Provide+Protection+Against+Critical+Incidents+and+Competitive+Switching+Incentives_The+Differential+Impact+of+Affective+and+Calculative+Commitment.pdf " href="https://lirias.kuleuven.be/bitstream/123456789/198353/1/9+sept.+08+-+Does+Commitment+Really+Provide+Protection+Against+Critical+Incidents+and+Competitive+Switching+Incentives_The+Differential+Impact+of+Affective+and+Calculative+Commitment.pdf ">PDF</a>]. Their conclusions:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Based on the results of the preliminary analysis, we can conclude that <b>affective commitment [e.g., the Human impulse] plays a much more important role in building customer loyalty than calculative commitment [the Vulcan impulse]</b>. Not only does it have a stronger effect on customer loyalty in general, it also increases the ‘goodwill’ to accept (temporary) service failures and provides a much stronger protection against competitive inroads. </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><br>The distinction with more calculative based commitment becomes especially important when the effect of major events (service failures of the favorite company or switching incentives of competitive companies) and longer term effects (share-of-wallet) are considered. In the short term, and when the consequences of the event are of minor importance, calculative commitment may provide a similar protection as affective commitment. Yet, the probability that the trade-off between switching advantages and costs will turn out in the advantage of competitive companies is substantially higher when customers are only committed to the product or service based on expected rewards rather than satisfaction with and a positive attitude towards the product or service.</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><div>Yes, as Esteban Kolsky argues, "<a mce_href="http://www.estebankolsky.com/2009/09/01/loyalty-can-be-bought/" href="http://www.estebankolsky.com/2009/09/01/loyalty-can-be-bought/">Loyalty Can Be Bought</a>". But if you buy your customers’ loyalty with a loyalty reward program geared towards rational rewards, don’t be surprised when such customers (the "mercenaries" of <a mce_href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model">the Apostle Model</a>) sell their loyalty to a competitor who offers even better rational rewards. </div><div> </div><div>To build true loyalty, you have to appeal to the Human impulse, not the Vulcan impulse. Your loyalty reward program should work to increase customers’ affective commitment: increase the pleasure your customers take in doing business with you, strengthen their sense of belonging to your brand and deepen their sense of identification with your organization.</div><div> </div><div>In the 21st century or the 23rd century, Humans make more loyal customers than Vulcans do.</div> </article> <article> <h1>Jumping into the Pool before You Know the Water Depth</h1> <p>Jeffrey Henning — Mon, 31 Aug 2009 21:18:00 GMT</p> <div><i>This is a guest post by </i><a mce_href="http://www.middlesexconsulting.com/about.html" href="http://www.middlesexconsulting.com/about.html"><i>Sam Klaidman</i></a><i>, Principal Adviser at </i><a mce_href="http://www.middlesexconsulting.com" href="http://www.middlesexconsulting.com"><i>Middlesex Consulting Group</i></a><i>. Sam helps small and medium businesses grow by increasing their customer loyalty. </i></div><div><br></div><div>People initiate satisfaction and loyalty tracking programs for one simple reason – they want to know what their customers think. Maybe about their company, their products, their services or their perceived value being delivered to customers. But, while they may suspect the answers, they are uncertain and therefore they seek the <a mce_href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition" href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition">Voice of their Customer</a>. This represents real progress.</div><div><br></div><div>Once they decide to kick-off a survey program, they frequently begin to immediately craft the questions. This is where I get very concerned. The process owners acknowledge that they don’t know what their customers are thinking, but they consciously or unconsciously believe they know what’s important to the customer base. Sounds to me like a case of not knowing what you don’t know! </div><div><br></div><div>In my opinion, creating a survey program is a five-step process with the first step often skipped.</div><div><br></div><div><b>Step 1 – Identify your target customer’s most important interactions with your business.</b></div><div><br></div><div>Here are two ways to identify the main areas of interest (touch-points):</div><div><ul><li><i>The cheap and cheerful way</i> – Identify a reasonable number of customers that you believe represent a broad spectrum of your target audience. Talk with them, either face-to-face or by telephone, explain what your company is going to do, and why, and gather their opinion of the most important touch-points that shape their relationship with your company. Make sure to discuss every interaction between the company and the customer. </li><li><i>The rigorous and more expensive approach</i> – Commission an external organization to conduct a series of focus groups with representatives of your target. The primary advantages of this approach are:</li><ul><li>Unbiased results</li><li>“Better” answers</li><li>All areas will be probed (the knowledge of what to question is a big part of what you are paying for)</li><li>You show customers and employees how serious your business is about this initiative.</li></ul></ul></div><div>On the other hand, of course, it will take longer, and cost more, that doing it internally.</div><div><br></div><div><b>Step 2 – Identify the key drivers of customer loyalty.</b></div><div><br></div><div>Plan a survey program to obtain a <a mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">statistically valid</a> analysis of the “key drivers” of customer loyalty by doing it yourself or with a company that specializes in satisfaction and loyalty surveys. Based on knowledge resident within your company, preferably at the operational level, identify the elements of each touch-point that your customers responded were important to their relationship with your business. And question the overall level of satisfaction and loyalty and likelihood of additional purchases (the real reason for the whole program). </div><div><br></div><div>Survey a statistically valid sample of your target segment or audience. When all results are obtained, perform a regression analysis to identify the key drivers of satisfaction, loyalty and additional purchases. Make certain that the results are valid by looking at the <a mce_href="http://en.wikipedia.org/wiki/Coefficient_of_determination" href="http://en.wikipedia.org/wiki/Coefficient_of_determination">R-squared</a>. From this analysis, the company will have a very good idea about what is important for each key touch-point your customers identified. And it will be able can see which variable, if any, impacts more that one key parameter. </div><div><br></div><div><b>Step 3 – Identify the key drivers of satisfaction for touch-points identified in the previous step.</b></div><div><br></div><div>The objective is to identify the attributes of the selected touch-points that most strongly influence how your customers feel about their experiences with your business’ most influential interactions. Use the same techniques and rigor as in Step 2 since the results will be driving investments and actions. Do not let individual bias guide which attributes you include. Be as sweeping as possible since, once again, you may not know what you don’t know.</div><div><br></div><div>Then the fun begins!</div><div><br></div><div><b>Step 4 – Implement a relationship or </b><a mce_href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey" href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey"><b>transactional survey</b></a><b> process.</b></div><div><br></div><div>Using the results of Step 3, plan a survey program that focuses on those areas with the greatest bang for the buck. As progress is made, begin tackling the lower impact items on the list. Remember to make sure you get enough completed surveys so that the results are statistically representative of your whole target audience.</div><div><br></div><div><b>Step 5 – Re-verify the key drivers identified in Steps 2 and 3.</b></div><div><br></div><div>Unless there are dramatic changes in the companies’ business environment it is unlikely that the key drivers will change drastically in a year. Yes, your company should periodically revalidate its key driver list but you have time to implement improvements first.</div><div><br></div><div>And remember, if it isn’t important then why spend the time and money to find out how the customers perceive your performance? So focus your ongoing survey efforts on the items your customers say are most important to their ongoing relationship with your business. Now, at last, you know what you didn't know when you jumped headfirst into the survey pool!</div> </article> <article> <h1>Rating Scale Best Practices</h1> <p>Jeffrey Henning — Fri, 28 Aug 2009 08:55:00 GMT</p> <DIV>Most survey researchers pay little mind to the research into the efficacy of different scales, frequently using obsolete rating scales in their surveys: changing scales can be a real bugbear.</DIV> <DIV> </DIV> <DIV>That said, here are some well-researched best practices when it comes to scales:</DIV> <DIV> <UL> <LI>Use 5-point scales for unipolar scales and 7-point scales for <A href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">bipolar scales</A>:</LI></UL></DIV> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>"To explore the relation between scale length and reliability, we conducted a meta-analysis of the results of many past studies. Our data consist of results from 706 tests of reliability taken from thirty different between-subject studies. We combined various measures of reliability and various sample sizes, controlling for these and other factors in determining the relation of scale length to reliability. In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five." - Jon Krosnick, professor of communication at Stanford, <A href="http://communication.stanford.edu/faculty/krosnick/" target=_new mce_href="http://communication.stanford.edu/faculty/krosnick/">"The Optimal Length of Rating Scales to Maximize Reliability and Validity"</A></BLOCKQUOTE></BLOCKQUOTE> <DIV> <UL> <LI>Use <A href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">fully labeled scales</A> without showing respondents numeric ratings. Such scales are preferred by respondents and have higher reliability and predictive validity than numeric scales.</LI> <LI>Exclude “Don’t know” and <A href="http://blog.vovici.com/blog/bid/20397/No-Opinion-as-a-Question-Choice" mce_href="http://blog.vovici.com/blog/bid/20397/No-Opinion-as-a-Question-Choice">“No opinion” as a choice</A> when presenting your scale.</LI> <LI>The 0-to-10 rating scale for Net Promoter has the lowest reliability and predictive validity <A href="http://mattberent.net/Netpromoter_-_AAPOR.pdf" target=_new mce_href="http://mattberent.net/Netpromoter_-_AAPOR.pdf">of four scales tested</A>.</LI></UL></DIV> <DIV>The above findings are backed up by scientific research. The following best practices, on the other hand, are my personal preferences, for which I was not able to find supporting data:</DIV> <DIV> <UL> <LI>Use <A href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">unipolar scales instead of bipolar scales</A> wherever possible.</LI> <LI>List <A href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First" mce_href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First">rating scales</A> with the most negative item first.</LI></UL></DIV> <DIV>I try not be dogmatic about scale research and often field questionnaires that contradict some of the above points, if I think it is better for respondents to do so or if I need to benchmark the question results to older questions that use older scales. And I’m late to embracing the 5-point scale, as reading some of my past posts will reveal (<A href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">Rating Scale Comparison: Weighing Different Scales for Survey Research</A> and <A href="http://blog.vovici.com/blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template" mce_href="http://blog.vovici.com/blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template">Apostle Model Best Practices and Survey Template</A>).</DIV> <DIV> </DIV> <DIV>That said, factoring in the research and my recommendations, this is what I consider to be the best <A href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question" mce_href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question">CSAT</A> scale:</DIV> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>What is your overall satisfaction with our company?<BR></BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> <UL> <LI>Not at all satisfied</LI> <LI>Slightly satisfied</LI> <LI>Moderately satisfied</LI> <LI>Very satisfied</LI> <LI>Completely satisfied </LI></UL></BLOCKQUOTE> <DIV>Sometimes no common scale will do and you need to <A href="http://blog.vovici.com/blog/bid/18620/Custom-Scale-Development" mce_href="http://blog.vovici.com/blog/bid/18620/Custom-Scale-Development">develop a custom scale</A>.</DIV> <DIV> </DIV> <DIV>When a study mixes different lengths of scales, consider <A href="http://blog.vovici.com/blog/bid/18584/Standardization-of-Scales-in-Survey-Analysis" mce_href="http://blog.vovici.com/blog/bid/18584/Standardization-of-Scales-in-Survey-Analysis">standardizing the scales in survey analysis</A>, for instance by mapping scales to a 0 to 10 scale. This can make reports of the results easier to understand. While respondents dislike numeric scales, fully labeled scales are typically analyzed numerically, and the 0-to-10 mapping can aid analysis.</DIV> <DIV> </DIV> <DIV>Most organizations fail to <A href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales" mce_href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales">standardize on rating scales</A>, making it difficult to compare the results from study to study, from department to department. If you haven’t yet done so, please consider coming up with standard practices to guide your research. To contradict Emerson, when it comes to rating scales, a foolish <I>inconsistency </I>is the hobgoblin of little minds.</DIV> </article> <article> <h1>Competitive Market Research</h1> <p>Jeffrey Henning — Thu, 27 Aug 2009 15:26:00 GMT</p> <div>Surveys can be used in many different ways to gather competitive <a mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx " href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx ">customer satisfaction</a> and loyalty data:</div><div><ul><li><b>Account Manager Surveys</b> – You can survey your company’s sales staff about which competitors they run into, how often, how often you win or lose and why. </li><ul><li><b><i>Bias: </i></b>The results are filtered through account-manager perceptions. </li><li><i><b>Benefit:</b> </i>This is a cheap and fast method that sums up what your sales force sees. It’s great to do once a year, as an added input to planning exercises.</li></ul><li><b>Won-Business Surveys</b> – You can send a follow-up survey to new customers asking them about their purchase process, who else they considered and why, and then why they bought from you instead. </li><ul><li><b><i>Bias: </i></b>Even more biased than your lost-business sample.</li><li><b><i>Benefit: </i></b>Such surveys help identify the differentiating messages you should be emphasizing in your marketing and provide an ongoing stream of competitive intelligence.</li></ul><li><b>Lost-Business Surveys</b> – You can survey prospects who haven’t purchased in X months (where X equals whatever makes sense given your sales cycle). Ask them if they purchased elsewhere, and—if so—find out who they purchased from, why and how satisfied they are. </li><ul><li><b><i>Bias: </i></b>you are only surveying your competitors’ customers who considered you – since many of your competitors probably have vertical markets or functional niches where you might rarely be considered, you are not getting a true picture of their customer satisfaction. </li><li><b><i>Benefit: </i></b>You are getting a picture of competitors’ performance where it most directly hurts you and on an ongoing basis.</li></ul><li><b>General Market Research Surveys</b> – If you have high market share, then rent panel of your appropriate target demographic from a company like <a mce_href="http://www.erewards.com" target="_new" href="http://www.erewards.com">eRewards</a>, <a mce_href="http://www.greenfield.com/" target="_new" href="http://www.greenfield.com/">Greenfield Online</a> or <a mce_href="http://www.surveysampling.com/" target="_new" href="http://www.surveysampling.com/">SSI</a>. Top avoid having your brand bias the results in any way, do not give away your organization’s identity: host your survey elsewhere, such as on the panel provider’s systems, on your <a mce_href="http://www.vovici.com" href="http://www.vovici.com">survey software</a> platform (if you don’t have a custom domain name) or with a third-party market researcher. If you have low market share, then you will probably need to rent the list from a magazine or newsletter.</li><ul><li><i><b>Bias:</b></i> Every panel has its own bias, as does every periodical; your survey will really only be representative of the magazine readership or panel membership. </li><li><b><i>Benefit: </i></b>This is far less biased than your lost- and won-business surveys but is typically only an annual or biennial point-in-time survey.</li></ul></ul></div><div>Since your <a mce_href="http://blog.vovici.com/blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship" href="http://blog.vovici.com/blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship">survey response rates</a> are proportional to your relationship with respondents, you will get the lowest response rates from lost-business surveys and general market-research surveys. To boost those response rates, make sure to send out <a mce_href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">survey reminders</a> and consider offering a <a mce_href="http://blog.vovici.com/blog/bid/18167/Survey-Incentive-Strategies" href="http://blog.vovici.com/blog/bid/18167/Survey-Incentive-Strategies">survey incentive</a>.</div><div><br></div> </article> <article> <h1>Net Promoter Score is a Misnomer</h1> <p>Jeffrey Henning — Thu, 27 Aug 2009 10:44:00 GMT</p> <div>One of my frustrations with the <a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Net Promoter Score</a> is that the analyst is supposed to interpret the question differently than the respondent. The <i>respondent </i>is asked a unipolar question, measuring the single dimension of likelihood to recommend an organization:</div><div> </div><div></div><div> </div><div>The <i>analyst</i>, however, is told to interpret this as a <a mce_href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales" href="http://blog.vovici.com/blog/bid/21024/Bipolar-Unipolar-Scales">bipolar scale</a>: likelihood to promote vs. likelihood to detract. A respondent who says they are “Not at all likely” to recommend is treated as a detractor, as is a respondent who says they are moderately likely to recommend (rating of 6). The absurdity of this interpretation is made clear when you ask follow-up questions to the respondents that assume this interpretation.</div><div><ul><li>Why are you likely to recommend against our company? [Asked only of so-called “detractors”]</li><ul><li>“I am not in a position to recommend or not: 5 was neutral so why do I have this question to answer?”</li><li>“That’s not what I said.”</li><li>“I'm never in the position [to recommend].”</li></ul><li>Why are you only somewhat likely to recommend our company? [Asked only of “passives”]</li><ul><li>“A 7 on a scale of 10 is good! Depends on the person's needs.”</li><li>“It is unlikely I would be asked for a recommendation.”</li><li>“Because you made me answer this question before I could finish the survey.”</li></ul></ul></div><div><b>Since the Net Promoter Scale doesn’t actually ask about detracting behavior</b>, it should not be interpreted as a “net” of promoters minus detractors; <b>at best, it can be interpreted as a Net Very-Likely-to-Recommend Score.</b></div><div> </div><div>To correct this flaw, <a mce_href="http://mattberent.net/Netpromoter_-_AAPOR.pdf" href="http://mattberent.net/Netpromoter_-_AAPOR.pdf">Schneider, Berent, Thomas and Krosnick</a> recommend using a fully labeled seven-point scale:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">How likely is it that you would recommend us or recommend against us to a friend or colleague?<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><ul><li>Extremely likely to recommend against</li><li>Moderately likely to recommend against</li><li>Slightly likely to recommend against</li><li>Neither likely to recommend nor recommend against</li><li>Slightly likely to recommend</li><li>Moderately likely to recommend</li><li>Extremely likely to recommend </li></ul></blockquote><div>So this is one of the rare examples where a bipolar scale is more appropriate than a unipolar scale. That said, the suggestion above is a verbose, confusing question for many respondents. If a client really wants a promoter/detractor segmentation, I do use this question, but otherwise I use a unipolar likely-to-recommend question and steer my client to richer segmentation schemes instead.</div> </article> <article> <h1>Bipolar & Unipolar Scales</h1> <p>Jeffrey Henning — Wed, 26 Aug 2009 16:13:00 GMT</p> <div>Survey rating scales are either unipolar or bipolar. A unipolar scale prompts a respondent to think of the presence or absence of a quality or attribute: <i>not at all satisfied, slightly satisfied, moderately satisfied, very satisfied or completely satisfied</i>. Statisticians often map these answers to a scale from 0 to 1 (e.g., 0.00, 0.25, 0.50, 0.75, 1.00 for a five-point scale).</div><div> </div><div>Where a unipolar scale has that one “pole”, a bipolar scale has two polar opposites. A bipolar scale prompts a respondent to balance two opposite attributes in mind, determining the relative proportion of these opposite attributes. A common bipolar scale: <i>Completely dissatisfied, mostly dissatisfied, somewhat dissatisfied, neither satisfied nor dissatisfied, somewhat satisfied, mostly satisfied, completely satisfied</i>. Statisticians often map these answers to a scale with 0 in the middle: -3, -2, -1, 0, 1, 2, 3.</div><div> </div><div></div><div> </div><div>Which type of scale should you use? Wherever possible, a unipolar scale will be the better choice.</div><div><ol><li><b>It’s less mentally taxing for the respondent</b>: instead of balancing two opposing attributes in mind, they only have to consider one attribute.</li><li><b>It's more economical</b>, by providing fewer choices: five-point scales are the most valid for unipolar scales, compared to seven-point scales for bipolar scales.</li><li><b>You’re less likely to get it wrong</b>: a survey author can choose attributes for both poles that are not in fact polar opposites, further confusing respondents. For instance, asking a respondent to rate competitors on a scale from “very affordable” to “very high quality”.</li><li><b>Many bipolar scales really measure one dimension</b>: if “not at all X” is synonymous with “very dis-X” then use the unipolar scale instead. For instance, “not at all important” is synonymous with “very unimportant”, “not at all appropriate” is synonymous with “very unimportant”, and so on, making those perfect candidates for unipolar scales.</li></ol></div><div>Rarely, though, a bipolar scale may be the better choice. But I’ll save that for <a mce_href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer" href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer">tomorrow’s blog post</a>.</div> </article> <article> <h1>Survey Testing Case Study</h1> <p>Jeffrey Henning — Tue, 25 Aug 2009 15:44:00 GMT</p> <div>Brian’s post on our <a mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx" href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx">CE IQ survey</a> reminded me of the <a mce_href="http://blog.vovici.com/blog/bid/18842/Survey-Test-Mode" href="http://blog.vovici.com/blog/bid/18842/Survey-Test-Mode">survey test</a> and review process I went through for that questionnaire. Here are the types of edits I made at each of the four levels of testing:</div><div> </div><div><b>Self-Test</b> – “Run through the survey, answering it yourself, multiple times.”</div><div>I spent five hours reviewing the questionnaire. Each time I read it, I found a new issue. My goal was to get to the point where I would self-test it and not find a single issue. Some of the items I encountered:</div><div><ol><li>Upon re-reading, I found quite a few questions were worded awkwardly. I did lots of proofreading and wordsmithing to make sure questions were clear. </li><li>I encountered a few <a mce_href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions" href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions">double-barreled questions</a> that I split into separate questions.</li><li>I only had three <a mce_href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys" href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys">skip patterns</a> configured but upon testing discovered I had specified two of them incorrectly!</li><li>I discovered that I didn’t have <a mce_href="http://blog.vovici.com/blog/bid/20397/No-Opinion-as-a-Question-Choice" href="http://blog.vovici.com/blog/bid/20397/No-Opinion-as-a-Question-Choice">no-opinion choices</a> on the required questions.</li><li>I realized one <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">rating scale</a> was backwards from the others (choices listed from most favorable to least favorable where the rest were listed with <a mce_href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First" href="http://blog.vovici.com/blog/bid/20516/Survey-Scales-Listing-Negative-Choices-First">negative choices first</a>).</li><li>I wanted to make sure each <a mce_href="http://blog.vovici.com/blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices" href="http://blog.vovici.com/blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices">choose all that apply</a> question had an “Other” choice in the list.</li><li>I noticed things that would only bother a copy editor or someone, like me, who had done copy-editing in the past. For instance, this type on inconsistency bothered me: ending choice lists with “Other, Please specify:”, “Other (please specify)” and “Other, please specify” rather than one consistent phrase.</li><li>On the last pass, I realized that I had asked for <a mce_href="http://blog.vovici.com/blog/bid/18047/Firmographic-Definition-Template" href="http://blog.vovici.com/blog/bid/18047/Firmographic-Definition-Template">firmographics</a> but no <a mce_href="http://blog.vovici.com/blog/bid/18176/Demographic-Questions-Sample-Survey-Template" href="http://blog.vovici.com/blog/bid/18176/Demographic-Questions-Sample-Survey-Template">demographics</a>.</li></ol></div><div>Eventually I gave up. There was one choose all that apply question where two of the choices were mutually exclusive – that question really should have been reworded, but I noticed it on my last pass through the questionnaire and was far over my time budget for this phase of testing.</div><div> </div><div><b>Pre-Test</b> – “Invite coworkers or friendly outsiders to take the survey.”</div><div>The great thing about doing a survey with others is that the more eyes the more likely you are to catch problems. The open-source movement has an axiom for this: "Given enough eyeballs, all bugs are shallow” (also known as <a mce_href="http://en.wikipedia.org/wiki/Linus'_Law" target="_new" href="http://en.wikipedia.org/wiki/Linus'_Law">Linus’s Law</a>).</div><div> </div><div>The bigger issue that was noticed was that the questionnaire consisted of a mix of British English and American English, which made it annoying to both Americans and Brits. <a mce_href="http://www.vovici.com/about/news-and-press/2009/CGA.aspx" href="http://www.vovici.com/about/news-and-press/2009/CGA.aspx">CGA, one of our partners</a> in the UK, wrote the first draft of the questionnaire, and I wrote the subsequent drafts. To clean it up and make both sides of the Pond happy, I ended up fielding the survey in U.S. English with a U.K. English translation (e.g., <i>organization</i> to <i>organisation</i>, <i>dollars </i>to <i>pounds sterling</i>, etc.).</div><div> </div><div>The pre-test also caught a minor font issue (serif in tables, sans serif elsewhere) that had broken a customized display theme.</div><div> </div><div>During the pre-test, we made sure to test the questionnaire in three browsers (Internet Explorer, Firefox and Chrome).</div><div> </div><div><b>Pilot Test</b> – “Invite 10% of the targeted list to take the survey.”</div><div>I’m always nervous about sending out an invitation to my entire list of potential respondents, in case I realize one click too late that I’ve made a mistake. I sent the initial survey invitation out to 10% of the first wave of invitations, and everything went fine.</div><div> </div><div><b>Publish</b> – “Invite the world to take the survey.”</div><div>Once it was clear that the survey was running smoothly, I invited everyone on the first wave of my mailing list.</div><div> </div><div>Was I happy to end up spending over 8 hours testing a fairly straightforward 35-question survey? No, but I owed it to my hundreds of respondents to make the survey as streamlined and simple as possible.</div> </article> <article> <h1>Employee-Customer Engagement Best Practices</h1> <p>Brian Koma — Fri, 21 Aug 2009 08:44:00 GMT</p> <div>According to the <a mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx" href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx">CE IQ study</a>, organizations with the most loyal customers not only measure and monitor employee interactions with customers but then share that feedback with employees (a strong correlation of 0.54 to <a mce_href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas" href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas">the study’s loyalty index</a>). This creates a closed feedback loop that allows customer-facing staff members to understand the impact of their interactions on customers and enables the creation of programs that allow employees to improve those interactions over time. </div><div> </div><div> </div><div>In addition, companies with the most loyal customers also periodically share <a mce_href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies" href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies">VOC</a> (<a mce_href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition" href="http://blog.vovici.com/blog/bid/18497/Voice-of-the-Customer-Definition">Voice of the Customer</a>) information with employees across the organization, ensuring widespread understanding of the customer’s point of view. Taken a step further, these organizations also engage customers in two-way dialogues through online communities. As Jeffrey wrote in <a mce_href="http://blog.vovici.com/blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009" href="http://blog.vovici.com/blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009">The Top Ten Reasons for Building an Online Community in 2009</a>, “This allows a wide variety of individuals within the business to interact and engage with customers. Dell has 40 employees participate in <a mce_href="http://blog.vovici.com/Blog/bid/18060/Turning-Online-Communities-into-Sustainable-Competitive-Advantage" href="http://blog.vovici.com/Blog/bid/18060/Turning-Online-Communities-into-Sustainable-Competitive-Advantage">a team called Communities & Conversations</a>. CEOs always talk about making their organizations ‘customer centric’: by talking to customers and evangelizing their viewpoints across the company, as these Dell team members do, the organization truly becomes centered on the customer.”</div><div><br></div><div>Finally, organizations with the most loyal customers also recognize <a mce_href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty" href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty">the link between satisfied employees and loyal customers</a> by measuring employee satisfaction on a regular basis. Satisfied and loyal employees reflect their positive attitudes and good behaviors to customers, who in turn increase their loyalty to the business. </div><div> </div><div>Averaging these four best practices together, the resulting index has a 0.65 correlation to customer loyalty, highest of any of <a mce_href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas" href="http://blog.vovici.com/blog/bid/20810/Customer-Experience-Best-Practice-Areas">the six practice areas</a> examined. Clearly, employee-customer engagement is a key to unlocking greater customer loyalty.</div> </article> <article> <h1>Customer Experience Best Practice Areas</h1> <p>Jeffrey Henning — Thu, 20 Aug 2009 16:55:00 GMT</p> <div>In the <a mce_href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx" href="http://www.vovici.com/about/news-and-press/2009/CEIQ.aspx">CE IQ study</a>, Vovici and CGA asked organizations to rate themselves on implementation of 24 <a mce_href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices" href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices">customer-experience best practices</a> in six areas:</div><div><ol><li><b>Employee-Customer Engagement</b> – the extent to which customer input is shared with customer-facing employees and how that input is used to modify employee behavior</li><li><b>Co-Creation </b>- how organizations engage their customers in providing formal input on products and services and the extent to which they collaborate with customers in jointly creating products and services that best meet customer needs</li><li><b>Strategic Commitment</b> - the level of commitment an organization has to measuring the customer’s experience and listening to the <a mce_href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies" href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies">Voice of the Customer</a></li><li><b>Customer Relationship Management</b> – the extent to which organizations use CRM data in their feedback efforts </li><li><b>Feedback Management</b> – how organizations gather, measure and track customer feedback and the metrics associated with this feedback</li><li><b>Voice of the Customer</b> – the extent to which the organization gathers, uses and shares VOC data within the entire organization</li></ol></div><div>We then correlated each best practice against a loyalty index comprised of an average of these four customer-loyalty metrics:</div><div><ol><li><a mce_href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model">Willingness to repurchase</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Likelihood to recommend</a></li><li><a mce_href="http://blog.vovici.com/blog/bid/18715/Forrester-Loyalty-Metrics" href="http://blog.vovici.com/blog/bid/18715/Forrester-Loyalty-Metrics">Reluctance to switch</a></li><li>Increasing purchase amount</li></ol></div><div>For purposes of presenting the results, the <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">7-point</a> <a mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">fully-labeled</a> bipolar agreement scale was <a mce_href="http://blog.vovici.com/blog/bid/18584/Standardization-of-Scales-in-Survey-Analysis" href="http://blog.vovici.com/blog/bid/18584/Standardization-of-Scales-in-Survey-Analysis">mapped to a 0-10 scale</a>. Here are the results for each loyalty aspect:</div><div> </div><div> </div><div>Not surprisingly, the traditional loyalty measure, <i>willingness to repurchase</i>, had the highest correlation to the overall index. Nor is it surprising, given the recession, that <i>increasing purchase amount</i> had the lowest correlation to the index and the lowest level of strong agreement, with just 5.2% of respondents strongly agreeing.</div><div> </div><div>Here, then, are the correlations against the six practice areas:</div><div> </div><div> </div><div>Each practice area had a strong correlation to loyalty. Interestingly, customer relationship management, despite its maturity, remains difficult for most organizations to implement well: only 6.0% of respondents strongly agreed that their organization followed the four CRM/CE best practices. Surprisingly to us, the highest level agreement was for a <a mce_href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings" href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings">strategic, executive-level commitment to Customer Experience Management</a>, with strong agreement of 18.1%.</div><div> </div><div>In future posts, Brian Koma and I will look in detail at each CE practice area and how they distinguish organizations with highly loyal customers from those without.</div> </article> <article> <h1>EFM as the Hub of the Extended CRM Application Infrastructure</h1> <p>Roderick Morris — Wed, 19 Aug 2009 08:24:00 GMT</p> <div>William Band is the author of a new independent report published by Forrester, "<a mce_href="http://www.forrester.com/Research/Document/Excerpt/0,7211,53227,00.html " target="_new" href="http://www.forrester.com/Research/Document/Excerpt/0,7211,53227,00.html ">TechRadar™ For BP&A Professionals: The Extended CRM Application Ecosystem, Q3 2009</a>". The document provides a unique and much needed analysis of all the applications that <a mce_href="http://www.vovici.com/crm-connector/feedback-data-integration.aspx " href="http://www.vovici.com/crm-connector/feedback-data-integration.aspx ">integrate with Customer Relationship Management</a> systems. The paper sorts 19 application categories into five broad areas: Customer Understanding (the hub, containing <a mce_href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx " href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx ">EFM</a>, CBI and CDM), Customer Targeting, Customer Acquisition, Customer Retention and Customer Collaboration. </div><div> </div><div style="text-align: center;"> </div><div style="text-align: center;"><i>(Used by permission of Forrester.)</i> </div><div> </div><div>As <a mce_href="http://blog.vovici.com/blog/bid/17845/Father-of-EFM" href="http://blog.vovici.com/blog/bid/17845/Father-of-EFM">the pioneers of EFM</a>, and the ones who named the category, we at Vovici are glad to see that Forrester has recognized the central role that enterprise feedback management can play in extending the capabilities of traditional CRM. Clearly, customer understanding is critical to each of the other segments:</div><div><ul><li><b>Customer targeting</b> works best when organizations understand the types of customers that are most satisfied and most loyal, as discovered through EFM surveys. One could argue that customer targeting without customer surveying is like shooting in the dark.</li><li><b>Customer acquisition</b> works best when deploying sales force automation. And sales force automation works best when you have deep understanding of win/loss and rep productivity through transactional surveys. Acquisition is also supported when EFM is used to audit, measure and alert issues in Contract Life-cycle Management (CLM) systems.</li><li><b>Customer retention</b> is a key focus of <a mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx " href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx ">customer satisfaction</a> and customer loyalty studies. By using surveys with email alerts and triggers, EFM systems help organizations <a mce_href="http://blog.vovici.com/blog/bid/18667/Email-Trigger-a-Key-Aspect-of-EFM" href="http://blog.vovici.com/blog/bid/18667/Email-Trigger-a-Key-Aspect-of-EFM">intervene to improve customer loyalty</a>…not just measure it.</li><li><b>Customer collaboration</b> has been a particular area of focus for EFM vendors. Vovici integrates with third-party customer forums and customer community platforms, as well as offers our own online <a mce_href="http://www.vovici.com/survey-solutions/online-communities.aspx " href="http://www.vovici.com/survey-solutions/online-communities.aspx ">community platform</a> or <a mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community" href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community">MROC</a> specific to research.</li></ul></div><div>Back at the center of the hub, Vovici has long integrated with third-party BI systems, and—in our <a mce_href="http://www.vovici.com/survey-analytics/survey-data-analysis.aspx " href="http://www.vovici.com/survey-analytics/survey-data-analysis.aspx ">Feedback Intelligence</a> product—has a BI system that uniquely combines survey and operational data. There is no more powerful tool on the market for creating highly analytical and segmented reports on the fly.</div><div> </div><div>EFM has come a long way since Vovici introduced the first enterprise feedback management system (under that label) in 2004. Expect the pace to accelerate as EFM solutions are more tightly woven into the fabric of complementary CRM applications.</div> </article> <article> <h1>When DIY Surveys Become DYI Surveys</h1> <p>Jeffrey Henning — Tue, 18 Aug 2009 10:49:00 GMT</p> <div>Last month’s <i>Research</i> magazine had as its cover story “<a mce_href="http://www.research-live.com/features/world-of-temptation/4000459.article" target="_new" href="http://www.research-live.com/features/world-of-temptation/4000459.article">World of temptation</a>”, focusing on the negative ramifications of DIY (Do It Yourself) survey research:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">The problem for the future of research as a discipline isn’t so much that <a mce_href="http://blog.vovici.com/blog/?Tag=Survey%20Monkey" href="http://blog.vovici.com/blog/?Tag=Survey%20Monkey">SurveyMonkey</a> has made simple research more accessible, but that the people doing the research might not be very good at it. This could be, as the prostitute said looking at two lovers kissing, a great profession ruined by amateurs.</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><br>“It’s great on one hand because it shows how much passion there is to get information about customers, but if any monkey can do a survey, perhaps any monkey will. In some cases there will be <a mce_href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions" href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions">no control over design</a>, no control over the <a mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">sample</a>,” says [Brad Bortner, a Forrester Research analyst specializing in the market research industry].</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><div>This article has inspired follow-on editorials and blog posts (e.g., “<a mce_href="http://www.research-live.com/4000522.article" target="_new" href="http://www.research-live.com/4000522.article">Setting research free</a>”) and quite a bit of discussion. One comment in particular stood out to me: “DIY is great as a means for firms to engage with their customers and generate <a mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx" href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx">customer feedback</a>,” commented <a mce_href="http://www.gillwales.co.uk/" target="_new" href="http://www.gillwales.co.uk/">Gill Wales</a>, an independent market researcher. “But the purveyors of DIY solutions need to make clear the limitations of the DIY solution in unskilled hands.”</div><div> </div><div>As a fine purveyor of <a mce_href="http://www.vovici.com" href="http://www.vovici.com">survey software</a> since 1993, I’ll take Gill up on her challenge. If survey tool providers were to write a Surgeon General’s style warning about the dangers of Do-It-Yourself research, what should it say? My take on it:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 16px; line-height: normal; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px;"></span></font></blockquote><div> </div><div>If you were the <i>Survey </i>General, what would you put on the label? How would you keep Do-It-Yourself surveys from becoming Do-Yourself-In surveys?</div><div> </div><div><i>See also:</i></div><div><ul><li><a mce_href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department" href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department">Rethinking the Role of the Market Research Department</a> </li><li><a mce_href="http://blog.vovici.com/blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise" href="http://blog.vovici.com/blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise">The Roles of Assisted Survey Author and Survey Reviewer</a></li></ul></div> </article> <article> <h1>TNS Customer Loyalty Index</h1> <p>Jeffrey Henning — Mon, 17 Aug 2009 12:32:00 GMT</p> <div><a mce_href="http://blog.vovici.com/blog/bid/17914/TNS-Taylor-Nelson-Sofres-Partners-with-Vovici" href="http://blog.vovici.com/blog/bid/17914/TNS-Taylor-Nelson-Sofres-Partners-with-Vovici">TNS, a Vovici partner</a>, has a CLI (Customer Loyalty Index) it uses in its business-to-business satisfaction research that measures loyalty across four questions: </div><div><ol><li>Overall satisfaction (<a mce_href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question" href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question">CSAT</a>)</li><li><a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Likelihood to recommend</a></li><li>Likelihood to repurchase</li><li>The competitive advantage your company provides.</li></ol></div><div>That last question is unique to the index and reflects the focus on B2B sales.</div><div><br></div><div>Here is a generic version of the TNS CLI template:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">We'd like to ask you about your overall satisfaction with [organization/brand]. Considering everything you know about this company, its relationship with you and its products, services, and/or support would you say you are:<br><ol><li>Very dissatisfied</li><li>Somewhat dissatisfied</li><li>Somewhat satisfied</li><li>Very satisfied</li></ol>Based on your experience, how likely would you be to recommend [organization/brand] to a friend or colleague looking for [product/service category]?<br><ol><li>Definitely would not</li><li>Probably would not</li><li>Might or might not</li><li>Probably would</li><li>Definitely would</li></ol>For your next similar purchase of [product/service category], how likely would you be to buy from [organization/brand] again? <br><ol><li>Definitely would not</li><li>Probably would not</li><li>Might or might not</li><li>Probably would</li><li>Definitely would</li></ol>In general, how would you rate the competitive advantage provided to your company by using [organization/brand] rather than using any other company that provides similar solutions, services, products, support, etc.? <br><ol><li>No advantage at all</li><li>Only a slight advantage</li><li>Some advantage</li><li>Big advantage</li><li>Vital advantage</li></ol></blockquote><div>Reminder: If you adapt this template for your own use, <a mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">don't display the numbers in your rating scales</a>.</div><div> </div><div>The TNS CLI is an average that can range from 0% to 100%. It is the average of % Satisfied (scores 3 or 4 on the 4-point scale), the % Likely to Recommend (scores 4 or 5 on the 5-point scale), the % Likely to Repurchase (scores 4 or 5) and the % Competitive Advantage (score 4 of 5, “Big advantage” or “Vital advantage”).</div><div><br></div><div>TNS reports an average of 85.3% to <a mce_href="https://partner.microsoft.com/UK/40018120" target="_new" href="https://partner.microsoft.com/UK/40018120">its Loyalty Index across all Microsoft partners</a>, which showcases the fact that many partners have small, fiercely loyal customer bases.</div><div><br></div><div>One of the keys to note about the TNS CLI is that it, like many loyalty indices, integrates satisfaction directly into the measure.</div><div> </div><div><b><i>Update (9/3):</i></b> The <a mce_href="http://blog.vovici.com/blog/bid/21293/Champion-Model" href="http://blog.vovici.com/blog/bid/21293/Champion-Model">Champion Model</a> is a TNS customer segmentation built on top of the TNS CLI.</div> </article> <article> <h1>Core Values as a Foundation of Employee Satisfaction</h1> <p>Jeffrey Henning — Fri, 14 Aug 2009 08:55:00 GMT</p> <a target="_" href="http://en.wikipedia.org/wiki/Perseus_(constellation)"></a><div>Sixteen years ago today my only freelance customer and I decided that we should found a company to sell the <a mce_href="http://blog.vovici.com/blog/bid/17889/Firing-Up-Your-Laptop-with-Our-New-CAPI-Computer-Assisted-Personal-Interviewing-Solution" href="http://blog.vovici.com/blog/bid/17889/Firing-Up-Your-Laptop-with-Our-New-CAPI-Computer-Assisted-Personal-Interviewing-Solution">CAPI</a> software application that I had developed for him. Looking up into the sky, where the annual meteor shower was underway, Rich suggested we name the company Perseides Software. </div><div> </div><div>A day or two later, after both of us independently realized that no one we bounced the name off of knew how to spell Perseides, I suggested we call it Perseus instead, after the constellation in which the meteor showers appear. As Boston-based entrepreneurs, we loved Lotus Development Corp., and Rich suggested we name the company Perseus Development Corp.</div><div> </div><div>Despite its astronomical eponym, Perseus did not have stellar growth initially. Rich and I had made the classic mistake of thinking there was a large market for a product that <i>we </i>needed. In 1996, we extended the product (called Perseus IntelliWriter) to support web surveys, and then in 1997 we launched one of the first two applications dedicated to <a mce_href="http://www.vovici.com" href="http://www.vovici.com">web surveys</a>, Perseus SurveySolutions for the Web. We grew dramatically from then on, making the Inc. 500 list twice and the Deloitte & Touche New England Fast 50 list four times. </div><div> </div><div>Since Perseus grew at a time of record low unemployment, we often had difficulty hiring qualified staff. Some staff didn’t work out, as they would have done better in a large, stable business with more formal processes than we had. One of our clients automated an applicant assessment process to identify which types of jobs their applicants were most suited for, and that inspired me to come up with the Perseus core values. I agonized over every firing that I had done over the years, and I thought about each of them to come up with the following.</div><div> </div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">These are the values that have enabled us to build the leading company in our industry; these are the values that will fuel our worldwide growth in the coming years; and these are the values that will drive your career here:<br><ul><li><b>Professionalism</b> – Demonstrating professional methods, character and standards. Treating prospects, clients and co-workers generously and charitably at all times, but especially in the face of adversity.</li><li><b>Enthusiasm</b> – Showing excitement, optimism and passion for your work.</li><li><b>Resourcefulness</b> – Acting effectively and imaginatively to produce great results from scarce resources.</li><li><b>Self-directedness</b> – Working independently and autonomously to achieve the goals set by management.</li><li><b>Ethics</b> – Acting in accordance with the accepted principles of right and wrong that govern the conduct of our profession.</li><li><b>Unselfishness</b> – Putting others before yourself, giving your time and effort for prospects, clients and co-workers. Showing cooperative effort as the member of a group to achieve a common goal.</li><li><b>Strategic-mindedness</b> – Suggesting and implementing long-term improvements springing from a sequence of short-term tasks.</li></ul></blockquote><div>OK, I probably worked too hard to have the core values spell <b>Perseus</b>! But it certainly made it easier for me to remember them when interviewing applicants.</div><div> </div><div>Too often <a mce_href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx " href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx ">employee satisfaction</a> research overlooks the fact that to have truly satisfied employees you must hire the right types of people in the first place. The Perseus core values became a way for us to do this with much greater consistently. As you think about employee loyalty for your organization, consider profiling respondents by assessment questions, where they indicate where they fall on a continuum for each value (e.g., for self-directedness, from a continuum of "prefers close direction" to "prefers independence"). </div><div> </div><div>My partner and I sold our company to Austin Ventures in 2006, and AV purchased WebSurveyor and merged it with Perseus to form Vovici. (For the record, Vovici is neither a constellation nor a meteor shower!) While the Perseus name may no longer live on in the market, as a mnemonic for our coworker values it lives on each time I interview a new applicant for Vovici.</div> </article> <article> <h1>The Seven Habits of Highly Successful Surveys</h1> <p>Jeffrey Henning — Thu, 13 Aug 2009 16:43:00 GMT</p> <div>One of our more popular webinars is <i>The Seven Habits of Highly Successful Surveys</i>. We’ve turned this into a short white paper, and it’s also the basis for my free 73-page ebook, <a mce_href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx" href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx"><i>Survey Software Success</i></a>. Here’s the Cliff Notes version of the seven habits of successful survey researchers (with apologies to <a mce_href="http://www.franklincovey.com/" target="_new" href="http://www.franklincovey.com/">Dr. Stephen Covey</a>):</div><div><ol><li><a mce_href="http://blog.vovici.com/blog/bid/18186/Good-Surveys-start-with-Good-Goals " href="http://blog.vovici.com/blog/bid/18186/Good-Surveys-start-with-Good-Goals "><b>Focus on Your Goal</b></a>: “Don't let your survey project go astray from the start: be certain to focus on a specific goal. Be precise about what information you need to gather and what you plan on doing with it.”</li><li><span class="Apple-style-span" style="text-decoration: underline; "><b>Survey the Right Group of People</b></span>: Use <a mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sampling</a> with these <a mce_href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys" href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys">recommendations for sample size</a> if you have thousands or more potential respondents; if you don’t have that many, then <a mce_href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">attempt a census</a> instead.</li><li><b><span class="Apple-style-span" style="text-decoration: underline;">Craft Your Invitation Carefully</span></b>: You have a lot of <a mce_href="http://blog.vovici.com/blog/bid/18258/Hurdles-in-Race-to-Turn-Recipients-into-Respondents" href="http://blog.vovici.com/blog/bid/18258/Hurdles-in-Race-to-Turn-Recipients-into-Respondents">hurdles to jump in the race to turn recipients into respondents</a>; make sure your <a mce_href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers" href="http://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers">email list is representative</a>, maintain and use the unsubscribe list, get past spam filters, craft compelling subject lines and induce recipients to click that survey link and take the survey.</li><li><a mce_href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically" href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically"><b>Order Questions Logically</b></a>: Put screener questions first, then open-ended questions, general questions, specific questions, demographics or <a mce_href="http://blog.vovici.com/blog/bid/18047/Firmographic-Definition-Template " href="http://blog.vovici.com/blog/bid/18047/Firmographic-Definition-Template ">firmographics</a> and finally any follow-up questions. (Up-front open-ended questions are not appropriate to every survey but are important for needs analysis research, to prevent bias from later <a mce_href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">closed-end questions</a>.)</li><li><a mce_href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions" href="http://blog.vovici.com/blog/bid/18230/Writing-Objective-Survey-Questions"><b>Write Objective Questions</b></a>: “The saying Garbage In, Gospel Out reflects our willingness to believe computer output, even if it was generated from bad input. Survey researchers are no more immune to this tendency than computer scientists, as poorly worded questions can lead to suspect results and erroneous conclusions.”</li><li><a mce_href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length"><b>Shorten the Survey</b></a>: It’s rare that I work on a research project where the questionnaire doesn’t need pruned before being published. Check out <a mce_href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">this post</a> for six detailed suggestions on how to trim that survey.</li><li><a mce_href="http://blog.vovici.com/blog/bid/18174/Closing-the-Feedback-Loop-Sharing-Results-with-Online-Community-Members-Respondents" href="http://blog.vovici.com/blog/bid/18174/Closing-the-Feedback-Loop-Sharing-Results-with-Online-Community-Members-Respondents"><b>Close the Feedback Loop</b></a>: “Your respondents complete the survey because they value their relationship with you, and they want to see you improve. Implicit in the fact that you sent them a survey is your intention to learn, adapt and change based on the results.”</li></ol></div><div>OK, so now you don’t have to watch that particular webinar. So go watch one of our other complimentary <a mce_href="http://www.vovici.com/about/webinars.aspx" href="http://www.vovici.com/about/webinars.aspx">survey webinars</a> instead!</div> </article> <article> <h1>Transactional Surveys that Build Customer Loyalty</h1> <p>Jeffrey Henning — Thu, 13 Aug 2009 08:02:00 GMT</p> <div>One of the exemplary aspects of the <a mce_href="http://blog.vovici.com/blog/bid/18410/Case-Study-Domino-s-Pizza-Transactional-Survey" href="http://blog.vovici.com/blog/bid/18410/Case-Study-Domino-s-Pizza-Transactional-Survey">Domino’s Pizza transactional survey</a> is that the questionnaire is updated in real time to display the name of your cook and the name of your delivery driver. This clearly communicates to the respondent that they are rating employees and that this feedback will most likely make its way back to those employees. </div><div>How valuable an approach is this? As Brian Koma reported in his summary of our <a mce_href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings " href="http://blog.vovici.com/blog/bid/20360/Customer-Experience-Study-Findings ">Customer Experience study findings</a>, one of the CEM best practices with the highest correlation to customer loyalty is sharing with employees the ratings of them done by customers.</div><div> </div><div>So expanding your existing <a mce_href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey" href="http://blog.vovici.com/blog/bid/18874/Follow-up-Survey-Transaction-Survey">transactional surveys</a> to close the feedback loop in this way can help you improve customer loyalty over the long term. That said, it does require changing much about your survey process. Many transactional surveys are <i>pulse surveys</i>, taking the overall pulse of customer satisfaction with your service, rather than providing detailed diagnostics. Here are some of the changes you will need to make:</div><div><ul><li><b>Larger Sample</b> - A <a mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sample</a> of, say, 1 out of 5 customers is perfectly appropriate for the traditional pulse survey and provides an accurate measurement of overall service levels while not inviting every customer. Unfortunately, if you want to be able to provide cross-tabulations by employee, you will want at least 30 responses per employee. As a result, you will need to increase the incidence rate dramatically, possibly even moving to an <a mce_href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" href="http://blog.vovici.com/blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">attempted census</a>, contacting every single customer.</li><li><b>New Personalized Reports for Employees</b> - Often, when moving to transactional surveys broken out by employee, the desire is for fresh data about an employee daily. That is often misleading, and we encourage weekly scorecards instead. One day’s results are simply too small to be meaningful. In fact, with small cell sizes per employee, it may even be necessary to provide a rolling multi-week average of an employee’s ratings rather than an average for just that week. </li><li><b>New Personalized Reports for Managers</b> – A daily summary of results for all of a manager’s direct reports, treated as a group, provides trend tracking that can act as an early warning system, alerting a manager to important issues. <a mce_href="http://blog.vovici.com/blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day" href="http://blog.vovici.com/blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day">Hierarchical reports</a> can even be rolled up to provide higher levels of aggregation, by contact center or division, for instance.</li><li><b>New Executive Reports</b> – Another useful way to analyze the data is by reviewing all the <a mce_href="http://blog.vovici.com/blog/bid/18095/Open-Ended-Questions" href="http://blog.vovici.com/blog/bid/18095/Open-Ended-Questions">open-ended comments</a> for the highest-rated employees and contrasting them with the comments for the lowest-rated employees. This can then be used to develop training materials to educate low performers about the practices that have proven successful with the high performers.</li></ul></div><div>While the data gathered from an employee-segmented transactional survey is most valuable as a way of coaching and mentoring employees, for some staff it will be necessary to use this data to justify terminating them. With the employee-employer relationship being the most regulated relationship in America, you will want to make sure that you are collecting and analyzing this data in an objective and appropriate manner.<br></div><div> </div><div>When first rolling out such a system, it is far better to concentrate on the opportunities for improving employee engagement with customers, providing valuable feedback to employees to help them serve the customer better. That’s the big opportunity to build customer loyalty.</div> </article> <article> <h1>The Economics of Customer Retention</h1> <p>Brian Koma — Wed, 12 Aug 2009 14:11:00 GMT</p> <div>Customer loyalty is essential to any organization seeking to maintain existing revenue or trying to create revenue growth. Holding onto existing customers can dramatically reduce the amount of time, effort and money required to grow the organization. </div><div> </div><div>Customer retention rates can be low – or as high as 100%. Lower customer retention/renewal rates are problematic because they are expensive and inefficient to counteract, requiring a very high investment in sales and marketing programs to drive new customer acquisition. Higher customer retention rates can dramatically lower these costs and can enable a company’s growth to far outpace its rivals at much lower cost. Winning new customers while losing a significant share of existing customers is like filling a bathtub with the drain open.</div><div> </div><div>The examples below show the stark differences between a 74% customer retention rate and a 90% customer retention rate. </div><div> </div><div>At a 74% customer retention rate, a $30 million business will lose $7.8 million in revenue year-over-year and will need to sell almost $17 million in new business to reach $40 million in sales.</div><div> </div><div> </div><div>At a 90% customer retention rate, a business will reduce its year-over-year revenue loss to $3 million and will only need to sell an additional $12 million to reach $40 million in sales. </div><div> </div><div> </div><div> </div><div>As these illustrations show, loyal customers can make a dramatic difference in an organization’s overall financial health and dramatically reduce the cost of growth, even in a challenging economy. </div><div> </div><div>Compound your retention rate over the years and you have <a mce_href="http://www.crm2day.com/content/t6_librarynews_1.php?id=EpFEAEyFpuIBCdXUbD " target="_new" href="http://www.crm2day.com/content/t6_librarynews_1.php?id=EpFEAEyFpuIBCdXUbD ">a dramatic impact on your corporation’s profits</a>.</div><div> </div><div>All of this makes it more important than ever to implement <a mce_href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices" href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices">CEM (Customer Experience Management) best practices</a> that drive customer loyalty.</div> </article> <article> <h1>Employee Loyalty Benchmark from Walker Information</h1> <p>Jeffrey Henning — Wed, 12 Aug 2009 08:49:00 GMT</p> Besides the <a mce_href="http://blog.vovici.com/blog/bid/19085/Employee-Net-Promoter-Score" href="http://blog.vovici.com/blog/bid/19085/Employee-Net-Promoter-Score">Employee NPS</a> and the <a mce_href="http://blog.vovici.com/blog/bid/18535/Employee-Engagement-Survey-The-Gallup-Q12" href="http://blog.vovici.com/blog/bid/18535/Employee-Engagement-Survey-The-Gallup-Q12">Gallup Q12</a>, another employee-satisfaction benchmark is Walker Loyalty. Walker Information, a Vovici partner, fields its own benchmark consisting of 80 questions across eight sections:<div><ol><li>Attitudes toward the Organization</li><li>Work-Related Behaviors</li><li>Questions about the Organization and Your Work</li><li>More Information about Work Factors </li><li>Other General Opinions about the Organization </li><li>Rating the Influence of Work Factors </li><li>Opinions about the Integrity of the Organization</li><li>Any Comments in Your Own Words?</li></ol></div><div>The results of key questions are used to segment employees by attitude and behavior:</div><div><ul><li><b>Truly Loyal</b> – positive attitude, positive behavior – These employees plan to remain employed and want to work for your organization.</li><li><b>Accessible</b> – positive attitude, negative behavior – Accessible employees want to remain employed but may not be able to, because of outside circumstances or better opportunities elsewhere.</li><li><b>Trapped</b> – negative attitude, positive behavior – Trapped employees plan to remain employed, but would prefer to work elsewhere.</li><li><b>High Risk</b> – negative attitude, negative behavior – High Risk employees do not plan to remain employed and no longer want to be employed by your organization.</li></ul></div><div> </div><div> </div><div>In 2007, Walker reported that 34% of U.S. employees were Truly Loyal, 7% were Accessible, 23% were Trapped and 36% were High Risk. </div><div> </div><div>Besides its segmentation of employees by attitude and behavior, Walker Loyalty also segments attributes into strengths and areas for improvement: Top Priorities (high performance gap, high influence on employee engagement), Lesser Priorities (high performance gap, low influence on engagement), Leverageable Strengths (low gap, high influence) and Other Strengths (low gap, low influence). This quadrant analysis makes it easy for organizations to determine what attributes they should focus on to improve employee engagement. </div><div> </div><div>The following criticisms can be made about the Walker Information employee benchmark:</div><div><ol><li>While the attitudinal index used in the segmentation is derived from three questions, giving it greater accuracy and stability, the behavioral measure is based on the answer to one question.</li><li>The survey is only used for benchmarking U.S. organizations.</li><li>Little research has been done outside Walker to independently attest to the predictive validity of its loyalty segmentation.</li></ol></div><div><i>See also:</i></div><div><ul><li><a mce_href="http://blog.vovici.com/blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession" href="http://blog.vovici.com/blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession">Five Reasons You Must Measure Employee Loyalty During a Recession</a></li><li><a mce_href="http://blog.vovici.com/Blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty" href="http://blog.vovici.com/Blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty">Correlation between Employee Loyalty & Customer Loyalty</a></li><li><a mce_href="http://blog.vovici.com/Blog/bid/18084/The-True-Cost-of-Low-Employee-Loyalty/" href="http://blog.vovici.com/Blog/bid/18084/The-True-Cost-of-Low-Employee-Loyalty/">The True Cost of Low Employee Loyalty</a></li></ul></div> </article> <article> <h1>Survey Scales Listing Negative Choices First</h1> <p>Jeffrey Henning — Tue, 11 Aug 2009 18:39:00 GMT</p> Kristi—one of our <a mce_href="http://www.vovici.com/about/webinars.aspx" href="http://www.vovici.com/about/webinars.aspx">survey webinar</a> attendees—asks, “Why on the satisfaction scale do you begin with the negative and go towards positive,e.g. ‘Falls Short’ to ‘Exceeds Expectations’? What's the reasoning? Is this standard?”<p class="MsoNormal">Great question! Respondents are busy and often <a mce_href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">satisfice in surveys</a>, picking an approximate answer rather than the most appropriate answer. Typically, due to <a mce_href="http://blog.vovici.com/blog/bid/18257/Order-Effects-Early-Choices-in-Long-Choice-Lists-Are-Selected-More-Often-than-Later-Choices" href="http://blog.vovici.com/blog/bid/18257/Order-Effects-Early-Choices-in-Long-Choice-Lists-Are-Selected-More-Often-than-Later-Choices">recency effects</a>, listing the negative choices first makes it slightly more likely that the negative choices will be selected by respondents.</p><p class="MsoNormal">Too often respondents take the easy way out in a survey, rating you highly. You want to make sure that rating is earned. (One of our partners takes an even more extreme approach with <a mce_href="http://blog.vovici.com/Blog/bid/18112/Less-Really-is-More-When-it-Comes-to-Response-Scales" href="http://blog.vovici.com/Blog/bid/18112/Less-Really-is-More-When-it-Comes-to-Response-Scales">three-item response scales</a> designed to get lower averages than traditional five-point scales.)</p><p class="MsoNormal">As <a mce_href="http://blog.vovici.com/Blog/bid/17878/The-Folk-Apostle-Model" href="http://blog.vovici.com/Blog/bid/17878/The-Folk-Apostle-Model">Sasser and Jones showed for Xerox</a>, exceptional answers really do signify a different behavior:<span style="mso-spacerun:yes"> </span>respondents who answered a 5 on a 5-point scale were six times more likely to repurchase in the next 18 months than respondents who answered a 3 or 4. You want to come by those exceptional answers because they were earned.</p><p class="MsoNormal">Finally, and tactically, most <a mce_href="http://www.vovici.com" href="http://www.vovici.com">survey software</a> applications assign numbers behind the labels, in numeric order starting with the first item. Placing the worst item first results in a 1 being the worst rating, which is better for the type of analysis that I like to do.</p><p class="MsoNormal">While putting the most negative response item first is standard for me, I don’t believe it is a standard used by other survey authors.</p> </article> <article> <h1>'No Opinion' as a Question Choice</h1> <p>Jeffrey Henning — Fri, 07 Aug 2009 14:31:00 GMT</p> <DIV>When writing <A href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" mce_href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">closed-end questions</A> should you include a choice for “Don’t know”, “Not applicable” or “No opinion”? The fear is that including this as an option will give respondents an easy way out (e.g., <A href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" mce_href="http://blog.vovici.com/blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">survey satisficing</A>) rather than actually thinking through their best answer to the question. </DIV> <DIV> </DIV> <DIV>And in fact, this is just what Jon Krosnick and others found in their research, “<A href="http://poq.oxfordjournals.org/cgi/content/abstract/66/3/371" target=_new mce_href="http://poq.oxfordjournals.org/cgi/content/abstract/66/3/371">The Impact of "No Opinion" Response Options on Data Quality</A>” [bullets added]: </DIV> <DIV><A href="http://poq.oxfordjournals.org/cgi/content/abstract/66/3/371" mce_href="http://poq.oxfordjournals.org/cgi/content/abstract/66/3/371"></A></DIV> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>According to many seasoned survey researchers, offering a no-opinion option should reduce the pressure to give substantive responses felt by respondents who have no true opinions. By contrast, the survey satisficing perspective suggests that no-opinion options may discourage some respondents from doing the cognitive work necessary to report the true opinions they do have. We address these arguments using data from nine experiments carried out in three household surveys. Attraction to no-opinion options was found to be greatest:<BR></BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> <OL> <LI>among respondents lowest in cognitive skills (as measured by educational attainment),</LI> <LI>among respondents answering secretly instead of orally, </LI> <LI>for questions asked later in a survey, </LI> <LI>and among respondents who devoted little effort to the reporting process. </LI></OL></BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>The quality of attitude reports obtained (as measured by over-time consistency and responsiveness to a question manipulation) was not compromised by the omission of no-opinion options. These results suggest that inclusion of no-opinion options in attitude measures may not enhance data quality and instead may preclude measurement of some meaningful opinions.</BLOCKQUOTE> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> </BLOCKQUOTE> <DIV>Use of no-opinion responses is greater for respondents “answering secretly instead of orally” - i.e., for respondents doing self-administered surveys such as paper, web and kiosk surveys, rather than responding to telephone or face-to-face surveys. The reason for this difference is that every respondent sees the no-opinion choice on a written survey, but in an oral survey that response is typically not read aloud to a respondent but is kept in reserve, checked off only if the respondent brings it up.</DIV> <DIV> </DIV> <DIV>One way to approximate this in an online survey is to not include such a response in the choice list for a question, if an answer is not required. The respondent therefore doesn’t see the “Don’t know” option but, upon consideration of the available choices, can simply skip answering the question altogether. Accordingly, I prefer to only use no-opinion responses in choice lists only if the question is required, and only if the required question may be hard to answer for respondents: for instance, when asking about specific details about a past transaction or when asking for details about the respondent’s organization that they simply might not know.<BR><BR></DIV> <DIV>When analyzing no-opinion responses it is often handy to omit such responses from pie charts and frequency percentages. Survey software applications may let you assign a code to a choice (e.g., in Vovici v4, you can assign a choice to a precoded meaning “Not applicable”, “Don’t know” or “Refused”) with such coded values then omitted from pie charts and the percentage column of frequency tables. Rather then say, “70% said yes, 20% said no and 10% didn’t know” with this option you can say “78% who knew said yes, 22% said no.”</DIV> <DIV> </DIV> <DIV>I hope this gives you enough information to now have an opinion on the use of no-opinion responses! </DIV> </article> <article> <h1>Paper Survey to Web Survey</h1> <p>Jeffrey Henning — Thu, 06 Aug 2009 19:04:00 GMT</p> <div>Reading about the <a mce_href="http://www.google.com/hostednews/ap/article/ALeqM5irzC2kcTNhcoBX5lcC52Yw6BigmwD99SRM884" target="_new" href="http://www.google.com/hostednews/ap/article/ALeqM5irzC2kcTNhcoBX5lcC52Yw6BigmwD99SRM884">U.S. Post Office's financial shortfall</a> this week reminded me of the following question, which highlights the small role we've played in decreasing the volume of U.S. mail. Roberta, a recent attendee to one of our <a mce_href="http://www.vovici.com/about/webinars.aspx" href="http://www.vovici.com/about/webinars.aspx">research webinars</a>, posed the following question:</div><div> </div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Our customer base strongly skews to an older, retired (and presumably less technically-savvy) demographic. Accordingly, we have continued to use traditional postal surveys to measure satisfaction in this conservative customer base.</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">We are eager to switch to an online survey methodology, but have two related concerns: (1) comparability of historical data (data collected in postal survey vs. online) and (2) switching methodology may effectively 'disenfranchise' some older customers who may not have email access.</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">From a technical standpoint, how serious are these concerns and what, if anything, should we do to address these concerns -- either in terms of survey methdology or at the analytical stage?</blockquote><div><br></div><div>If all of your customers are older, then you have less to worry about than if your customers span a wide range of ages. If you have a mix of under-30 customers and over-60 customers, for instance, than switching to the web would change the proportions that responded, which would have a dramatic effect on the results; for such a case, I would advise weighting the results by age segment to reflect the age distribution of customers (see my <a mce_href="http://blog.vovici.com/blog/bid/18077/Age-Question" href="http://blog.vovici.com/blog/bid/18077/Age-Question">Age Question</a> blog post for how to ask respondents their age, if you don't already have it).</div><div><br></div><div>Our most conservative client migrated data collection methodologies over the course of 12 months, ratcheting up the percent of surveys using the new methodology every month until the end of the first year, at which point all surveys were done using the new method. Each month along the way, they compared the answers of the two groups to see where there were differences and to understand what the causes of the differences were. For analysis, they reported the overall results and the results by each collection mode.</div><div><br></div><div>If you did something similar, at the end of the year you could do year-over-year comparisons using only the results collected the year before using the new method.</div><div><br></div><div>As to disenfranchising customers, you could position the move to the Web as being done for environmental reasons, to minimize your firm's use of paper and the fuel necessary to ship the mail back and forth. (Hat-tip to Gartner for teaching me how to sell <a mce_href="http://blog.vovici.com/blog/bid/17901/Selling-Online-Surveys-as-Green" href="http://blog.vovici.com/blog/bid/17901/Selling-Online-Surveys-as-Green">online surveys as 'green'</a>.) You could say that all people would be moved to the new method unless they specifically returned a postcard opting to still receive the surveys by mail. Such a transition to the web is rarely needed for more than a year.</div><div> </div><div>Moving from paper surveys to web surveys will:</div><div><ul><li>Lower your costs by eliminating the need for printing and mailing surveys and then doing data entry on the completed surveys</li><li>Dramatically speed up reporting by eliminating the current lag where surveys are in transit to you and then queued up for data entry</li><li>Enable you to set up <a mce_href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">survey alerts and trigger emails</a> to immediately take action when a customer is unhappy.</li></ul><div>Oh, and it is definitely better for the environment! (If not the Post Office!)</div></div> </article> <article> <h1>Customer Experience Study Findings</h1> <p>Brian Koma — Thu, 06 Aug 2009 14:14:00 GMT</p> <div>In the most challenging economic environment in generations, organizations are more focused than ever on retaining customers and maintaining existing revenue. Improving customer loyalty has replaced growth as the focal point for many organizations, and many businesses have put significant resources into implementing Customer Experience Management (CEM) and Voice of the Customer (VOC) programs, or are in the process of doing so. </div><div> </div><div>But what constitutes success in this arena? Of all the programs that organizations have implemented, which are most effective at driving customer loyalty? The <a mce_href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices" href="http://blog.vovici.com/blog/bid/19709/Customer-Experience-Management-Best-Practices">Customer Experience IQ (CEIQ) study</a>, conducted jointly by Vovici and CGA, pinpoints the most effective elements of customer experience programs and answers these important questions:</div><div><ul><li>Are Customer Experience Management and Voice of the Customer programs effective at creating customer loyalty?</li><li>What are the most effective methods for creating customer loyalty in the first global economic contraction since 1945?</li><li>What specific steps can organizations take to create loyal customers based on best practices from around the globe?</li></ul></div><div>Derived from quantitative data obtained through an online survey conducted among more than 200 companies around the globe, the Customer Experience IQ (CEIQ) report identifies the six secrets of highly effective customer experience programs that every company can use to create true customer loyalty. Whether your organization has already implemented a Customer Experience Management (CEM) or Voice of the Customer (VOC) program or is just thinking about it, the elements uncovered in the CEIQ study can help you dramatically enhance the effectiveness of your customer loyalty programs.</div><div> </div><div><b>Key Findings</b></div><div><ul><li>The downturn has caused an upturn in interest in measuring the customer experience. In 2008, just under half (47%) of survey respondents said that CEM was important, while in 2009, almost two of out every three organizations (63%) say CEM is important to their business.</li><li>There’s a disconnect between what organizations say and what they do when it comes to CEM/VOC programs. Almost 75% of respondents say that they have a CEM/VOC strategy, yet less than half of respondents say they have formal programs and compensation systems in place to implement the strategy. </li><li>Only those organizations who have implemented formal CEM programs are able to translate those activities into higher customer loyalty. It’s not enough to merely have a strategy – it must be articulated throughout the organization and executed in a coordinated fashion.</li><li>The most effective CEM programs include a standard set of metrics by which the customer experience is measured. Organizations with the highest customer loyalty use one of the many standardized measures (<a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">NPS</a>, the <a mce_href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model">Apostle model</a>) or a set of proprietary metrics, in order to apply consistent measures across the organization.</li><li>Organizations with the highest customer loyalty integrate what they know about customers from Customer Relationship Management (CRM) systems into their feedback efforts. CRM integration enables personalization of surveys and invitations and reduces survey fatigue and helps to maintain high response rates.</li><li>Organizations with the highest customer loyalty share CEM and VOC feedback across organizational boundaries throughout the organization. Feedback is not held in silos; rather it is shared freely throughout the organization and viewed as a strategic corporate asset. </li><li>Organizations that have the highest customer loyalty constantly obtain feedback on the performance of customer-facing employees and then share that feedback with those team members. By doing this, they are able to provide constructive suggestions on how the individuals can improve their performance</li><li>Finally, organizations with the highest customer loyalty engage in a co-creative process with their customers. They have a formal process for obtaining customer input and then incorporate those ideas into strategic initiatives and tactical process improvements.</li></ul></div><div><b>Methodology</b></div><div> </div><div>Vovici and CGA conducted an online <a mce_href="http://www.vovici.com" href="http://www.vovici.com">web survey</a> of 208 organizations between May 20th and June 2, 2009. Respondents include a wide variety of organizations, including small (1-19 employees), medium (20-499 employees) and large companies (500+ employees). Over 62% of respondent organizations are located in North America and United Kingdom, with the balance participating from Asia/Pacific, Africa and South America. Respondents hail from a diverse set of industries, including Marketing, Services, Financial Services, Healthcare, Manufacturing, Technology, Retail and a variety of other segments. </div><div> </div><div>Participants were decision makers who have a comprehensive view of their business operations. Over 70% of respondents hold senior-level titles, including Manager, Director, Vice President or President/CEO/MD. Job responsibilities ranged from marketing, customer service, operations, research and development, administration and market research.</div><div> </div><div>Respondents were provided with an incentive of an early copy of the results of the survey in consideration of their participation. No monetary incentive was provided to any participant.</div><div> </div><div>The results are not statistically representative of any market but provide strong qualitative indication as to the effectiveness of different customer-experience best practices.</div> </article> <article> <h1>Apple Does "No Market Research", So You Don't Have To Either</h1> <p>Jeffrey Henning — Tue, 04 Aug 2009 13:31:00 GMT</p> <a href="http://en.wikipedia.org/wiki/File:Steve_Jobs.jpg"></a><div>Last year, in a comment made newly popular by <a mce_href="http://bokardo.com/archives/steve-jobs-on-why-apple-doesnt-do-market-research/" target="_new" href="http://bokardo.com/archives/steve-jobs-on-why-apple-doesnt-do-market-research/">a post on Bokardo</a>,<a mce_href="http://money.cnn.com/galleries/2008/fortune/0803/gallery.jobsqna.fortune/3.html " target="_new" href="http://money.cnn.com/galleries/2008/fortune/0803/gallery.jobsqna.fortune/3.html ">Steve Jobs told Fortune magazine</a> that Apple doesn’t do MR:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">We do no market research. We don't hire consultants. The only consultants I've ever hired in my 10 years is one firm to analyze Gateway's retail strategy so I would not make some of the same mistakes they made [when launching Apple's retail stores]. But we never hire consultants, per se. We just want to make great products.</blockquote><div> </div><div>Apple staff primarily want to make great products <i>for themselves:</i></div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">We did iTunes because we all love music. We made what we thought was the best jukebox in iTunes. Then we all wanted to carry our whole music libraries around with us. The team worked really hard. And the reason that they worked so hard is because we all wanted one. You know? I mean, the first few hundred customers were us.<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">It's not about pop culture, and it's not about fooling people, and it's not about convincing people that they want something they don't. We figure out what we want. And I think we're pretty good at having the right discipline to think through whether a lot of other people are going to want it, too. That's what we get paid to do.<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">So you can't go out and ask people, you know, what the next big [thing]. There's a great quote by Henry Ford, right? He said, ‘If I'd have asked my customers what they wanted, they would have told me "A faster horse."’</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><div>When Jobs says Apple does “no market research”, I believe he means that Apple no longer does market research for new product development. After all, <a mce_href="http://www.davidalison.com/2008/06/apple-does-it-right-when-it-comes-to.html " target="_new" href="http://www.davidalison.com/2008/06/apple-does-it-right-when-it-comes-to.html ">Apple surveys extensively</a>, in order to determine customer satisfaction after transactions (purchases, repairs, support calls).</div><div> </div><div>Apple used to do MR for product development. I recall, after John Sculley had fired Steve Jobs, seeing some of the market research behind the Apple Newton. I was an analyst in the mobile computing space and was briefed on the Newton prior to its launch. It was well researched -- targeting consumers who did not yet have computers and who felt that pocket organizers were not enough, Apple put an impressive amount of research into discovering their pain points. </div><div> </div><a href="http://www.doonesbury.com/strip/doonsignup.html"></a><div>Of course, Sculley’s Apple got the market for the <i>personal digital assistant </i>(a term John Sculley coined) wrong: selling tech to people who are technology laggards is practically quixotic. The early market for PDAs actually turned out to be for what were later characterized as “mobile companions” – devices that let computer users take their digital data with them. And that ended up being far less important than what were next called “personal communicators” (nowadays, “smart phones”).</div><div> </div><div>The failure of the Newton, which was well researched if not as well researched as the Edsel (to continue Jobs’ allusion to Ford), may be something that Jobs has internalized as a reason to avoid MR.</div><div> </div><div>Speaking of failure, it is easy to overlook Jobs’ own impressive record of failure: <a mce_href="http://en.wikipedia.org/wiki/Apple_Lisa" href="http://en.wikipedia.org/wiki/Apple_Lisa">the Lisa</a>, NeXT, Motorola ROKR, Apple TV. That said, 8 of the <a mce_href="http://www.newlaunches.com/archives/top_10_apple_products_which_flopped.php " target="_new" href="http://www.newlaunches.com/archives/top_10_apple_products_which_flopped.php ">Top 10 Apple Products Which Flopped</a> were designed during the Jobs interregnum, when Jobs was not with Apple. But this just demonstrates how hard it is to launch groundbreaking new products.</div><div> </div><div>So how does Apple try to do just that? The <i>Pragmatic Marketing</i> article <a mce_href="http://www.pragmaticmarketing.com/publications/magazine/6/4/you_cant_innovate_like_apple " target="_new" href="http://www.pragmaticmarketing.com/publications/magazine/6/4/you_cant_innovate_like_apple ">You Can’t Innovate Like Apple</a> frames it this way:</div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Start with a gut sense of an opportunity, and the conversations start rolling.<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">What do we hate?<br>A: Our cell phones.<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">What do we have the technology to make?<br>A: A cell phone with a Mac inside.<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">What would we like to own?<br>A: An iPhone, what else?<br></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">But Jobs also explained that in this specific conversation, there were big debates across the organization about whether or not they could and should do it. Ultimately, he looked around and said, “Let’s do it.”</blockquote><div> </div><div>To paraphrase <i>Pragmatic Marketing</i>, “you can’t innovate without market research the way Apple claims to”. First, your organization is not Apple. Apple develops hardware, <a mce_href="http://money.cnn.com/galleries/2008/fortune/0803/gallery.jobsqna.fortune/8.html " target="_new" href="http://money.cnn.com/galleries/2008/fortune/0803/gallery.jobsqna.fortune/8.html ">operating systems</a> and Internet services, giving it a unique capability among the technology firms it competes with to innovate across all three layers; most other companies can innovate along only one of these dimensions and must partner with other firms. Second, your CEO (apologies to him or her) is not Steve Jobs. Steve Jobs is a unique innovator in American industry, Disneyesque for his stature and record of innovation. Arguably no other American company has a CEO with Steve’s track record for successfully introducing new products.</div><div> </div><div>Face it, as even Jobs’ and Apple’s track record shows, many innovative new products fail—launching a new product or service is probably the most difficult task any organization undertakes. The odds are against you, and therefore even well researched products can fail. But the issue is not that market research was done at all but that it wasn’t done to the level needed; perhaps it wasn’t framed correctly or wasn’t timely enough: for the Newton, the group researched were consumers without computers, a poorly chosen target market if ever there was one; for the Edsel, the research characterized car buyers in the early 1950s but their needs and wants had changed by 1957 (thanks to two recessions since the research was commissioned and thanks to Mercury repositioning to address some of that opportunity).</div><div> </div><div>And -- sorry, Henry Ford -- but learning that your customers want a faster horse is actually incredibly valuable input. True disruptions in your industry, powered by these seemingly impossible requests, provide the seeds for incredible growth from innovation. To identify those disruptions, you need to thoroughly understand what your customers dislike about your current industry and your current offerings. It’s not the job of market research to creatively determine the solutions—it’s the job of market research to identify the problems that, if solved by some means, create these enormous opportunities.</div><div> </div><div>Blockbuster customer satisfaction research showed that customers hated late fees and were disappointed in the size of store inventories, but those facts were baked into Blockbuster’s business model. It took creative thinking and iteration for Netflix to determine how to solve these pain points (people forget that Netflix also had late fees when it started, before moving to monthly subscriptions two years after its founding). Iteration, based on research into earlier failure, is essential: the Macintosh followed the Lisa, the iPhone followed the Apple-inspired Motorola ROKR, and so on.</div><div> </div><div>As <a mce_href="http://www.90percentofeverything.com/" target="_new" href="http://www.90percentofeverything.com/">Harry Brignull</a> wrote on Bokardo, “What makes a more compelling story? To focus on 99% perspiration (boring) or the 1% inspiration (amazing)?” </div><div> </div><div>Market research, diligently done with perspiration, can provide the spark that ignites inspiration.</div><div><br></div> </article> <article> <h1>Advocacy Loyalty Index (ALI) and Purchasing Loyalty Index (PLI)</h1> <p>Jeffrey Henning — Wed, 22 Jul 2009 08:28:00 GMT</p> <div><a mce_href="http://www.businessoverbroadway.com/about.htm" target="_new" href="http://www.businessoverbroadway.com/about.htm">Business Over Broadway (BOB)</a> developed two loyalty indices by analyzing the results from two satisfaction surveys that included eight satisfaction and loyalty questions:</div><div><ol><li>Overall satisfaction</li><li>Likelihood to choose again for the first time</li><li>Likelihood to recommend</li><li>Likelihood to continue purchasing same products/services</li><li>Likelihood to purchase different products/services</li><li>Likelihood to increase purchase size</li><li>Likelihood to increase frequency of purchasing</li><li>Likelihood to switch to a different provider</li></ol></div><div>BOB than conducted a factor analysis on the results to determine which of three types of loyalty a question would better measure:</div><div><ul><li><b>Advocacy Loyalty</b>—reflecting the degree to which customers will recommend the company to others</li><li><b>Purchasing Loyalty</b>—reflecting the degree to which customers will increase their purchasing behavior</li><li><b>Defection Loyalty</b>—reflecting the degree to which customers will switch to a different company</li></ul></div><div>This factor analysis showed which factor each question best fit with:</div><div><table class="MsoTableGrid" border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none;mso-border-alt:solid black .5pt; mso-border-themecolor:text1;mso-yfti-tbllook:1184;mso-padding-alt:0in 5.4pt 0in 5.4pt"> <tbody><tr style="mso-yfti-irow:0;mso-yfti-firstrow:yes"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-left:none;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Advocacy Loyalty<o:p></o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-left:none;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Purchasing Loyalty<o:p></o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-left:none;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Defection Loyalty<o:p></o:p></p> </td> </tr> <tr style="mso-yfti-irow:1"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Overall satisfaction<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:2"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to choose again for the first time<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:3"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to recommend<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:4"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to continue purchasing same products/services<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:5"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to purchase different products/services<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:6"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to increase purchase size<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:7"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to increase frequency of purchasing<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> </tr> <tr style="mso-yfti-irow:8;mso-yfti-lastrow:yes"> <td width="256" valign="top" style="width:192.1pt;border:solid black 1.0pt; mso-border-themecolor:text1;border-top:none;mso-border-top-alt:solid black .5pt; mso-border-top-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">Likelihood to switch to a different provider [scale reversed for analysis]<o:p></o:p></p> </td> <td width="124" valign="top" style="width:93.1pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="133" valign="top" style="width:99.45pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><o:p> </o:p></p> </td> <td width="126" valign="top" style="width:94.15pt;border-top:none;border-left: none;border-bottom:solid black 1.0pt;mso-border-bottom-themecolor:text1; border-right:solid black 1.0pt;mso-border-right-themecolor:text1;mso-border-top-alt: solid black .5pt;mso-border-top-themecolor:text1;mso-border-left-alt:solid black .5pt; mso-border-left-themecolor:text1;mso-border-alt:solid black .5pt;mso-border-themecolor: text1;padding:0in 5.4pt 0in 5.4pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal">X<o:p></o:p></p> </td> </tr></tbody></table></div><div> </div><div>Each index is calculated simply by averaging the rating of each component. Since no benchmark information is provided, you are free to use <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">whichever scales you find appropriate</a>. Unfortunately, the original BOB scales use a 0-10 bipolar scale for satisfaction and 0-10 unipolar scale for likelihood.</div><div> </div><div>Add this to the growing body of research that shows that <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">likelihood to recommend</a> is not a unique measure of advocacy.</div><div> </div><div>Two counterintuitive findings from the research:</div><div><ul><li>Overall satisfaction was found to be a factor of loyalty, despite significant research into <a mce_href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/blog/bid/17877/The-Apostle-Model">how satisfaction and loyalty can differ</a>. That said, part of the disconnect can be explained by the fact that satisfaction contributes primarily to one type of loyalty: advocacy rather than purchasing loyalty.</li><li>The likelihood to repurchase the same products was less an indicator of <i>purchasing </i>loyalty than it was of <i>advocacy </i>loyalty.</li></ul></div><div>Of all the loyalty indices I’ve studied, the ALI and PLI are the most rigorously developed. For more detail, refer to the third edition of <a mce_href="http://www.amazon.com/Measuring-Customer-Satisfaction-Loyalty-Third/dp/0873897439" target="_new" href="http://www.amazon.com/Measuring-Customer-Satisfaction-Loyalty-Third/dp/0873897439"><i>Measuring Customer Satisfaction and Loyalty</i></a> by Bob E. Hayes, Ph.D.</div> </article> <article> <h1>CEM (Customer Experience Management) Best Practices</h1> <p>Rod Morris — Thu, 16 Jul 2009 18:59:00 GMT</p> <p>Last month we completed new research that shows that business leaders are placing more importance on customer experience management during the global economic downturn. Forty-seven percent of respondents said that customer experience management (CEM) was important to them in 2008, and for 2009 this number increased to 63% of respondents. This is evidence that the recession has driven many organizations to focus more closely on understanding how customers are interacting with them, putting increased attention on CEM and Voice of the Customer (VOC) programs.</p><p>With our partner CGA, a customer-experience consultancy in the United Kingdom, we surveyed more than 200 organizations between May 20 and June 2, 2009, to determine what the most successful organizations are doing to achieve measurable improvements in <a mce_href="http://www.vovici.com/survey-solutions/Voice-of-the-Customer.aspx" href="http://www.vovici.com/survey-solutions/Voice-of-the-Customer.aspx">customer loyalty</a>. </p><p>The results of the Vovici/CGA Customer Experience IQ study strongly show that CEM and VOC programs can be effective at maintaining and increasing customer loyalty even in a very difficult economic environment. We are hosting a webinar on the results of this study, highlighting the seven best practices of organizations with highly loyal customers; you can sign up for this one-hour webcast, Improving Your Organization's Customer Experience IQ, which will be on Tuesday, July 21st, 2 PM ET/11 AM PT: <a mce_href="http://www.vovici.com/webinars/20090721.aspx" href="http://www.vovici.com/webinars/20090721.aspx">http://www.vovici.com/webinars/20090721.aspx<br></a></p><p>Research findings from the Customer Experience IQ study also reveal that, out of 24 best practices that were surveyed, possessing a formal customer experience strategy with tactical plans to execute against it is the best practice most highly correlated to high customer loyalty. And what better way to create such a plan than to look at the seven best practices with the highest correlation to customer loyalty?</p><p>1. Creating a formal customer experience management strategy and executing against it<br>2. Integrating customer relationship management data into feedback efforts<br>3. Including customer ideas for both strategic initiatives and tactical process improvements<br>4. Defining planning cycles to include formal processes for obtaining customer input<br>5. Systematically sharing feedback with customer-facing employees<br>6. Engaging customers in co-creation<br>7. Sharing Voice of the Customer data across organizational boundaries<br><br>These findings help us narrow our advice to organizations seeking to improve customer loyalty. Every organization has finite resources; while it would be wonderful if an organization adopted all 24 best practices, they need to prioritize those that they are not currently doing that will have the biggest impact.</p><p>The Vovici/CGA study also shows that organizations with the highest customer loyalty share CEM (Customer Experience Management) and VOC (Voice of the Customer) feedback freely throughout the enterprise. By coordinating feedback efforts, organizations are less apt to repeatedly contact customers with redundant requests, thus eliminating feedback fatigue and keeping customers engaged. Solutions such as Vovici v4 address this by offering a common platform that enables individuals to obtain feedback and easily share it throughout their organization.</p><p>Look for more to come from us in the coming weeks as we continue to analyze and present this important research.</p> </article> <article> <h1>The Phone Survey in Decline</h1> <p>Jeffrey Henning — Mon, 13 Jul 2009 15:04:00 GMT</p> <div>Last week a webinar attendee asked me why I had mentioned that usage of phone surveys is declining. I see five reasons driving this trend:</div><div><ol><li><span class="Apple-style-span" style="font-weight: bold; ">Many of the 72+% of American households on the Do Not Call list refuse to take a survey when the phone rings</span>. In the United States, the <a mce_href="https://www.donotcall.gov/" href="https://www.donotcall.gov/">national Do Not Call registry</a> allows households to remove themselves from unsolicited phone calls from organizations that they haven’t done business with. While the Do Not Call list does not opt households out of receiving calls from survey researchers, it has created the expectation that consumers do not have to take unsolicited calls from market researchers.</li><li><b>The rise in “cord cutters” – households with no landline telephone but with only cell phones – has increased the cost of telephone surveys</b>. By law, you can only use automatic dialing for landlines, not cell phones. Manual dialing is much more labor intensive: that annoying pause you get when you answer such a call is because the dialer is now transferring the call to an active agent, after having tried unsuccessfully to get an answer on many prior calls; without such dialers, agents are spending their time making calls that aren't answered.</li><li><b>Web surveys are much less expensive than telephone surveys</b>. The respondent to a web survey is, in effect, donating the data entry cost, as they select their answers; with a phone survey, you are paying a call center representative to transcribe respondents’ answers.</li><li><b>Web surveys are preferred by most respondents</b>. When we invite people to take a survey and offer them a choice of survey modalities, they almost always choose the web link over a paper survey or telephone survey. </li><li><b>Phone surveys with random digit dialing are no longer representative of the U.S. population</b>, as <a mce_href="http://blog.vovici.com/blog/bid/17843/What-s-the-Catch-Does-Sample-Sourcing-Matter" href="http://blog.vovici.com/blog/bid/17843/What-s-the-Catch-Does-Sample-Sourcing-Matter">landline telephones skew to older respondents</a>. From the 1950s to the 1990s randomly composing a phone number and calling it provided a true probability sample of the U.S. population, but <a mce_href="http://blog.vovici.com/blog/bid/17847/Random-Thoughts-on-2008-MRA-Annual-Conference" href="http://blog.vovici.com/blog/bid/17847/Random-Thoughts-on-2008-MRA-Annual-Conference">that Golden Age</a> has ended.</li></ol></div><div>I try to answer all questions sent my way from webinars, though I can’t always do so in a timely fashion. To see if there is a webcast that you might want to attend, please check out our <a mce_href="http://www.vovici.com/about/webinars.aspx" href="http://www.vovici.com/about/webinars.aspx">list of upcoming webinars</a>.</div> </article> <article> <h1>Data Cleaning</h1> <p>Jeffrey Henning — Wed, 08 Jul 2009 07:41:00 GMT</p> <p>Early in my career I learnt survey data cleaning firsthand from <a mce_href="http://www.strategyanalytics.com/default.aspx?mod=Biography&a0=751" href="http://www.strategyanalytics.com/default.aspx?mod=Biography&a0=751"><font class="Apple-style-span" color="#000000"><span class="Apple-style-span" style="text-decoration: none;">Jo Ann De Clercq</span></font></a>, who also taught me how to code responses to open-ended questions. Back then, we had a body of practices that we used from study to study, but no formal documentation of those practices.</p><div>The paper "<a mce_href="http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020267#pmed-0020267-g001" href="http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020267#pmed-0020267-g001">Data Cleaning: Detecting, Diagnosing, and Editing Data Abnormalities</a>", published in <i>PLoS Medicine</i> in 2005, provided the first systematic review of data cleaning. The authors offer this bit of background:</div><div> </div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Data cleaning is emblematic of the historical lower status of data quality issues and has long been viewed as a suspect activity, bordering on data manipulation. Armitage and Berry almost apologized for inserting a short chapter on data editing in their standard textbook on statistics in medical research. Nowadays, whenever discussing data cleaning, it is still felt to be appropriate to start by saying that data cleaning can never be a cure for poor study design or study conduct. Concerns about where to draw the line between data manipulation and responsible data editing are legitimate. Yet all studies, no matter how well designed and implemented, have to deal with errors from various sources and their effects on study results.</blockquote><p>The authors outline a data-cleaning process with three steps: Screening Phase, systematically looking for problems with the data; Diagnostic Phase, identifying the condition of the suspect data; and Treatment Phase, deleting or editing the data or leaving it as is.</p><p> </p><h4>Screening Phase</h4><div>Examine data for five different kinds of possible errors:</div><div><ol><li><b>Lack of data</b> – Do some questions have far fewer answers than surrounding questions?</li><li><b>Excess of data</b> – Are there duplicate responses?</li><li><b>Outliers/inconsistencies</b> – Are there values that are so far beyond the typical that they seem potentially erroneous?</li><li><b>Strange patterns</b> – Are there patterns that imply cheating rather than honest answers? For instance, does a respondent alternate between ratings of 4 and 5 on every other topic in a matrix question?</li><li><b>Suspect analysis results</b> – Do the answers to some questions seem counterintuitive or extremely unlikely?</li></ol></div><h4>Diagnosis Phase</h4><p>From the Screening Phase you have highlighted data that needs investigation. To clarify suspect data, you often must review all of a respondent’s answers to determine if the data makes sense taken in context. Sometimes you must review a cross-section of different respondents’ answers, to identify issues such as a skip pattern that was specified incorrectly. </p><div>With this research complete, what is the true nature of the data that you’ve highlighted? The five possible values the authors give:</div><div><ol><li><b>Missing data</b> – Answers omitted by the respondent or questions skipped over </li><li><b>Errors</b> – Typos or answers that indicate the question was misunderstood</li><li><b>True extreme</b> – An answer that seems high but can be justified by other answers (e.g., the respondent working 100 hours a week because they work a full-time job and two part-time jobs)</li><li><b>True normal</b> – A valid answer </li><li><b>No diagnosis, still suspect</b> – The verdict is out on this “idiopathic” data. When it comes time for the Treatment Phase, you may need to make a judgment call on how to treat this data.</li></ol></div><h4>Treatment Phase</h4><div>You’ve screened the data and tried to come to a verdict on whether suspect data is guilty or innocent. You have three choices for what to do with suspect data:</div><div><ol><li><b>Leave it unchanged</b> – The most conservative course of action is to accept this data as a valid response and make no change to it. The larger your sample size, the less that one suspect response will affect the analysis; the smaller your sample size, the more difficult the decision.</li><li><b>Correct the data</b> – If the respondent’s original intent can be determined, then I am in favor of fixing their answer. For instance, perhaps it is clear from the respondent’s explanation for their ratings that they reversed the scale in their minds; you can invert each of their answers to this question to correct the issue. Some statisticians will argue for imputation, replacing the answers with imputed values, such as the mean for that variable, but the techniques for imputation can become quite elaborate and are best left to professional statisticians.</li><li><b>Delete the data</b> – The data seems illogical and the value is so far from the norm that it will affect descriptive or inferential statistics. What to do? Delete just this response or delete the entire record? Whenever you begin to toss out data, it raises the possibility that you are “cherry picking” the data to get the answer you want. </li></ol></div><div>However you choose to treat the data, make sure to document in your survey report what steps you took, how many responses were affected and for which questions.</div><h4>Conclusion</h4><p>Data cleaning is time-consuming, troublesome and potentially contentious. Further, many issues can be avoided by setting up data validation during the survey design. For instance, recently for an <a mce_href="http://blog.vovici.com/blog/bid/18077/Age-Question" href="http://blog.vovici.com/blog/bid/18077/Age-Question">age question</a>, I actually looked up the age of the oldest person alive and used that as my boundary condition. I did double check the results to make sure I didn’t have a surfeit of centenarians answer the survey, but this validation kept someone from entering 1697 as their birth year (as a typo for 1967); had they done so, the survey would have immediately alerted them to the fact. Far better to let the respondent catch and fix their mistakes than have to do it for them. </p><div>When I started my career, if an answer seemed wrong, we could pull a paper questionnaire to see if a data entry mistake had been made. On the other hand, if it was for a telephone survey, we often had to interpret the response, where the interviewer had clearly misheard the name of a vendor or the acronym used to describe a technology. Web surveys give you the opportunity to prevent many types of errors from being recorded—many, but not all. Cleaning data is never pretty, but it’s an important step that should be taken for any strategic survey.</div><div><br></div> </article> <article> <h1>Employee Net Promoter Score</h1> <p>Jeffrey Henning — Tue, 07 Jul 2009 07:55:00 GMT</p> <div><p>The <a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Net Promoter Score (NPS)</a> has jumped from use as a customer retention measure to use as an employee retention model. Despite <a mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">criticisms of NPS</a>, it remains popular because it is well marketed, easy to understand and its model makes intuitive sense: every organization wants more promoters than detractors.</p><p>As a result of this popularity, NPS has crossed over to employee loyalty research, where it is known as ENPS, the Employee Net Promoter Score.</p><p> </p><p>Symantec, Atlas Copco, Holcim and Celanese are international brands that have adopted ENPS. </p><p>Holcim Ltd., a global provider of building materials, uses ENPS worldwide to measure the loyalty of its 90,000 staff, but doesn’t benchmark staff in different countries against one another, finding that cultural differences contribute to dramatic measurement differences, according to Christian Birck, a senior vice president of Holcim.</p><p>Celanese, a chemical producer, was an early adopter of ENPS, embracing it in September of 2007. Alan Maxwell, Vice President of Corporate Human Resources, said, “From a Six Sigma perspective, there was a need to baseline current state and track progress as the model/programs were implemented. ENPS hit the mark for us.” Since quarterly tracking began, Celanese has seen its ENPS steadily increase, from -8% to -3% to 7% to 24%. Further, KPIs have also improved, with turnover dropping by half, for example.</p><p>ENPS offers a model that makes intuitive sense, but—while Fred Reichheld contends that NPS correlates to company growth for eight industries—no such correlation has yet been demonstrated for employee loyalty. Further, employees can be willing to recommend you and yet still disloyal, as shown by this Walker analysis of the true cost of <a mce_href="http://blog.vovici.com/Blog/bid/18084/The-True-Cost-of-Low-Employee-Loyalty/" href="http://blog.vovici.com/Blog/bid/18084/The-True-Cost-of-Low-Employee-Loyalty/">low employee loyalty</a>.</p><p>Of course, if your primary concern is around recruiting, ENPS can be a good choice. ENPS is simple to implement and is good for organizations looking to “do it themselves”, whereas <a mce_href="http://blog.vovici.com/blog/bid/18535/Employee-Engagement-Survey-The-Gallup-Q12" href="http://blog.vovici.com/blog/bid/18535/Employee-Engagement-Survey-The-Gallup-Q12">Gallup Q12</a> and <a mce_href="http://blog.vovici.com/blog/bid/18068/Employee-Satisfaction-Benchmark-from-Walker-Information" href="http://blog.vovici.com/blog/bid/18068/Employee-Satisfaction-Benchmark-from-Walker-Information">Walker Loyalty</a> are copyrighted instruments of their respective organizations, and can only be implemented by those firms or their partners.</p><p>ENPS does not provide much in the way of actionable information that organizations can use to improve. Gallup and Q12 offer an extensive program of organizational-effectiveness consulting that can be used to drive greater results. Walker—a Vovici partner—provides a detailed battery of items to evaluate, and includes a detailed and segmented prioritization of attributes that affect employee engagement.</p><p>With only 4.3% of North American organizations surveyed by Vovici <a mce_href="http://blog.vovici.com/blog/bid/17838/Benchmarking-of-Interest-but-Little-Used" href="http://blog.vovici.com/blog/bid/17838/Benchmarking-of-Interest-but-Little-Used">using employee-satisfaction benchmarking</a>, your organization should look at the <a mce_href="http://blog.vovici.com/blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession" href="http://blog.vovici.com/blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession">opportunities offered by such benchmarks</a>.</p></div> </article> <article> <h1>VOC ROI: The Return on Investment of Voice of the Customer</h1> <p>Jeffrey Henning — Mon, 06 Jul 2009 10:52:00 GMT</p> <p>At the Forrester Customer Experience Forum last month, Forrester announced the winners of its <a mce_href="http://blogs.forrester.com/customer_experience/2009/06/voice-of-the-customer-winners-experian-progressive-and-vanguard.html" href="http://blogs.forrester.com/customer_experience/2009/06/voice-of-the-customer-winners-experian-progressive-and-vanguard.html"><font class="Apple-style-span" color="#000000"><span class="Apple-style-span" style="text-decoration: none; ">first-ever Voice of the Customer Award</span></font></a>: Experian, Progressive and Vanguard. Unlike the other two winners, <a mce_href="http://blogs.forrester.com/files/2009forrestervocaward_vanguard_share.pdf" href="http://blogs.forrester.com/files/2009forrestervocaward_vanguard_share.pdf"><font class="Apple-style-span" color="#000000"><span class="Apple-style-span" style="text-decoration: none;">Vanguard in its award application</span></font></a> quantified the financial benefits of acting on the Voice of the Customer in detail.</p><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Vanguard’s attention to voice of the customer significantly affects our business results. In the past two years alone, issues identified through listening to both our clients and our crew [employees] have resulted in approximately $6.9 million in cost reduction or avoidance in our Retail business:<br><ul><li>$2.6 million from process efficiencies</li><li>$1.1 million from improving the asset transfer process</li><li>$1.0 million from improving fulfillment of literature</li><li>$0.9 million from improving our problem resolution efforts</li><li>$0.9 million from document management improvements</li><li>$0.4 million from improving transaction processing quality and efficiency</li></ul><p>In our Institutional Retirement Plan Services (IRPS) business:</p></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Client identified issues with interactions including plan contributions, loan repayment and beneficiary designations led to a multi-year initiative to overhaul and improve workflow. While the project is not yet complete, it has already resulted in cost savings of approximately $1.3 million. Anticipated savings in 2009 exceed five times that amount [$6.5+ million].</blockquote></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">A separate project in our IRPS business focused on e-delivery:</blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Clients told us they wanted less mailbox clutter, more information available online and for Vanguard to decrease their impact on the environment. Vanguard began providing electronic statements, confirmations, notices, and newsletters as well as multi-media learning. As a result, we saw both a decrease in customer complaints and reduced or avoided approximately $6.5 million in costs.</blockquote></blockquote><p>So from acting on the Voice of the Customer that’s a total savings of $14.7 million in 2007 and 2008, growing to at least another $21.2 million in 2009. While bottom-line savings are important, top-line growth is even better:</p><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">Since 2005, the voice of the customer program has identified numerous opportunities for our phone representatives to better meet the needs of both clients and prospects. Acting on voice of the customer data, one retail business unit developed new training programs to focus on client identified opportunities for our representatives to improve, revised the quality measures that are part of representatives’ performance evaluations to reflect client-critical factors, and adjusted monitoring processes so that supervisors could review the entire client experience, not simply one call. Due to these and other changes made in this business area, client conversion rates have improved by 32% since 2005. This translates into a 94% increase in new assets from these clients.</blockquote><p>Obviously, for competitive reasons, Vanguard does not release all the details on its spending on Voice of the Customer or what the top-line impact was of its near-doubling of new assets. While this makes it impossible to generate a true ROI for Vanguard VOC efforts, it presents as full a public picture of the sound economic reasons for investigating in Voice of the Customer initiatives as we’ve seen.</p><div>Congratulations to Vanguard for this well-deserved award, as well as a special thanks to them for highlighting Vovici as one of the technology vendors “critical to their success”.</div> </article> <article> <h1>Vovici Appoints Greg Stock Chairman and CEO</h1> <p>Vovici Marketing — Wed, 01 Jul 2009 07:59:00 GMT</p> <p>DULLES, Va.--(<a href="http://www.businesswire.com/" mce_href="http://www.businesswire.com/">BUSINESS WIRE</a>)--Vovici, the leading provider of enterprise feedback management (<a href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-software%2Fenterprise-survey-software.aspx&esheet=5998330&lan=en_US&anchor=EFM&index=1" target="_blank" mce_href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-software%2Fenterprise-survey-software.aspx&esheet=5998330&lan=en_US&anchor=EFM&index=1" shape="rect">EFM</a>) solutions and <a href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-software%2Fonline-survey-software.aspx&esheet=5998330&lan=en_US&anchor=survey+software&index=2" target="_blank" mce_href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-software%2Fonline-survey-software.aspx&esheet=5998330&lan=en_US&anchor=survey+software&index=2" shape="rect">survey software</a>, today announced that its board of directors has appointed Greg Stock as the company’s Chairman and Chief Executive Officer (CEO), effective July 1. A proven technology leader, Stock brings more than 20 years of technology industry and marketing experience to Vovici. </p> <p> "As a CEO, I know how important it is to understand the collective voice of customers," said Stock. "Vovici's solutions enable organizations to take customer feedback and turn it into customer loyalty. Vovici has already delivered high-impact feedback solutions to more than half the Fortune 500. I am excited to join the team and build on this foundation of success." </p> <p> An accomplished executive, Stock joins Vovici having most recently served as president and CEO of Mirage Networks, a provider of network security solutions based in Austin, Texas. During his tenure, Stock led the company from start-up to market leader with more than 600 customers across 40 countries. Mirage was successfully acquired by Trustwave in February, 2009. </p> <p> Prior to Mirage Networks, Stock was a key player in successful DC-area technology companies. As Vice President, Marketing, Stock drove Vastera, a provider of global trade management solutions, to the market leading position and was instrumental in the company’s successful initial public offering in September, 2000. Previous to this, Stock helped Manugistics, the supply chain management leader, grow to more than $200M in revenues and initiated the company’s expansion into China and Australia. </p> <p> Vovici’s investors include Austin Ventures and Mayfield Fund. “Greg is an outstanding choice to lead Vovici,” said Chris Pacitti, General Partner at Austin Ventures and Vovici board member. “A tenacious leader with an inspiring approach that energizes people to achieve, Greg has a demonstrated track record for establishing market leadership and global expansion initiatives. Much of Greg’s past experience is in leading sales and marketing organizations, positioning him well to lead a company such as Vovici that is focused on helping its clients create <a href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2FVoice-of-the-Customer.aspx&esheet=5998330&lan=en_US&anchor=customer+loyalty&index=3" target="_blank" mce_href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2FVoice-of-the-Customer.aspx&esheet=5998330&lan=en_US&anchor=customer+loyalty&index=3" shape="rect">customer loyalty</a> solutions.” </p> <p> Stock is succeeding Dean Wiltse who has served as Vovici’s Chairman and CEO since 2005. Wiltse oversaw the merger of Perseus and Websurveyor in 2006, re-branded the company as Vovici, and converted the business into a software-as-a-service (SaaS) enterprise. “The Board would like to thank Dean for his achievements to date and know he is looking forward to spending more time with his family in Arizona,” said David Lack, Partner at Austin Ventures. </p> <p> “We are delighted to welcome Greg to Vovici,” said Dean Wiltse. ”Vovici has established itself as the SaaS leader in <a href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2Fenterprise-feedback-management.aspx&esheet=5998330&lan=en_US&anchor=enterprise+feedback+management&index=4" target="_blank" mce_href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2Fenterprise-feedback-management.aspx&esheet=5998330&lan=en_US&anchor=enterprise+feedback+management&index=4" shape="rect">enterprise feedback management</a>. I look forward to working with Greg as the company enters its next phase of growth, delivering EFM solutions to clients so that they can better capture and analyze their <a href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2FVoice-of-the-Customer.aspx&esheet=5998330&lan=en_US&anchor=customer+feedback&index=5" target="_blank" mce_href="http://cts.businesswire.com/ct/CT?id=smartlink&url=http%3A%2F%2Fwww.vovici.com%2Fsurvey-solutions%2FVoice-of-the-Customer.aspx&esheet=5998330&lan=en_US&anchor=customer+feedback&index=5" shape="rect">customer feedback</a>.”</p> </article> <article> <h1>Negative Feedback is a Positive for Online Communities</h1> <p>Jeffrey Henning — Mon, 29 Jun 2009 11:14:00 GMT</p> <div>Far and away, the most common objection we hear from organizations considering extending their survey panel into an online community is that community members will hear each other’s negative feedback. When you survey customers, employees or other key constituencies, only the survey administrator sees all the negative comments. When you move to an online community, anyone who logs in can see the negative comments.</div><div> </div><div>This is more an opportunity than an issue. In fact, the absolute value of negative feedback is positive. Here’s why:</div><div><ol><li><b>Criticisms provide authenticity</b>. Imagine, for a moment, that you hosted an online community and never received any bad reviews or comments. Since every organization and its product and services has shortcomings, an online community where those shortcomings were never discussed would seem unauthentic. It would seem like a sham or a marketing exercise, rather than a true community.</li><li><b>Negative feedback is scarcer than positive feedback</b>. That scarcity makes it worth more. Online communities excel at generating thousands of positive ideas—more ideas than your organization can implement. Negative feedback is rarer than organizations realize and often clusters around key areas that your organization has ignored or handled poorly; negative feedback gives you a chance to prioritize these issues and focus on improving them.</li><li><b>Negative feedback is actionable</b>. Negative feedback gives you an opportunity to respond. If a customer tells you ten features they want in the next edition of your product or service, there is nothing you can do for them today but to let them know you are listening and building a list. If they complain about a product damaged in shipping, or a mistake on an invoice, or confusion around a feature, you can immediately help them resolve the issue. Quick resolution of these issues demonstrates that your organization listens and cares.</li><li><b>Better to manage negative feedback on your turf</b>. The fact of the matter is that people are commenting on your organization all over the World Wide Web: on Twitter, on Facebook, on LinkedIn and on the next big social networking site that we haven’t heard of yet. Brand monitoring solutions can catch some of this commentary, but not all of it: many social networks only show designated friends each other’s comments. Right now on Facebook there’s a long discussion going on about why your organization is awful to do business with, and you will never see or be able to respond to that discussion. When the criticism happens on your site, you can instantly read and respond.</li></ol></div><div>For further reading:</div><div><ul><li><a mce_href="http://blog.vovici.com/Blog/bid/18121/The-Seven-Deadly-Sins-of-Online-Community-Management/" href="http://blog.vovici.com/Blog/bid/18121/The-Seven-Deadly-Sins-of-Online-Community-Management/">The Seven Deadly Sins of Online Community Management</a>, of which the first is “Pride: Preventing community members from criticizing you”.</li><li>An “<a mce_href="http://blog.vovici.com/Blog/bid/17885/Dealing-with-Detractors-in-Online-Communities" href="http://blog.vovici.com/Blog/bid/17885/Dealing-with-Detractors-in-Online-Communities">abuse grid</a>” for dealing with detractors in online communities.</li><li><a mce_href="http://blog.vovici.com/Blog/bid/17882/Gramma-C-s-Process-for-Handling-Irate-Community-Members" href="http://blog.vovici.com/Blog/bid/17882/Gramma-C-s-Process-for-Handling-Irate-Community-Members">Three rules about negative feedback</a> with advice on how to respond to detractors.</li></ul></div> </article> <article> <h1>Follow-up Survey/Transaction Survey</h1> <p>Jeffrey Henning — Fri, 26 Jun 2009 16:18:00 GMT</p> <p>The follow-up or transactional survey concerns itself with getting customer feedback regarding a specific transaction, such as a purchase, a call to a contact center, a request for service or a product return.</p><p>Such surveys can be conducted for multiple reasons. They are a great way to perform quality control to determine the level of service being provided and can be used to determine inconsistencies in providing service. Follow-up surveys can identify dissatisfied customers so that service recovery can be attempted and can measure the effectiveness of service staff.</p><p>Here are some of the more common mistakes I’ve seen when organizations conduct transactional surveys:</p><div><ul><li>Asking respondents to specify details about the transaction rather than using <a mce_href="http://blog.vovici.com/blog/bid/18044/Panel-Management-Software-and-Data-Integration" href="http://blog.vovici.com/blog/bid/18044/Panel-Management-Software-and-Data-Integration">data integration</a> behind the scenes to record that information</li><li>Allowing service staff to select or influence potential respondents, for instance, by transferring some calls but not others to an IVR system—this results in skewed results that typically overstate satisfaction</li><li>Failing to include any <a mce_href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">survey alerts or email triggers</a> to enable service recovery to be attempted</li><li>Failing to <a mce_href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" href="http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">invite the recipient more than once</a> to take the survey, which can result in bias</li><li><a mce_href="http://blog.vovici.com/blog/bid/18109/Employee-Compensation-Plan-Incorporating-Survey-Results" href="http://blog.vovici.com/blog/bid/18109/Employee-Compensation-Plan-Incorporating-Survey-Results">Compensating employees on survey results</a> in a way that encourages gaming the system and unethical behavior</li><li>Not sharing survey results with service staff</li><li>Having <a mce_href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" href="http://blog.vovici.com/blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">the questionnaire take more time to complete</a> than the transaction itself—here’s a <a mce_href="http://www.greatbrook.com/home_depot_customer_survey.htm" href="http://www.greatbrook.com/home_depot_customer_survey.htm">retail example</a> and an <a mce_href="http://blog.vovici.com/blog/bid/17894/When-Surveys-Go-Bad" href="http://blog.vovici.com/blog/bid/17894/When-Surveys-Go-Bad">auto club example</a></li><li>Inviting participants on a monthly basis rather than weekly or daily—respondents are typically unable to answer in detail after more than a week has passed</li><li>Conversely, inviting participants to take the survey before the incident is resolved</li><li>Failing to implement touch-frequency rules where respondents are not invited too often; for instance, not invited more than once in a 30-day period</li><li>Failing to implement a <a mce_href="http://blog.vovici.com/blog/bid/18071/Unsubscribe-Survey-Templates" href="http://blog.vovici.com/blog/bid/18071/Unsubscribe-Survey-Templates">survey unsubscribe process</a> so that customers can opt out of recieving surveys altogether</li></ul></div><div>What mistakes have you seen in transactional surveys you've taken?</div> </article> <article> <h1>Survey Test Mode</h1> <p>Jeffrey Henning — Thu, 25 Jun 2009 16:28:00 GMT</p> <div>I hate that every gas station near me has disabled the ability for me to set the fuel dispenser to fill the tank and then walk away, safe in the knowledge the automatic cut-off will kick in. Now I have to stand at the pump manually holding the switch on the fuel dispenser until the tank is full. No doubt this is a safety innovation in case the auto cut-off failed.</div><p>In a similar way, many users of our <a mce_href="http://www.vovici.com" href="http://www.vovici.com"><font class="Apple-style-span" color="#000000"><span class="Apple-style-span" style="text-decoration: none;">survey software</span></font></a> dislike the fact that the first time they publish a new survey, it goes into test mode. This is a not-so-subtle reminder that they should double-check the survey before inviting participants to respond.</p><div>Did I say <i>double-check?</i> How about triple-check and quadruple-check? For very important surveys, you should:</div><div><ol><li><b>Self-Test</b> – Run through the survey, answering it yourself, multiple times.</li><li><b>Pre-Test</b> – Invite coworkers or friendly outsiders to take the survey.</li><li><b>Pilot Test</b> – Invite 10% of the targeted list to take the survey.</li><li><b>Publish</b> – Invite the world to take the survey.</li></ol></div><h4>Self Test</h4><div>Here are some of the things that I self-test or review before publishing a survey:</div><div><ul><li><b>Question flow</b>: Do <a mce_href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically" href="http://blog.vovici.com/blog/bid/18285/Order-Questions-Logically">questions proceed in a logical manner</a> from topic to topic?</li><li><b>Question wording</b>: Is each question worded clearly and unambiguously and is it free of typos and grammatical errors? The preceding question itself would make a horrible survey question: whenever one question asks for an answer on multiple, different topics, it should be split into multiple questions.</li><li><b>Question types</b>: Do the <a mce_href="http://blog.vovici.com/blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires" href="http://blog.vovici.com/blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires">question types</a> match the wording? Can respondents <a mce_href="http://blog.vovici.com/blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires" href="http://blog.vovici.com/blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires">answer both "yes" and "no"</a>, or do the choices not correspond to the question? This frequently happens when a series of questions has the same choices or scale, where one or more questions is force-fitted into a method that doesn’t work.</li><li><b>Scale consistency</b>: Are related questions using the same scale? I just edited a questionnaire this week that had two <a mce_href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales" href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales">different satisfaction scales</a>, because one was written off the top of the survey author’s head and the other was copied from the question library. Do different scales arrange items consistently from best to worst or worst to best? I once had to ignore a question in analysis because it used a 1-5 rating scale with 1=best, 5=worst, in reverse of all earlier questions; the open-ended responses made it clear that some customers used the scale as written, and others used the scale as expected based on the earlier scales. (Yet another reason to <a mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">avoid numbers in favor of labels in rating questions</a>.)</li><li><b>Answer validation</b>: Is the survey configured to enforce the validation described in each question? For instance, making sure an email address is in the format <span class="Apple-style-span" style="text-decoration: underline; ">jane@example.com</span>, that that a <a mce_href="http://blog.vovici.com/blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires#fillintheblank" href="http://blog.vovici.com/blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires#fillintheblank">fill-in-the-blank question</a> is limited to numbers or that a <a mce_href="http://blog.vovici.com/blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices" href="http://blog.vovici.com/blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices">choose-many question</a> has a limit to the number of choices that can be selected</li><li><b>Required answers</b>: Are required answers used sparingly but appropriately, especially for critical questions and for questions that drive skip patterns? For <a mce_href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" href="http://blog.vovici.com/blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">closed-ended questions</a> that are required, is there an appropriate choice in each case, such as “Don’t know”, “Can’t remember” or “Not applicable”?</li><li><b>Skip patterns</b>: Do the <a mce_href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys" href="http://blog.vovici.com/blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys">skips and conditional branches</a> take respondents where we intended? Editing a question can sometimes delete or invalidate skip logic.</li><li><b>Errors of omission</b>: What questions did you leave out that you should have included?</li></ul></div><h4>Pre-Test</h4><p>That last question is particularly hard to answer in a self-test. When I am feeling very unsure of a study, or the results are strategic rather than tactical, I will pre-test it on coworkers or, even better, on a small sample of the target audience (no more than 50). I will end the survey with some questions about the survey itself, to identify areas or survey structures that were confusing or ambiguous. </p><h4>Pilot Test</h4><div>Sometimes after a pre-test, I will pilot-test the survey to 10% of the participant list, in a “shakedown cruise” of what one favorite client describes as the “final draft but not the final <i>final </i>draft” of the questionnaire. This gives even more opportunities to catch errors before the survey goes live to the full list. </div><h4>Publish</h4><div>Ok, now you can publish the survey and invite one billion people to complete it. You still missed something—trust me. Most likely something related to one of your last-minute changes. But you’ve dramatically lowered your odds of missing something major.</div><div> </div><div>Take it from hard-won experience: If your CEO or the CEO of your client cares about this survey, you definitely want to make sure you self-test, pre-test and pilot-test before you publish. </div><div> </div><div>And that’s why Vovici surveys go into test mode first. Now fill up the tank and go on a test drive before that road trip.</div> </article> <article> <h1>Forrester Loyalty Metrics</h1> <p>Jeffrey Henning — Tue, 23 Jun 2009 09:18:00 GMT</p> <DIV>Often when people talk about “<A href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx" mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx ">customer satisfaction</A>”, they really are interested in “customer loyalty”. While <A href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question" mce_href="http://blog.vovici.com/blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question">CSAT</A> has been a commonly used customer-satisfaction question for years, different organizations often use quite different loyalty measures, depending on their industry and past experience.</DIV> <DIV> </DIV> <DIV>When considering loyalty metrics, <A href="http://www.forrester.com/" target=_new mce_href="http://www.forrester.com">Forrester Research</A> is a useful firm to start with, because Forrester doesn’t provide one overriding loyalty index but instead looks at three questions to determine types of loyalty behavior:</DIV> <DIV> <OL> <LI>Willingness to consider the provider for another purchase</LI> <LI><A href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" mce_href="http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Likelihood to recommend</A> the provider to a friend or colleague </LI> <LI>Reluctance to switch business away from the provider</LI></OL></DIV> <DIV>The actual question text for each is proprietary to Forrester and has not been published. The following are generic versions of these questions.</DIV> <DIV> <OL> <LI><B>Willingness to repurchase</B>: How likely are you to repurchase from us? <I>Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely</I></LI> <LI><B>Likelihood to recommend</B>: How likely is it that you would recommend us to a friend or colleague? <I>Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely</I></LI> <LI><B>Reluctance to switch</B>: How reluctant are you to switch from us to another provider for these products or services? <I>Not at all reluctant, Slightly reluctant, Somewhat reluctant, Moderately reluctant, Very reluctant</I></LI></OL></DIV> <DIV>For each measure, Forrester looks at the percent that answer 4 or 5 (“Moderately *” or “Very *” in the scales above). Bruce Temkin has published some <A href="http://experiencematters.wordpress.com/2009/05/26/retailers-lead-tv-service-providers-lag-in-loyalty/" target=_new mce_href="http://experiencematters.wordpress.com/2009/05/26/retailers-lead-tv-service-providers-lag-in-loyalty/">Forrester loyalty results</A> using these measures on his blog.</DIV> <DIV> </DIV> <DIV>To speed up the questionnaire for respondents, you can use the same scale as the prior two questions by inverting the last question:</DIV> <BLOCKQUOTE class=webkit-indent-blockquote style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0px; BORDER-TOP: medium none; PADDING-LEFT: 0px; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; BORDER-LEFT: medium none; PADDING-TOP: 0px; BORDER-BOTTOM: medium none">3. How likely are you to switch from us to another provider for these products or services? <I>Not at all likely, Slightly likely, Somewhat likely, Moderately likely, Very likely</I></BLOCKQUOTE> <DIV>If you do that, then for consistency of analysis with the other two measures, reverse the coding of this one question so that “Very likely” is 1 and “Not at all likely” is 5.</DIV> <DIV> </DIV> <DIV>These measures are unusual in that Forester elected not to roll them up into a <A href="http://blog.vovici.com/blog/bid/18157/Multi-Item-Scale-Index-Construction-in-Survey-Research" mce_href="http://blog.vovici.com/blog/bid/18157/Multi-Item-Scale-Index-Construction-in-Survey-Research">multi-item scale</A> but kept them separate. Next time around I’ll look at a loyalty index developed by a Vovici partner.</DIV> <DIV><BR></DIV> </article> <article> <h1>Unpopular Posts from "Voice of Vovici"</h1> <p>Jeffrey Henning — Mon, 22 Jun 2009 12:51:00 GMT</p> <p><a href="http://en.wikipedia.org/wiki/Popular_%28song%29"></a>So in the recent redesign of this blog we added a <i>Posts by Popularity</i> sidebar. I’m glad for the addition, as that it always one of my favorite pages on news sites. That said, it reminds me of grade school. </p><p>I was the youngest in class, small for my age, and was always picked last in gym class by the <a href="http://en.wikipedia.org/wiki/Bombardment_Game#Bombardment" target="_new" mce_href="http://en.wikipedia.org/wiki/Bombardment_Game#Bombardment">Bombardment</a> captains. I recall the one time I got to be captain, and I picked all the unathletic kids first. Yeah, we lost. The game was still fun.<br></p><p>In that spirit, here are the unpopular posts of this blog.<br></p><ul><li><a href="http://blog.vovici.com/blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department/" mce_href="http://blog.vovici.com/blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department/">Rethinking the Role of the MR Dept.</a> – In these times of change, MR departments need to mentor and assist fellow employees who are doing their own surveys using <a href="http://www.vovici.com" mce_href="http://www.vovici.com">survey software</a>. This can be a jarring change from how such departments operate today.</li><li><a href="http://blog.vovici.com/blog/bid/18111/Market-Research-Blogs/" mce_href="http://blog.vovici.com/blog/bid/18111/Market-Research-Blogs/">Market Research Blogs</a> – Yes, I realize blogs are passé now and Twitter is where the action is: this article lists the blogs and Twitter accounts of a number of MR professionals that I read.</li><li><a href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community/" mce_href="http://blog.vovici.com/blog/bid/17991/MROC-Market-Research-Online-Community/">MROC</a> – Market-research online communities are popular, even if calling them <i>MROCs</i> isn’t. This is an abstract of Brad Bortner’s white paper on “Web 2.0 and MR” and includes my suggestion of one way to position MROCs.</li><li><a href="http://blog.vovici.com/blog/bid/18071/Unsubscribe-Survey-Templates/" mce_href="http://blog.vovici.com/blog/bid/18071/Unsubscribe-Survey-Templates/">Unsubscribe Survey Questions</a> – Letting subscribers unsubscribe from email is necessary but unpopular. Since you have to do it, modify one of these templates to make sure that you are soliciting feedback to understand why people are unsubscribing.</li><li><a href="http://blog.vovici.com/blog/bid/18133/Recommended-Survey-Length/" mce_href="http://blog.vovici.com/blog/bid/18133/Recommended-Survey-Length/">Recommended Survey Length</a> – I know, short surveys are all the rage today. But sometimes a long survey is appropriate for the research task at hand, especially for surveys of major accounts.</li><li><a href="http://blog.vovici.com/blog/bid/17890/When-to-Conduct-Face-to-Face-Interviews/" mce_href="http://blog.vovici.com/blog/bid/17890/When-to-Conduct-Face-to-Face-Interviews/">Face-to-Face Interviews</a> – Yes, they’re unpopular because they’re expensive, but face-to-face interviews provide great insight and great experiences. If all you’ve ever done is write questionnaires to administer online, you’re missing the understanding of survey research that comes from seeing the respondent react to the questions you wrote, the wording you used and the order you put the questions in.</li></ul><p>Let the bombardment of comments begin.</p> </article> <article> <h1>Email Trigger a Key Aspect of EFM</h1> <p>Jeffrey Henning — Sat, 20 Jun 2009 09:48:00 GMT</p> <div>Email triggers highlight two key aspects of enterprise <a mce_href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx " href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx ">feedback management</a>: 1) transforming surveys from projects and integrating them into processes and 2) distributing survey data to employees. Typically, a <a mce_href="http://www.vovici.com" href="http://www.vovici.com">web survey</a> of customers is set up with triggers or alerts that are fired off to staff each time a respondent gives a low rating to <a mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx " href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx ">customer satisfaction</a> or loyalty questions.</div><div> </div><div>For instance, through an automated process, each call to a contact center generates a follow-up email survey within 24 hours of the issue being marked as resolved. This survey is short and—behind the scenes, thanks to <a mce_href="http://www.vovici.com/crm-connector/feedback-data-integration.aspx " href="http://www.vovici.com/crm-connector/feedback-data-integration.aspx ">CRM integration</a>—includes data about the transaction being rated. If the customer rates the service poorly on one of several key measures, an email is triggered: this notification of a poor rating is sent to a contact center manager, and includes within it the customer’s answers to the survey as well as data about the customer, product line and call-center transaction.</div><div> </div><div> </div><div> </div><div>By receiving this notification moments after the customer has completed the survey, the manager is able to begin customer recovery with a call or email. The actual manager notified might vary depending on other fields contained within the survey, such as location of the contact center or location of the customer.</div><div> </div><div>Of course, the survey response is compiled and aggregated for reporting purposes, as with a traditional survey. However, thanks to the user of email triggers, measuring satisfaction has been transformed into intervening to improve that satisfaction.</div><div> </div><div>See also: <a mce_href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" href="http://blog.vovici.com/blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">Survey Alerts (Trigger Emails) Improve Satisfaction</a></div> </article> <article> <h1>Custom Scale Development</h1> <p>Jeffrey Henning — Fri, 19 Jun 2009 09:35:00 GMT</p> <div>Given the importance of <a mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point " href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point ">labeling each point in a scale</a>, how do you decide the right labels to use in those circumstances when no <a mce_href="http://blog.vovici.com/vovici_blog/2009/05/common-scales-for-questions.html" href="http://blog.vovici.com/vovici_blog/2009/05/common-scales-for-questions.html">common rating scale</a> is appropriate?</div><div> </div><div>If you are simply developing a list of choices to a choose-one question, and those choices have no relative relationship to one another, than you are not developing a rating scale. When you report on these choices, you will simply report on the frequency with which each choice was selected and highlight the most frequently selected choices. Use these <a mce_href="http://blog.vovici.com/vovici_blog/2009/01/openended-questions-vs-closedended-questions.html" href="http://blog.vovici.com/vovici_blog/2009/01/openended-questions-vs-closedended-questions.html">choose-one best practices</a> to come up with the appropriate choice list.</div><div> </div><div>For a rating scale, on the other hand, you want each label to represent a standard interval from one another. You plan on reporting on the arithmetic mean of the answers, not just choice frequencies, and you may try to discover correlations between the numeric rating and other variables in the survey. How to label the scale depends in part on whether the scale is unipolar (ranging from 0% to 100% of a property) or bipolar (where the zero point is in the middle and the end points are opposites, such as “completely dissatisfied” and “completely satisfied”).</div><div> </div><div>If you are developing a unipolar rating scale, use a five-point numeric scale such as 0 to 4 or 1 to 5, choosing a label for each point. A common approach to unipolar scales follows this wording:</div><div><ul><li>Not at all cromulent</li><li>Slightly cromulent</li><li>Moderately cromulent</li><li>Very cromulent</li><li>Extremely cromulent</li></ul></div><div>For a bipolar rating scale, use a seven-point scale ranging from -3 to 3, choosing a label for each point. Bipolar rating scales are easier to write, as the wording should be in parallel for positive and negative items with the same absolute value. For instance, for measuring satisfaction, a good bipolar scale is:</div><div><ul><li>Completely dissatisfied</li><li>Mostly dissatisfied</li><li>Somewhat dissatisfied</li><li>Neither satisfied nor dissatisfied</li><li>Somewhat satisfied</li><li>Mostly satisfied</li><li>Completely satisfied</li></ul>Purists typically insist that the midpoint take the form “Neither satisfied nor dissatisfied” but others prefer to label the midpoint “Neutral” for succinctness.</div><div> </div><div>If you want to use a different word or phrase for each label, take care that the words are approximately equally apart. For instance, Jon Krosnick and Leandre Fabrigar in “<a mce_href="http://smr.sagepub.com/cgi/content/refs/37/3/393" href="http://smr.sagepub.com/cgi/content/refs/37/3/393">Designing rating scales for effective measurement in surveys</a>” summarize the results of four studies into scale values for labels assessing liking.</div><div> </div><div> </div><div>Clearly, the scale “Very Poor, Poor, Fair, Good, Excellent” does a good (but not excellent!) job of spacing out each label. To develop an original scale such as this requires pre-testing and is probably inappropriate for most business researchers to attempt. In those cases where no other rating scale will do, instead use a <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">five-point unipolar scale</a> with just the endpoints labeled or a seven-point bipolar scale with the endpoints and midpoint labeled. While not ideal, and against best practices, you are less likely to go wrong using such an approach than simply making up a scale of your own.</div> </article> <article> <h1>Standardization of Scales in Survey Analysis</h1> <p>Jeffrey Henning — Wed, 17 Jun 2009 18:16:00 GMT</p> <span class="Apple-style-span" style="font-size: 13px; font-weight: normal; ">While I believe that the scales with the highest reliability and validity are <a mce_href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">5-point unipolar scales and 7-point bipolar scales</a>, and I prefer using <a mce_href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" href="http://blog.vovici.com/blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">fully labeled scales</a>, that is for gathering information from respondents. Presenting information to users of your <a mce_href="http://blog.vovici.com/blog/?Tag=Survey+Research" href="http://blog.vovici.com/blog/?Tag=Survey+Research">survey research</a> is a different matter.<span style="mso-spacerun:yes"> </span>Simply presenting scales designed for accurate data collection may not facilitate ease of understanding of the results.</span><p class="MsoNormal">For instance, since 7-point scales are rare in business use, readers of your report may have difficulty understanding the results if you use a 7-point scale.<span style="mso-spacerun:yes"> </span>If you are using a mix of 5- and 7-point scales when reporting results, you are certain to confuse some of your readers.<span style="mso-spacerun:yes"> </span>If you are doing a cross-survey analysis where similar questions in different surveys used different scales (see <a mce_href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales" href="http://blog.vovici.com/blog/bid/18284/Standardize-Your-Customer-Satisfaction-Questions-Rating-Scales">Standardize Your Customer Satisfaction Questions & Rating Scales</a>), you will definitely want to standardize your scales.</p><p class="MsoNormal">For presenting data, I typically prefer to map scales to a 0-10 scale.<span style="mso-spacerun:yes"> </span>I find that the business professionals I present to intuitively understand this scale. The broader range of values also makes it easier for readers to see differences in the results.</p><p class="MsoNormal">For instance, I would typically not present the following, even though this is an accurate representation of the gathered results.</p><p class="MsoNormal"><b>Rating of Attributes on a 1-5 Scale<br>from 1 = Not at all important to 5 = Extremely important</b></p><table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="296" style="margin-left: 4.65pt; border-collapse: collapse; width: 222pt; "> <tbody><tr style="mso-yfti-irow:0;mso-yfti-firstrow:yes;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Functionality of product<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">4.7<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:1;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Product learning curve<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">4.6<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:2;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Quality of technical support<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">4.4<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:3;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Ability to grow with product line<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">3.9<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:4;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Price of product<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">3.6<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:5;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Availability of free trial<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">3.5<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:6;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Helpful sales representative<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">3.4<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:7;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Third-party reviews of product<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">2.9<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:8;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Customer list of vendor<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">2.7<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:9;mso-yfti-lastrow:yes;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Vendor’s brand name<o:p></o:p></span></p> </td> <td width="61" nowrap="" valign="bottom" style="width:46.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">2.5<o:p></o:p></span></p> </td> </tr></tbody></table><p class="MsoNormal"><o:p>On a 1-5 scale, the midpoint is 3.0, not 2.5 (which would be the midpoint of a 0-to-5 scale).<span style="mso-spacerun:yes"> </span>To calculate the midpoint, simply average the lowest and highest ratings: for instance, (1+5)/2 = 3.</o:p></p><p class="MsoNormal">Instead of presenting the above, I standardize the results to a 0- to 10-point scale.<span style="mso-spacerun:yes"> </span>Standardization from a 5-point scale is not simply a matter of doubling the answers; doing that would produce ratings from 2 to 10. The 11-point scale has more than twice the granularity of a 5-point scale.<span style="mso-spacerun:yes"> </span>Instead, map the scale to a 0-to-1 scale:<span style="mso-spacerun:yes"> </span>(X-1)/4 produces a number from 0.0 to 1.0 for each rating.<span style="mso-spacerun:yes"> </span>Then map this to a 0-10 scale, so (X-1)/4*10 results in the new scores.</p><p class="MsoNormal"><span class="Apple-style-span" style="font-weight: bold; ">Rating of Attributes on a 0-10 Scale<br>from 0 = Not at all important to 10 = Extremely important<span class="Apple-style-span" style="font-weight: normal; "> </span></span></p><table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="303" style="margin-left: 4.65pt; border-collapse: collapse; width: 227pt; "> <tbody><tr style="mso-yfti-irow:0;mso-yfti-firstrow:yes;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"></td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">0-10 Scale<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:1;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Functionality of product<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">9.2<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:2;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Product learning curve<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">9.0<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:3;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Quality of technical support<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">8.4<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:4;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Ability to grow with product line<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">7.3<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:5;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Price of product<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">6.6<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:6;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Availability of free trial<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">6.3<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:7;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Helpful sales representative<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">6.0<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:8;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Third-party reviews of product<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">4.7<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:9;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Customer list of vendor<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">4.1<o:p></o:p></span></p> </td> </tr> <tr style="mso-yfti-irow:10;mso-yfti-lastrow:yes;height:15.75pt"> <td width="235" nowrap="" valign="bottom" style="width:176.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">Vendor’s brand name<o:p></o:p></span></p> </td> <td width="68" nowrap="" valign="bottom" style="width:51.0pt;padding:0in 5.4pt 0in 5.4pt; height:15.75pt"> <p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height: normal"><span style="font-size:8.0pt;font-family:" verdana","sans-serif";="" mso-fareast-font-family:"times="" new="" roman";mso-bidi-font-family:"times="" roman";="" color:#666666"="">3.8<o:p></o:p></span></p> </td> </tr></tbody></table><p class="MsoNormal">Standardizing bipolar scales typically involves mapping them to a –X through +X range. For instance, from -3 through +3 for a seven-point scale to -10 through +10.</p><p class="MsoNormal">Remember, your job as the survey analyst is not to provide an in-depth, detailed view of all the data collected, as it was collected. <span style="mso-spacerun:yes"> </span>Giving someone a spreadsheet of all the survey responses would do that. Your job as a survey analyst is to tease out the most important information and present to it your readers in an fashion that will maximize comprehension and understanding.<span style="mso-spacerun:yes"> </span>And one tool to accomplish that is standardizing scales for presentation purposes.</p> </article> <article> <h1>Employee Engagement Survey: The Gallup Q12</h1> <p>Jeffrey Henning — Tue, 16 Jun 2009 15:27:00 GMT</p> <a href="http://www.amazon.com/12-Elements-Managing-Rodd-Wagner/dp/159562998X/ref=sr_1_5?ie=UTF8&s=books&qid=1245183667&sr=8-5"></a><div>Gallup developed its Q12 benchmark specifically to correlate its measure of employee engagement to worker productivity, customer loyalty and sales growth (see this Walker Information <a mce_href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty" href="http://blog.vovici.com/blog/bid/18074/Correlation-between-Employee-Loyalty-Customer-Loyalty">correlation between employee satisfaction and customer satisfaction</a>). Gallup consultants sifted through hundreds of questions in hundreds of surveys before choosing the twelve questions with the highest correlations to external measures. </div><div> </div><div>Topics covered include workplace expectations, supervisory relations, even working with a best friend. Each of the <a mce_href="http://www.workforce.com/section/09/article/23/53/40.html" target="_new" href="http://www.workforce.com/section/09/article/23/53/40.html">12 questions</a> is rated on a five-point scale and is one of the following four categories:</div><div><ul><li>Basic Needs – two questions</li><li>Management Support – four questions</li><li>Teamwork – four questions</li><li>Growth – two questions</li></ul></div><div>The ratings from all twelve of these questions are then combined into an index, which can be used to segment employees into three categories:</div><div><ul><li><b>Engaged </b>employees work with passion. Because they feel a strong connection to the organization, they work hard to innovate and improve.</li><li><b>Not-Engaged </b>employees do the work expected of them, but do not put in extra effort.</li><li><b>Actively Disengaged</b> employees aren’t just unhappy, but are spreading their unhappiness to other staff. </li></ul></div><div>Nationally, in 2005, engaged employees made up 28% of the work force globally, not-engaged employees made up 54%, and actively disengaged made up 17%. Contrast this with the <a mce_href="http://blog.vovici.com/blog/bid/18070/An-Apostle-Model-for-Employee-Loyalty" href="http://blog.vovici.com/blog/bid/18070/An-Apostle-Model-for-Employee-Loyalty">Walker employee loyalty model</a>. </div><div> </div><div>The Q12 database, with 5.4 million responses, is by far the largest employee benchmark available. Gallup clients can benchmark their organizations employee-engagement levels against research across 620,000 workgroups, 504 organizations, 16 major industries and 137 countries. </div><div> </div><div>Gallup backs up its benchmarking with a full human-resource consulting program to help your organization use the results to improve your organization’s employee-engagement levels. Best Buy, International Paper, Swissôtel and B&Q are some of the notable subscribers to the Q12 benchmark.</div><div> </div><div>Gallup is for the most part a well accepted benchmark. Some constructive criticisms: </div><div><ol><li>It is unlikely that these twelve questions have equal value to every organization. For instance, one large government organization found that only five of the 12 questions differentiated the best workgroups (the top 10%) from the bottom 90%; other questions might have been more appropriate for them to examine.</li><li>Not all measures are actionable: for instance, the measure relating to having a best friend at work is not actionable, as there is little an organization can do to provide a best friend (buy every employee a company-owned dog?!).</li><li>Little research has been done outside Gallup to independently attest to the predictive validity of the measures used.</li></ol></div><div>A regular employee-pulse survey such as the Q12 is an important part of an overall <a mce_href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx" href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx">employee satisfaction</a> program and, for large organizations, should be fielded to a <a mce_href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/blog/bid/18107/Random-Sampling-Explained">random sample</a> of employees on a monthly or quarterly basis. Such surveys should be complemented with in-depth employee satisfaction research, offering every employee the chance to respond on a rotating basis at least once during the year.</div> </article> <article> <h1>Voice of the Customer Definition</h1> <p>Jeffrey Henning — Mon, 15 Jun 2009 16:06:00 GMT</p> <div><b>voice of the customer</b> (abbreviation <b>VOC</b>) – <i>noun</i></div><div><ol><li>The wants and needs of customers expressed with the customer’s own language. “Let’s get rid of ‘opinioneering’ and start building cars that meet the <i>voice of the customer!</i>” - <a mce_href="http://scitation.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=JCEMD4000129000003000314000001&idtype=cvips&gifs=yes" target="_new" href="http://scitation.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=JCEMD4000129000003000314000001&idtype=cvips&gifs=yes"> Neil Eldin and Verda Hikle</a>, 1987.</li><li>The perspective of the customer. “Ms MacFarlane is responsible for representing the <i>voice of the customer</i> and providing leadership on customer communication and support.” - <a mce_href="http://www.infomedia.com.au/Default.aspx?Page=Fy2008AnnualReport&arpage=8 " target="_new" href="http://www.infomedia.com.au/Default.aspx?Page=Fy2008AnnualReport&arpage=8 ">Infomedia Annual Report</a>, 2009.</li><li>Text or speech collected from customers through information systems including email, forums, surveys and call-center systems. “Develop the data collection and mining methodologies, statistical analysis tools, and analytics reporting package for <i>voice of the customer</i> inputs.” – job posting [now offline], 2009</li></ol></div><div>Related Post: <a mce_href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies" href="http://blog.vovici.com/blog/bid/18178/Voice-of-the-Customer-VOC-Techniques-Technologies">Voice of the Customer (VOC) Techniques & Technologies</a> </div><div> </div><div>I used the phrase “voice of the customer” last week in passing with a new colleague, who asked me for a definition. I rattled off something similar to the first definition, then went searching for a good web page to point him to. The few pages that I found had prescriptive definitions, defining the term how they want people to use it. No one actually seemed to consider how people really used it.</div><div><br></div><div>One of my favorite blogs is <a mce_href="http://languagelog.ldc.upenn.edu/nll/" target="_new" href="http://languagelog.ldc.upenn.edu/nll/">Language Log</a>, where linguists write for a general audience about various and sundry linguistic issues (one of the contributors is the author of <a mce_href="http://www.amazon.com/Eskimo-Vocabulary-Irreverent-Essays-Language/dp/0226685349" target="_new" href="http://www.amazon.com/Eskimo-Vocabulary-Irreverent-Essays-Language/dp/0226685349">The Great Eskimo Vocabulary Hoax</a>). Inspired by their coverage of how words change, and indulging my own inner lexicographer for the first time since I defined <a mce_href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template/" href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template/">firmographic</a>, let's look at how the jargon phrase “voice of the customer” has evolved.</div><div> </div><div>Google News indicates that the phrase originally grew out of the <a mce_href="http://www.qfdi.org/" target="_new" href="http://www.qfdi.org/">Quality Function Deployment</a> method of translating customer inputs into products and processes. Consider the following supporting quotes to be the voice of the customer of users of the phrase “voice of the customer”!</div><div><ul><li>1986</li><ul><li>“QFD—A Structured Approach to Understanding the Voice of the Customer” [PA Davis, IEEE Conference Proceedings, 1986 – the first occurrence I could find that didn’t literally mean a human voice]</li></ul><li>1987</li><ul><li>“Included among seven philosophical goals sought by the new program are that ‘the voice of the customer is understood and drives the whole process’ and that education and training will be continued ‘and will support the quality improvement process.’”</li></ul><li>1989</li><ul><li>“QFD, popularly called ‘the voice of the customer,’ is a team approach to product design involving representatives from the customer's organization, the manufacturer's organization, and the manufacturer's supplier's organization.”</li><li>“Specification limits or tolerances represent the ‘voice of the engineer,’ which frequently conflicts with the ‘voice of the customer.’”</li></ul><li>1990</li><ul><li>“Stempel has to create a corporate culture that turns catchy phrases like ‘listening to the voice of the customer’ into reality.”</li></ul><li>1991</li><ul><li>“The aim of TQE is to maintain employees' focus on the voice of the customer and to promote quality performance in every corporate task.”</li><li>“One of the reasons Taurus has been so successful is that its design has been guided by the voice of the customer throughout its product history.”</li><li>“Total quality brings the voice of the customer into the organization and transmits it to each link in the chain of operations.”</li></ul></ul></div><div>And some quotes from this year that show a greater diversity of meaning:</div><div><ul><li>“Chief Marketing Officers are the voice of the customer and define a vision for future organizational growth and vitality within a company.” </li><li>“My goal is to be the voice of the customer within InteliCloud. We can have the greatest product in our minds, but it's the customer that dictates the product path.”</li><li>“The Clarabridge Content Mining Platform provides Global 1000 enterprises an analytical view of text-based verbatims found in voice of the customer feedback channels such as call center notes, qualitative survey feedback, Web 2.0 content, online forums, reviews and customer warranty forms.” </li><li>“The quantitative stuff is taking the temperature twice a year with the voice of the customer. Both those surveys come in to that scorecard.” </li><li>“VOC: Voice of the customer, market research programs designed to uncover customer requirement and needs. While VOC is often considered a synonym for CEM, it is less oriented to improving day-to-day customer experiences as it is concerned with the longer view of business improvement.” [For the record, <b>VOC </b>as the abbreviation for “voice of the customer” outnumbers <b>VOTC </b>as the abbreviation by 15:1, according to Google.]</li><li>“We feel that with the three voices, you can find on Citysearch, the voice of the customer, the voice of the business owner and editor, we have restored a balance to city guides that has been lost.” </li></ul></div><div>Finally, I wonder how many words for “customer” those exceptional salespeople who can sell ice to Eskimos have?</div> </article> <article> <h1>Six Thinking Hats in Online Communities</h1> <p>Jeffrey Henning — Sat, 13 Jun 2009 11:11:00 GMT</p> <div>Kate Dibben, the Online Communications Officer of <a mce_href="http://www.educationau.edu.au/" target="_new" href="http://www.educationau.edu.au/">Education.au</a>, mentions in passing using “Six Thinking Hats” within communities in her presentation <a mce_href="http://www.slideshare.net/montgorp/online-communities-educause" target="_new" href="http://www.slideshare.net/montgorp/online-communities-educause">Making Online Communities Work</a>. It’s a good point, worthy of elaboration. If you’ve ever had important conversations in your communities run aground, then you might find the approach useful, especially in smaller private communities.</div><div> </div><div><div>Conversations in online communities, just like conversations around the water cooler, the soccer field or the bar stool, can segue from tangent to tangent to tangent. Oftentimes the journey itself is enjoyable, except when the conversation ends up back where it started, and you’ve made no progress on resolving the issue at hand. Sometimes such conversations are an awkward mix of brainstorming, negativity, new information, misplaced emotion, positive feedback and meta-discussions. This can be especially true in feedback communities, where you’re taught to value the feedback, whatever form it takes. For those times when an issue is being discussed for which you need to put in place an action plan, keep conversations on track by asking participants to take turns wearing the metaphorical Hats.</div><div> </div><div>Edward de Bono introduced the <a mce_href="http://www.amazon.com/Six-Thinking-Hats-Edward-Bono/dp/0316178314" target="_new" href="http://www.amazon.com/Six-Thinking-Hats-Edward-Bono/dp/0316178314">Six Thinking Hats in his eponymous book</a>. Here he describes each hat:</div><div><ul><li><b>White Hat</b> – “White is neutral and objective. The white hat is concerned with objective facts and figures.”</li><li><b>Red Hat</b> – “Red suggests anger (seeing red), rage and emotions. The red hat gives the emotional view.”</li><li><b>Black Hat</b> – “Black is somber and serious. The black hat is cautious and careful. It points out the weaknesses in an idea.”</li><li><b>Yellow Hat</b> – “Yellow is sunny and positive. The yellow hat is optimistic and covers hope and positive thinking.”</li><li><b>Green Hat</b> – “Green is grass, vegetation, and abundant, fertile growth. The green hat indicates creativity and new ideas.”</li><li><b>Blue Hat</b> – “Blue is cool, and it is also the color of the sky, which is above everything else. The blue hat is concerned with control, the organization of the thinking process, and the use of the other hats.”</li></ul></div><div>A moderator can open a discussion by laying out a sequence through the hats:</div></div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"> </blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;">“We’re going to start by each metaphorically wearing the White Hat and providing information that we have about this feature and how our organization uses it. After that, and only when we are done providing objective information, we will wear the Green Hat and imagine how this feature could be improved. Then we will wear the Yellow Hat and talk about the benefits of the suggestions. Then and only then will we wear the Black Hat and look at the weaknesses of what we’ve talked about. Before we wrap up, we will put on the Red Hat and talk about how the issues we’ve discussed make us feel. Finally, we will wear the Blue Hat and come to a conclusion about what we think our priorities should be.” </blockquote><div><div> </div></div><div></div><div> </div><div>As children we were told “to put on our thinking caps”, and De Bono’s Six Thinking Hats provide a simple but useful framework for helping groups think through issues. The Six Thinking Hats can be thought of as traffic cones to help you guide the conversation to its destination. Try it in your online community the next time your discussions need a little structure.</div> </article> <article> <h1>Case Study: Domino’s Pizza Transactional Survey</h1> <p>Jeffrey Henning — Fri, 12 Jun 2009 11:45:00 GMT</p> <DIV>Since the rise of web surveys, we’ve talked about real-time feedback. Domino’s Pizza is carrying it to the next level. Working late last night, I ordered a pizza online for me and a co-worker, and it actually built an image of the pizza (shown in the corner) as I added toppings. But even the survey itself changed before my eyes, as the cook and delivery person were assigned and appropriate question wording was updated dynamically!</DIV> <DIV> </DIV> <DIV>Where most restaurant sites give you a confirmation page, Domino’s gives you a Pizza Tracker: a dashboard for your pizza delivery.</DIV> <DIV>The handy progress-bar/thermometer keeps you glued to the page:</DIV> <DIV> </DIV> <DIV></DIV> <DIV> </DIV> <DIV> </DIV> <DIV>But wait, there’s a five-question survey. “Help Us Get Better” – a <A href="http://blog.vovici.com/vovici_blog/2009/04/compelling-survey-invitations.html" mce_href="http://blog.vovici.com/vovici_blog/2009/04/compelling-survey-invitations.html ">compelling subject line</A>. Then a <A href="http://blog.vovici.com/vovici_blog/2009/04/nps-net-promoter-score-criticisms-and-best-practices.html" mce_href="http://blog.vovici.com/vovici_blog/2009/04/nps-net-promoter-score-criticisms-and-best-practices.html">likelihood-to-recommend</A> question followed by a <A href="http://blog.vovici.com/vovici_blog/2009/06/customer-experience-management-with-the-forrester-cxpi.html" mce_href="http://blog.vovici.com/vovici_blog/2009/06/customer-experience-management-with-the-forrester-cxpi.html">customer-experience assessment</A> using superlative language: “We want your ordering experience to rock. How was it?” That’s a friendly, <A href="http://researchrants.wordpress.com/2009/01/27/stop-screwing-up-your-conversation/" mce_href="http://researchrants.wordpress.com/2009/01/27/stop-screwing-up-your-conversation/">conversational question</A> that sets a high bar for five-star ratings, which is what is <A href="http://blog.vovici.com/vovici_blog/2008/08/the-folk-apostl.html" mce_href="http://blog.vovici.com/vovici_blog/2008/08/the-folk-apostl.html">ideal from a measurement perspective</A>. Here’s a closeup of the survey box:</DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV>But wait, I can’t answer the other questions yet, because they know the pizza hasn’t even been delivered. I get the validation message “Can you let us know after your order arrives?” </DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV>It’s a unique problem because <A href="http://blog.vovici.com/vovici_blog/2009/03/recommended-survey-length.html" mce_href="http://blog.vovici.com/vovici_blog/2009/03/recommended-survey-length.html ">transactional surveys</A> are never delivered during the transaction. Normally transactional web surveys are sent a few hours or days later, with rules to screen out respondents who waited a week to reply, as they are assumed to no longer have a clear recollection of the transaction. Want the clearest possible recollection of the transaction? Ask them in the middle of it in an engaging fashion.</DIV> <DIV> </DIV> <DIV>The actual question itself blew me away. “Christopher custom made your order. How did everything taste?” I can’t talk some of our customers into <A href="http://blog.vovici.com/vovici_blog/2008/11/panel-managem-1.html" mce_href="http://blog.vovici.com/vovici_blog/2008/11/panel-managem-1.html">integrating their surveys with their CRM systems</A> using standard APIs, and here Domino’s built a custom application that not only infuses data for Christopher in real time (behind the screen no doubt is an employee ID) but actually updates the text of the survey once they know who it is. (I now know the guy who made my pizza by name, something that hasn’t happened to me since high school.)</DIV> <DIV> </DIV> <DIV>Asking the question with this wording was no doubt carefully thought out, but I’m of two minds about it. First, it’s clear that this rating reflects on Christopher (whose pizza making skills are excellent, by the way) and that my answer will have an impact on Christopher of Domino’s #3723 in some way. That’s good, and I like it. On the other hand, unlike the wording of the customer-experience assessment, this wording encourages ratings inflation: I really don’t want to give Christopher a low rating. I too well recall the joy of being a short order cook. (To the patron of McDonald’s #3570 in the spring of 1987 whose Quarter Pounder patty was absentmindedly cooked twice: I am <I>so </I>sorry.)</DIV> <DIV> </DIV> <DIV>And just because the staff at Domino’s are showoffs, they do it again on the third question, updating it once they know that John is the delivery person:</DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV>“How did John your delivery expert do today?” dynamically replaces the previous and overly corporate wording of “Our goal is exceptional delivery. How was your delivery experience?” (For the record, I usually like my delivery experiences with epidurals and <I>don’t-get-me-started</I> on how Ridley Scott had to have been inspired to film <I>Alien </I>by an awful delivery experience.)</DIV> <DIV> </DIV> <DIV>The last question, an <A href="http://%20blog.vovici.com/vovici_blog/2009/02/openended-questions.html" mce_href="http:// blog.vovici.com/vovici_blog/2009/02/openended-questions.html ">open-ended question</A>, is as friendly as they come, from “Use this handy box” to the request for “advice, grumblings or compliments”:</DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV>Finally, Domino’s does something pedestrian, limping across the finish line after an amazing race. The confirmation page asks me to call them if I need a response:</DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV><I>Call </I>them? Can’t I text them or something? Certainly I should call if there is something wrong with the pizza and I want it fixed this minute, but this would be an appropriate place to ask for my email address if I wanted a response. They’ve already had me give them my physical address and phone number when placing the order. Still, a minor quibble to a phenomenal transactional survey.</DIV> <DIV> </DIV> <DIV>Let’s recap:</DIV> <DIV> <UL> <LI>Domino’s took a customer-service question (“Where is my pizza?”) and transformed it into a hip, Web 2.0 opportunity to conduct a transactional survey.</LI> <LI>Domino’s made sure that its transactional survey didn’t <A href="http://blog.vovici.com/vovici_blog/2008/09/when-surveys-go.html" mce_href="http://blog.vovici.com/vovici_blog/2008/09/when-surveys-go.html">detract from the actual customer experience</A> but actually enhanced the experience.</LI> <LI>Domino’s has set themselves up for a large response rate. At a minimum, many people will rate two questions, and if they leave the web page open (thanks to the joy of tabbed browsers), then when they come back to their computer after eating their delicious pepperoni-half-onion-mushroom-pepper pizza, they will answer three more questions.</LI> <LI>Domino’s has tightly integrated their feedback platform with their operational systems, to the point where they make it clear to me by name the cook and delivery person that I am rating. (John, I should point out, was very friendly and seemed genuinely interested in what exactly Vovici does in its hallowed halls.)</LI> <LI>Domino’s demonstrated the utmost respect for respondents: the questionnaire was short and sweet and even conversational. Yet this is not a five-question survey, though respondents only have to answer five questions. Behind the scenes they integrate the feedback with the order itself and the operational data, collecting:</LI></UL></DIV> <DIV> <OL> <OL> <LI>The respondent’s phone number</LI> <LI>The address, allowing geographic analysis by city and state, and—by using the zip code—demographic analysis as well</LI> <LI>Whether this was an order sent to a home or business (specified with the address)</LI> <LI>All the attributes of the order itself</LI> <LI>The fact that they tried to upsell me the Buffalo Chicken Kickers® and I declined</LI> <LI>The time and date of the order</LI> <LI>The cook</LI> <LI>The delivery person</LI> <LI>…and some other fields I’m probably not thinking of</LI></OL></OL></DIV> <DIV>Domino’s has set itself up to do some incredibly sophisticated analysis of this <A href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx" mce_href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx ">customer feedback</A>. This is the best example of a transactional survey I’ve seen since that <A href="http://www.smartphonemag.com/cms/_archives/Nov05/dolphinquest.aspx" mce_href="http://www.smartphonemag.com/cms/_archives/Nov05/dolphinquest.aspx">whole swim-with-the-dolphins answer-a-survey-on-a-waterproof-PDA</A> thing. </DIV> <DIV> </DIV> <DIV>Domino’s, when it comes to transactional surveys, truly you are the upper crust.</DIV> </article> <article> <h1>Survey Glossary Can Clarify Your Questionnaire</h1> <p>Jeffrey Henning — Thu, 11 Jun 2009 15:16:00 GMT</p> <div>When I was a consultant, one of the most difficult challenges I faced when talking with customers was recognizing which acronyms I should know and which I shouldn’t. I still remember getting it wrong at a meeting in 1991 when I asked about a term that I should have known: not knowing that “CA” was an abbreviation for Computer Associates definitely damaged my credibility with that particular client. Fortunately, the hundreds of surveys I conducted with telecoms managers gave me a great background for our telco clients: I had a good accuracy rate there, asking them to define only those <a mce_href="http://blog.vovici.com/vovici_blog/2008/06/father-of-efm.html" href="http://blog.vovici.com/vovici_blog/2008/06/father-of-efm.html">three-letter acronyms</a> that were specific to their organizations, referring to custom MIS systems, market segments and even names of subsidiaries.</div><div> </div><div>You don’t want to make your respondents have to work so hard. They’re not trying to impress you. One of my eight recommendations for writing <a mce_href="http://blog.vovici.com/vovici_blog/2009/05/writing-objective-survey-questions.html" href="http://blog.vovici.com/vovici_blog/2009/05/writing-objective-survey-questions.html">objective questions</a> was to “avoid industry jargon and acronyms”. You’ve internalized many terms and phrases particular to your organization and industry.</div><div> </div><div>Avoiding jargon, though, is easier said than done. Sometimes the nature of the research you are doing requires a high degree of specialized terminology, terms which most of your respondents will know. For those that don’t, it is helpful to set up a survey glossary, such as the following illustration of a survey of retail employees:</div><div> </div><div> </div><div>If you set this glossary up in Vovici v4, in the questionnaire each keyword you defined will be underlined. If the respondent hovers over a keyword, the glossary entry will be displayed as a tooltip:</div><div> </div><div> </div><div>If you are using other survey applications, you can achieve this effect manually by turning keywords into hyperlinks to a popup glossary page.</div><div> </div><div>For those times when you can’t remove the jargon from your questions altogether, make sure you define and clarify your terms for your respondents.</div> </article> <article> <h1>Customer Experience Management with the Forrester CxPi</h1> <p>Jeffrey Henning — Wed, 10 Jun 2009 14:43:00 GMT</p> <p>Perhaps you could have safely ignored CEM (Customer Experience Management) three years ago, but you certainly can't now:<br></p> <ul> <li><a href="http://blog.vovici.com/Blog/bid/18191/Customer-Experience-Excellence-Why-What-and-How" class="" mce_href="http://blog.vovici.com/Blog/bid/18191/Customer-Experience-Excellence-Why-What-and-How">Customer experience</a> is growing up; it's been important to firms for long enough that your competitors are differentiating themselves on experience.</li> <li>The recession is strengthening the correlation between customer experience and customer loyalty, across all 12 B2C industries Forrester Research studied in its report "<a href="http://forrester.com/Research/Document/Excerpt/0,7211,53794,00.html" target="_blank">Customer Experience Correlates to Loyalty</a>" (published February 17, 2009). </li></ul>As CEM comes of age, most businesses will cut other areas disproportionately more than they will cut areas that impact customer experience. <br>So customer experience is more important than ever, but how should you measure your customers' perceptions of the experience you provide? The <a href="http://blog.vovici.com/Blog/bid/18271/Service-Quality-Gap-Model" class="" mce_href="http://blog.vovici.com/Blog/bid/18271/Service-Quality-Gap-Model">service quality gap model</a> is more useful for identifying areas that need improvement than for tracking overall sentiment. For this purpose, you might want to consider the Forrester CxPi, which is a succinct measure for evaluating customer experience that you can readily adapt to your own research.<br>Forrester calculates it customer-experience index using the net score for three key questions: <ol> <li><b>Usefulness</b> - Thinking about your recent interactions with these firms, how effective were they at meeting your needs? 1 - Didn't meet any of my needs to 5 - Met all of my needs</li> <li><b>Ease of Use</b> - Thinking about your recent interactions with these firms, how easy was it to work with these firms? 1 - Very difficult to 5 - Very easy</li> <li><b>Enjoyability</b> - Thinking about your recent interactions with these firms, how enjoyable were the interactions? 1 - Not at all enjoyable to 5 - Very enjoyable</li></ol> <p>The individual indexes for each measure are calculated by subtracting the percentage of a firm's customers that reported a bad experience from the percentage that reported a good experience: take the percentage of consumers who selected one of the top two choices (4 or 5) and subtract the percentage of consumers who selected the bottom two choices (1 or 2).</p> <p>To calculate the overall CxPi, Forrester Research averages the net scores for all three questions. An example:<br><br><a href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef01156ff0a4d1970c-pi.png" style="display: inline;"></a> <br><br>Forrester provides the following guidelines for judging a CxPi: </p> <ul> <li>Excellent: 85%+</li> <li>Good: 75% to 84%</li> <li>Okay: 65% to 74%</li> <li>Poor: 55% to 64%</li> <li>Very poor: <55%</li></ul> <p>What makes the CxPi an outstanding metric is its strong correlation to loyalty. Depending on industry, Forrester reports a very high correlation to <i>willingness to repurchase</i>, a high correlation to <a href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" class="" mce_href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices"><i>likelihood to recommend</i></a>, and a medium correlation to <i>reluctance to switch</i>.</p> <p>The minor disappointments with the index relate to the choice of scales. Unfortunately, the choice of scales is inconsistent: ease of use is a bipolar scale instead of a unipolar scale like <i>usefulness</i> and <i>enjoyability</i>. And scales that label each point have greater reliability and validity, especially among less educated consumers, than scales that label only the endpoints, as done by Forrester here. But those are minor quibbles. While the CxPi was only created in 2007, it has already emerged as a valuable and important index. </p> <p>Forrester makes available a complimentary copy of the <a href="http://www.forrester.com/Marketing/Campaign2/1,6538,1956,00.html" target="_blank">2008 results to the Customer Experience Index</a>. This report provides the results for 113 organizations in a dozen different industries, making it an excellent document to use for benchmark comparisons of the customer experience your organization provides.</p> </article> <article> <h1>Order Questions Logically</h1> <p>Jeffrey Henning — Tue, 09 Jun 2009 14:15:00 GMT</p> <p><a onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef011570da395f970b-pi.png"></a> Over the past 22 years, I have helped many clients write surveys, and more often than not, novice authors listed the questions in the questionnaire simply in the order in which the questions occurred to them. Good surveys have a natural flow to them, but to achieve that flow typically requires editing and reordering. <br></p> <p>By rearranging your questions to follow a standard order, your respondents will have a better experience. I often use an inverted pyramid approach, drilling down: </p> <ol> <li><a href="http://blog.vovici.com/Blog/bid/18203/Screening-Questions-May-Indicate-Need-for-Better-Survey-Profiles" class="" mce_href="http://blog.vovici.com/Blog/bid/18203/Screening-Questions-May-Indicate-Need-for-Better-Survey-Profiles">Screener questions</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions">Open-ended questions</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18268/Order-Effects-and-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18268/Order-Effects-and-Questions">General questions</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys">Specific questions</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18176/Demographic-Questions-Sample-Survey-Template" class="" mce_href="http://blog.vovici.com/Blog/bid/18176/Demographic-Questions-Sample-Survey-Template">Demographics</a> or <a href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template" class="" mce_href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template">Firmographics</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions#Followup" class="" mce_href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions#Followup">Follow-up questions</a></li></ol>You could follow the links and read each of these posts separately, but for a more cohesive view of structuring questionnaires, I'd recommend downloading a copy of the ebook <a href="http://blog.vovici.com/Blog/bid/18188/Survey-Software-Success-Free-EBook" class="" mce_href="http://blog.vovici.com/Blog/bid/18188/Survey-Software-Success-Free-EBook">Survey Software Success</a> and checking out chapter 5. </article> <article> <h1>Standardize Your Customer Satisfaction Questions & Rating Scales</h1> <p>Jeffrey Henning — Mon, 08 Jun 2009 14:04:00 GMT</p> <p>One of the reasons your organization may need <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a> is to standardize on the same question wording and scale across your products, departments and divisions. Whichever measures you decide are best for your organization, use them consistently. You want to be able to compare your <a href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" class="" mce_href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Net Promoter Scores</a>®, your <a href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation" class="" mce_href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation">ACSI Scores</a>, your <a href="http://blog.vovici.com/Blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question" class="" mce_href="http://blog.vovici.com/Blog/bid/18264/CSAT-the-Public-Domain-Customer-Satisfaction-Question">CSAT scores</a>, your <a href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model" class="" mce_href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model">Apostle Models</a> to one another. This will enable you to compare and contrast different parts of the business. Internal benchmarking can help your different groups learn from one another. </p><p>Sad to say, such standardization is the exception rather than the rule. In a review of 29 customer-satisfaction questions used by one Vovici partner, we identified eight different scales in use with nine different ways of wording the question: </p><blockquote> <table border="1" cellspacing="0"> <tbody> <tr> <td align="right"><b>Occurrences</b> </td> <td>Question Wording</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>13</b> </td> <td bgcolor="#9999cc">How would you rate your company's satisfaction with...</td></tr> <tr> <td align="right"><b>6</b> </td> <td>Please rate your satisfaction with...</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>4</b> </td> <td bgcolor="#9999cc">How satisfied are you with...</td></tr> <tr> <td align="right"><b>3</b> </td> <td>How would you describe your level of satisfaction with...</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>1</b> </td> <td bgcolor="#9999cc">Please indicate your level of satisfaction with...</td></tr> <tr> <td align="right"><b>1</b> </td> <td>Please describe your satisfaction with...</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>1</b> </td> <td bgcolor="#9999cc">In general how would your describe your level of satisfaction with...</td></tr></tbody></table></blockquote> <blockquote> <table border="1" cellspacing="0"> <tbody> <tr> <td align="right"><b>Occurrences</b> </td> <td>Scale</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>12</b> </td> <td bgcolor="#9999cc">1 (Poor) - 6 (Excellent)</td></tr> <tr> <td align="right"><b>5</b> </td> <td>1 (Not Satisfied) - 5 (Very Satisfied)</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>4</b> </td> <td bgcolor="#9999cc">1 (Not At All Satisfied) - 5 (Extremely Satisfied)</td></tr> <tr> <td align="right"><b>4</b> </td> <td>Very Dissatisfied, Dissatisfied, Somewhat Dissatisfied, Somewhat Satisfied, Satisfied, Very Satisfied, No Opinion</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>1</b> </td> <td bgcolor="#9999cc">Very Dissatisfied, Dissatisfied, Neither Satisfied Nor Dissatisfied, Satisfied, Very Satisfied</td></tr> <tr> <td align="right"><b>1</b> </td> <td>1 (Extremely Unsatisfied) - 5 (Extremely Satisfied)</td></tr> <tr> <td align="right" bgcolor="#9999cc"><b>1</b> </td> <td bgcolor="#9999cc">Extremely Dissatisfied, Slightly Dissatisfied, Neither Satisfied Nor Dissatisfied, Slightly Satisfied, Extremely Satisfied</td></tr> <tr> <td align="right"><b>1</b> </td> <td>1 - 4 [no labels specified!]</td></tr></tbody></table></blockquote> <p>Ironically, not a single rating scale followed the best practices of not showing numbers, of <a href="http://blog.vovici.com/Blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point" class="" mce_href="http://blog.vovici.com/Blog/bid/18076/Rating-Scale-Labels-Label-End-Points-or-Every-Point">labeling each point</a> and of using <a href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" class="" mce_href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">five points for unipolar scales and seven points for bipolar scales</a>.</p> <p>For individual groups that are reluctant to switch to the standard measure, allow them to run the new metrics alongside the old for a period of time so that they can understand the relationship between the two measures. Some Vovici clients have run the old and new methods side by side for a year, only retiring the old measure once year-over-year comparisons could be provided on the new measure.</p> <p>Then your staff can say, "Measure by measure, what's mine is yours, and what is yours is mine." Well, at least the English majors can, and they were probably never too fond of the old numbers anyway. </p> </article> <article> <h1>Rating Scale Labels: Label End Points or Every Point?</h1> <p>Jeffrey Henning — Fri, 05 Jun 2009 21:17:00 GMT</p> <P>Your boss says that questionnaire you're writing should ask your customers "What is your overall satisfaction with our product?" and should use a five-point scale. Sounds simple, you think to yourself, then go to write the question. Which format do you use?</P> <P align=center></P> <P align=left>Like many other tactical issues regarding questionnaire design, scale formats have been carefully studied. According to the summary of available research by Jon Krosnick and Leandre Fabrigar in "<A href="http://smr.sagepub.com/cgi/content/refs/37/3/393" target=_blank>Designing rating scales for effective measurement in surveys</A>":<BR></P> <UL> <LI>Respondents prefer rating scales with more verbal labels</LI> <LI>Respondents believe such scales provide more valid measurement</LI> <LI>Choosing a labeled choice is a more natural mental activity (not to mention <A href="http://researchrants.wordpress.com/2009/01/27/stop-screwing-up-your-conversation/" target=_blank>more conversational</A>) than selecting a number within a range</LI> <LI><A href="http://en.wikipedia.org/w/index.php?title=Longitudinal_study&oldid=282147787" target=_blank>Longitudinal</A> reliability is greater when using fully labeled scales instead of partially-labeled scales</LI> <LI>Validity, especially inter-rater validity, is greater using fully labeled scales</LI> <LI>Using fully labeled scales provides greater reliability and greater validity from respondents with low to moderate education</LI> <LI>Because numeric values can confuse respondents and affect the choices they make, it is better to omit numeric labels altogether.</LI></UL> <P align=left>Given the importance of fully labeling a rating scale, choose an existing <A href="http://blog.vovici.com/vovici_blog/2009/05/common-scales-for-questions.html"><B>common scale</B></A> where possible, rather than writing your own scale. Reword the question if necessary to fit a common scale. And, of course, take care when deciding <A href="http://blog.vovici.com/vovici_blog/2009/03/rating-scale-comparison-weighing-different-scales-for-survey-research.html"><B>how many points to use within the scale</B></A>.</P> </article> <article> <h1>Service Quality Gap Model</h1> <p>Jeffrey Henning — Thu, 04 Jun 2009 12:12:00 GMT</p> The gap model (also known as the "5 gaps model") of service quality is an important customer-satisfaction framework. In "<a href="http://www.jstor.org/pss/1251430" target="_blank">A conceptual model of service quality and its implications for future research</a>" (<i>The Journal of Marketing</i>, 1985), A. Parasuraman, VA Zeitham and LL Berry identify five major gaps that face organizations seeking to meet customer's expectations of the customer experience.<br><br><a onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef011570bf4d90970b-800wi.png"> <div style="text-align: center;"></div></a><br><br>The five gaps that organizations should measure, manage and minimize: <ul> <li>Gap 1 is the distance between what customers expect and what managers think they expect - Clearly <a href="http://blog.vovici.com/Blog/?Tag=Survey+Research" mce_href="http://blog.vovici.com/Blog/?Tag=Survey+Research">survey research</a> is a key way to narrow this gap.</li> <li>Gap 2 is between management perception and the actual specification of the <a href="http://blog.vovici.com/Blog/bid/18191/Customer-Experience-Excellence-Why-What-and-How" class="" mce_href="http://blog.vovici.com/Blog/bid/18191/Customer-Experience-Excellence-Why-What-and-How">customer experience</a> - Managers need to make sure the organization is defining the level of service they believe is needed.</li> <li>Gap 3 is from the experience specification to the delivery of the experience - Managers need to audit the customer experience that their organization currently delivers in order to make sure it lives up to the spec.</li> <li>Gap 4 is the gap between the delivery of the customer experience and what is communicated to customers - All too often organizations exaggerate what will be provided to customers, or discuss the best case rather than the likely case, raising customer expectations and harming customer perceptions.</li> <li>Finally, Gap 5 is the gap between a customer's perception of the experience and the customer's expectation of the service - Customers' expectations have been shaped by word of mouth, their personal needs and their own past experiences. Routine <a href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration" class="" mce_href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration">transactional surveys</a> after delivering the customer experience are important for an organization to measure customer perceptions of service.</li></ul>Each gap in the customer experience can be closed through diligent attention from management. <a href="http://www.vovici.com/">Survey software</a> can be key to assisting management with this crucial task. </article> <article> <h1>Weapons of Mass Collaboration</h1> <p>Jeffrey Henning — Wed, 03 Jun 2009 12:06:00 GMT</p> <p><a href="http://blog.vovici.com/Portals/60483/images/6a00d83548632853ef01156fc82309970c-pi.jpg" mce_href="http://blog.vovici.com/Portals/60483/images/6a00d83548632853ef01156fc82309970c-pi.jpg"></a> Dean Wiltse's first book, <i>Weapons of Mass Collaboration</i>, is now available. In it, Dean demystifies the phenomenon of online communities and explains how businesses can derive lasting benefit from immersion in such communities. He discusses the importance of transforming customer relationships by promoting deeper engagement in a collaborative, online environment.</p> <p>Dean's market-research experience as CEO of Greenfield Online and Vovici offers him a unique viewpoint on the industry. Leading the merger of WebSurveyor and Perseus into Vovici, Dean championed the need to extend enterprise feedback management into online communities and was the executive sponsor of Vovici <a href="http://www.vovici.com/online-community-software/customer-feedback.aspx" mce_href="http://www.vovici.com/online-community-software/customer-feedback.aspx">Community Builder</a>.</p> <p>The contents of <i>Weapons of Mass Collaboration</i> include:</p> <ol> <li>Online Communities Defined: <i>A Different Kind of Neighborhood</i></li> <li>Why Build an Online Community? <i>The Benefits of Holding the Conversation at Your Place</i></li> <li>Doing It the Right Way: <i>Best Practices for Online Communities</i></li> <li>The What? and the Why? <i>The Quantitative/Qualitative Divide</i></li> <li>Bridging the Quantitative/Qualitative Divide Good Things Happen: <i>When "What" Meets "Why"</i></li> <li>Blending Online Communities and Surveys to Create a Weapon of Mass Collaboration: <i>Reaping the rewards of being focused, faster and more-frequent</i></li> <li>Becoming a Feedback Driven Organization: <i>Not if, but when?</i></li></ol> <p>The book can now be <a href="http://www.amazon.com/gp/product/1439228698/sr=1-1/qid=1243374719/ref=olp_product_details?ie=UTF8&me=&qid=1243374719&sr=1-1&seller" target="_blank" mce_href="http://www.amazon.com/gp/product/1439228698/sr=1-1/qid=1243374719/ref=olp_product_details?ie=UTF8&me=&qid=1243374719&sr=1-1&seller">purchased from Amazon</a> in hard copy. For a limited time, Vovici is making an electronic copy of the book available <a href="http://www.vovici.com/forms/promos/signup-ebook-wmc.aspx" target="_blank" mce_href="http://www.vovici.com/forms/promos/signup-ebook-wmc.aspx">for free, simply for completing this form</a>.</p> </article> <article> <h1>Order Effects and Questions</h1> <p>Jeffrey Henning — Tue, 02 Jun 2009 11:55:00 GMT</p> <P>Having looked at the <A href="http://blog.vovici.com/Blog/bid/18257/Order-Effects-Early-Choices-in-Long-Choice-Lists-Are-Selected-More-Often-than-Later-Choices" mce_href="http://blog.vovici.com/Blog/bid/18257/Order-Effects-Early-Choices-in-Long-Choice-Lists-Are-Selected-More-Often-than-Later-Choices">order effects of choices in web surveys</A> a few weeks ago, I thought it appropriate to look at the order effects of questions themselves. Does re-arranging the order of the questions affect responses?</P> <P>The market researcher would prefer that the respondent consider each question in isolation, unrelated to any questions that have been asked before. Of course, <A href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" mce_href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">respondents are not robots</A>, and earlier questions will unfortunately bring topics to mind that can "contaminate" later answers. </P> <P>The example of such contamination that I have seen in my own surveys is the order of general questions vs. specific questions. If the general question is an open-ended question, many survey authors I've worked with prefer to put it after the closed-ended question, since open-ends are harder to answer (requiring thinking and typing rather than thinking and clicking a button). But when asking the verbatim question second, you will definitely get a greater percentage of respondents talking about the previous questions.</P> <P>In the paper "<A href="http://poq.oxfordjournals.org/cgi/reprint/45/2/208.pdf" target=_blank>Effects of Question Order on Survey Responses</A>" by Sam McFarland, some respondents were asked general questions (describing their interest in politics and religion) and then specific questions (evaluating the state of the economy and the energy market) while others were asked the specific questions first. Asking the specific first increased the likelihood that respondents would report an interest in the general questions. </P> <BLOCKQUOTE> <TABLE border=1 cellPadding=5> <TBODY> <TR> <TD><BR></TD> <TD>Test A</TD> <TD>Test B</TD></TR> <TR> <TD>Question Order</TD> <TD>1. General<BR>2. Specific</TD> <TD>1. Specific<BR>2. General</TD></TR> <TR> <TD>General Results</TD> <TD>Control</TD> <TD>Greater interest<BR>in specific items</TD></TR> <TR> <TD>Specific Results</TD> <TD>No change</TD> <TD>No change</TD></TR></TBODY></TABLE></BLOCKQUOTE> <P>As a result, my preference continues to be to ask an <A href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions" mce_href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions">open-ended question first</A> about how an organization can improve a product or service, then follow up with a <A href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" mce_href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">closed-ended question</A> presenting a range of items to be rated.</P> <P>Because of the ability for early questions to contaminate later questions, sometimes one question order for every respondent is the wrong approach. When asking a respondent to rate two or more contrasting items (typically products, services or organizations), it is customary to rotate the order of the items, so that the consistent assessment of one item before another doesn't introduce any bias into the results. In <A href="http://www.vovici.com/">survey software</A>, this is typically accomplished by setting up page rotations that randomly rotate pages or other blocks of questions. This is analogous to randomizing choices in a choice list.</P> <P>A free subscription to this blog to the first person with an RSS reader who can tie the photo to this topic! (Clearly I need to hire the MR blogger <A href="http://zebrabites.com/" target=_blank>Zebra Bites</A> as a photo consultant.)</P> </article> <article> <h1>Marketing in a Web 2.0 World</h1> <p>Jeffrey Henning — Mon, 01 Jun 2009 11:46:00 GMT</p> <a href="http://www.slideshare.net/Vovici/marketing-in-a-web-20-world?type=presentation" title="Marketing In A Web 2.0 World" mce_href="http://www.slideshare.net/Vovici/marketing-in-a-web-20-world?type=presentation">Marketing In A Web 2.0 World</a>:<br> <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=marketinginaweb2dot0world-090320123913-phpapp01&stripped_title=marketing-in-a-web-20-world"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> <br>View more <a href="http://www.slideshare.net/" mce_href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici" mce_href="http://www.slideshare.net/Vovici">Vovici</a>. </article> <article> <h1>CSAT, the Public Domain Customer-Satisfaction Question</h1> <p>Jeffrey Henning — Fri, 29 May 2009 11:41:00 GMT</p> <p>Many organizations use the following question, often called CSAT, to measure <a href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx">customer satisfaction</a>: </p><blockquote><p>What is your overall satisfaction with our company?<br>1. Very dissatisfied<br>2. Somewhat dissatisfied<br>3. Neither satisfied nor dissatisfied<br>4. Somewhat satisfied<br>5. Very satisfied </p></blockquote><p>This question has the twin advantages of brevity and familiarity; it is recognized and easily answered by most respondents. Any adult who has taken a survey has most likely answered a form of this question before.</p><p>CSAT is traditionally analyzed by tracking over time the percentage of "Satisfied" respondents, i.e., the percent who answer 4 or 5. Many organizations are happy to see that 70-80% of their customers are satisfied and feel little sense of urgency to make improvements so that satisfaction exceeds this level. When phrased as above, the satisfaction question ignores significant research into how to structure rating scales to provide the greatest reliability and validity (see the abstract of "The Optimal Length of Rating Scales to Maximize Reliability and Validity" by Jon Krosnick and Alex Tahk, which studied 706 tests). The CSAT question should instead be asked in one of the two following ways, with no numbers presented: </p><blockquote><p>What is your overall satisfaction with our company?</p><ul><li>Not at all satisfied</li><li>Slightly satisfied</li><li>Moderately satisfied</li><li>Very satisfied</li><li>Completely satisfied </li></ul></blockquote><p>Or:</p><blockquote><p>What is your overall satisfaction with our company? </p></blockquote><blockquote><ul><li>Completely dissatisfied</li><li>Mostly dissatisfied</li><li>Somewhat dissatisfied</li><li>Neither satisfied or dissatisfied</li><li>Somewhat satisfied</li><li>Mostly satisfied</li><li>Completely satisfied </li></ul></blockquote><p>Mere satisfaction alone is not enough: the key top-line number is the percent of respondents reporting themselves to be "Completely satisfied". This is a far more important metric. In the seminal paper, "Why Satisfied Customers Defect" by Thomas O. Jones and W. Earl Sasser, Jr. (covered in this description of the <a class="" mce_href="http://blog.vovici.com/Blog/bid/17878/The-Folk-Apostle-Model" href="http://blog.vovici.com/Blog/bid/17878/The-Folk-Apostle-Model">"folk" Apostle Model</a>), the authors report that for Xerox completely satisfied customers (rating of 5) were six times more likely to repurchase over the next 18 months than somewhat satisfied customers (ratings of 3-4).<br><br>Your first survey to report this statistic will certainly report a much lower percentage than you had hoped for, giving you a meaningful metric to track your performance against. Use the top-two percentage on the five-point scale for marketing purposes, and to compare yourself to other firms' self-reported numbers, but use the "completely satisfied" percentage to measure the results of your initiatives and innovation.</p><p>If satisfaction is just one part of the customer experience and loyalty survey you are conducting, the CSAT question can be an effective measure.</p><p>By itself, though, a single question can be very volatile from measurement period to measurement period. As a result, most professionally commissioned customer-satisfaction reports provide a customer-satisfaction index, derived from two to four questions. The <a class="" mce_href="http://blog.vovici.com/Blog/bid/18165/ACSI-American-Customer-Satisfaction-Index-Model-Strengths-and-Weaknesses" href="http://blog.vovici.com/Blog/bid/18165/ACSI-American-Customer-Satisfaction-Index-Model-Strengths-and-Weaknesses">American Customer Satisfaction Index</a> is the most famous of these and is useful if the principle concern of your research is customer satisfaction.</p> </article> <article> <h1>Survey Translations: The Translator may be a Traitor</h1> <p>Jeffrey Henning — Thu, 28 May 2009 11:35:00 GMT</p> <p><i>In March, Scott Blacker joined us as our new senior director of product management. In his career, Scott has held multiple product management positions, most recently with Rosetta Stone, the leader in language-learning software. Appropriately, then, Scott's first post is on the perils of translation and surveys.</i></p> <p>In the race for global market share, many organizations with an international customer base require surveys to be deployed in multiple languages. Typically, the survey is written in the native language of the survey author, translated into the target language(s), and then deployed. Unfortunately, this process misses a critical, often-overlooked step: back-translating a survey into the native language of the survey author.</p> <p>The reasons for skipping this step are easy enough to understand. Translation costs are expensive, and paying to both translate and back-translate a survey doubles these costs. Additionally, time demands on survey deployment are often intense, and back-translating can add valuable days to the survey deployment timeline. However, skipping this step can have serious consequences when ultimately analyzing survey response data, effectively killing the survey ROI.</p> <p>The original survey author is the subject matter expert on the topic at hand. The nuance of how a question is posed - and the specific word choices involved -matter greatly in determining the nature and validity of the final data collected. Translators may have several linguistically correct options to choose from during the translation process, but may choose a nuance that misses the original intention of the survey author. Back-translating through a second translator (who has no affiliation with the original translator) greatly reduces the likelihood of this type of error. Back translating allows the original survey author to:</p> <ul> <li>Validate the quality of the initial translation</li> <li>Ensure that the nuance of the translation matches the original intent</li> <li>Open up a dialogue with multiple translators to build a consensus around the best possible translation.</li></ul> <p>This actually happened to an associate of mine just last year. In deploying a satisfaction survey into the Japanese market, he paid a premium for a top-of-the-line translator. While the survey was linguistically correct and made perfect sense in Japanese, the translator has chosen a word for "satisfaction" that was closely aligned with "happy" in the Japanese language. When the results came, it appeared as if the product was a success - nearly 80% of Japanese indicated that they were "very happy" or "somewhat happy" with the product. </p> <p>However, when looking over some of the open-ended qualitative feedback, he realized that something was amiss. Japanese respondents had interpreted the word not as "happy", but as "fun". So yes...80% of Japanese respondents had indicated that the product was "fun" (which it was - it was a learning game), but fun in this case bore little correlation to satisfaction. Other elements, such as the ability to achieve a learning objective (after all, it was a learning game), turned out to be much more relevant to the customer's overall satisfaction.</p> <p>In this case, the problem was caught, but not before the results had been presented to the CEO. The survey had to be re-run, incurring additional costs and delaying decisions on whether a major marketing campaign should be run. It also didn't reflect well on the market research department that was responsible for deploying the survey. More seriously though, had the problem not been caught, the company might have invested millions of marketing dollars into a product that had no chance of succeeding in that market. </p> <p>As a survey author, you spend hours agonizing over diction when constructing questions in your native language...not investing the same time and resources into ensuring that the same nuance is appropriately reflected in your globally deployed surveys could cause you to fail just inches shy of the finish line.</p> </article> <article> <h1>Six-Sigma Survey Projects</h1> <p>Jeffrey Henning — Wed, 27 May 2009 11:31:00 GMT</p> <p>One of the factors that distinguishes Six Sigma from TQM (Total Quality Management) and earlier quality movements is its reliance on measurable data. Jiju Antony, in "<a target="_blank" href="http://www.improvementandinnovation.com/features/articles/pros-and-cons-six-sigma-academic-perspective">Pros and cons of Six Sigma: an academic perspective</a>", describes this difference like this:<br></p><blockquote><p>Six Sigma emphasises the importance of decision making based on facts and data rather than assumptions and hunches. Six Sigma forces people to put measurements in place. Measurement must be considered as a part of the culture change.</p></blockquote>Surveys are a key tool for transforming hypotheses and hunches about customer attitudes and outlooks into numbers and metrics. As a result, surveys are useful throughout Six Sigma work. <ul><li>Early in its own deployment of Six Sigma, Caterpillar conducted a Six Sigma Supplier Survey with its partners to understand how they had deployed Six Sigma and what lessons they had learned (see <a target="_blank" href="http://www.amazon.com/gp/product/0071385215"><i>Lean Six Sigma</i></a><i>: Combining Six Sigma Quality with Lean Production Speed</i> by Michael George).</li><li>When General Electric began its own use of Six Sigma, each GE division conducted detailed customer surveys, asking customers to rate GE products and services on CTQ (Critical To Quality) issues and to rate best-in-class performance. This evolved into a quarterly customer-satisfaction process for many divisions, with low-scoring items in the quarterly updates becoming candidates for subsequent Six Sigma projects (see <a target="_blank" href="http://www.amazon.com/gp/product/0471396737"><i>Managing Six Sigma</i></a><i>: A Practical Guide to Understanding, Assessing, and Implementing the Strategy That Yields Bottom-Line Success</i> by Forrest Breyfogle III, James Cupello and Becki Meadow).</li><li>Voice-of-the-Customer research is often conducted as an input to QFD (Quality Function Deployment), with QFD transforming customer needs into engineering and quality assurance methods for developing new, high-quality products and services.</li><li>One Six Sigma approach to web design involves an ongoing study of web-site effectiveness, which surveys visitors about their goals at the site and tracks the success rate of achieving those goals over time. Regular incremental improvements to the web site are evaluated by their effect on improving goal completion rates.</li><li>Another organization uses an employee survey to identify bottlenecks and excessive bureaucracy that reduce employee productivity, to highlight and prioritize areas for internal process improvement.</li><li>The book <i>Managing Six Sigma</i> is noteworthy among Six Sigma books because it actually practices what it preaches and includes within itself a readership satisfaction survey! The authors assert the results of this survey will help them prepare the next edition of the book.</li></ul>How have you used surveys in your Six Sigma projects? </article> <article> <h1>Common Rating Scales to Use when Writing Questions</h1> <p>Jeffrey Henning — Tue, 26 May 2009 11:21:00 GMT</p> <p>One of the most frequent mistakes I see when reviewing questionnaires are poorly written scales. Novice survey authors often create their own scale rather than using the appropriate common scale. It's hard to write a good scale; instead you are better off rewording your question slightly so that you can use one of the following.</p><table padding="5"><tbody><tr><td align="right" valign="top"><b>Acceptability</b></td><td>Not at all acceptable, Slightly acceptable, Moderately acceptable, Very acceptable, Completely acceptable</td></tr><tr><td align="right" valign="top"><b>Agreement</b></td><td>Completely disagree, Disagree, Somewhat disagree, Neither agree nor disagree, Somewhat agree, Agree, Completely agree</td></tr><tr><td align="right" valign="top"><b>Amount of Use</b></td><td>Never use, Almost never, Occasionally/Sometimes, Almost every time, Frequently use</td></tr><tr><td align="right" valign="top"><b>Appropriateness</b></td><td>Absolutely inappropriate, Inappropriate, Slightly inappropriate, Neutral, Slightly appropriate, Appropriate, Absolutely appropriate</td></tr><tr><td align="right" valign="top"><b>Awareness</b></td><td>Not at all aware, Slightly aware, Moderately aware, Very aware, Extremely aware</td></tr><tr><td align="right" valign="top"><b>Beliefs</b></td><td>Not at all true of what I believe, Slightly true of what I believe, Moderately true of what I believe, Very true of what I believe, Completely true of what I believe</td></tr><tr><td align="right" valign="top"><b>Concern</b></td><td>Not at all concerned, Slightly concerned, Moderately concerned, Very concerned, Extremely concerned</td></tr><tr><td align="right" valign="top"><b>Familiarity</b></td><td>Not at all familiar, Slightly familiar, Moderately familiar, Very familiar, Extremely familiar</td></tr><tr><td align="right" valign="top"><b>Frequency</b></td><td>Never, Rarely, Sometimes, Often, Always</td></tr><tr><td align="right" valign="top"><b>Importance</b></td><td>Not at all important, Slightly important, Moderately important, Very important, Extremely important</td></tr><tr><td align="right" valign="top"><b>Influence</b></td><td>Not at all influential, Slightly influential, Moderately influential, Very influential, Extremely influential</td></tr><tr><td align="right" valign="top"><b>Likelihood</b></td><td>Not at all likely, Slightly likely, Moderately likely, Very likely, Completely likely</td></tr><tr><td align="right" valign="top"><b>Priority</b></td><td>Not a priority, Low priority, Medium priority, High priority, Essential</td></tr><tr><td align="right" valign="top"><b>Probability</b></td><td>Not at all probable, Slightly probable, Moderately probable, Very probable, Completely probable</td></tr><tr><td align="right" valign="top"><b>Quality</b></td><td>Very poor, Poor, Fair, Good, Excellent</td></tr><tr><td align="right" valign="top"><b>Reflect Me</b></td><td>Not at all true of me, Slightly true of me, Moderately true of me, Very true of me, Completely true of me</td></tr><tr><td align="right" valign="top"><b>Satisfaction (bipolar)</b></td><td>Completely dissatisfied, Mostly dissatisfied, Somewhat dissatisfied, Neither satisfied or dissatisfied, Somewhat satisfied, Mostly satisfied, Completely satisfied</td></tr><tr><td align="right" valign="top"><b>Satisfaction (unipolar)</b></td><td>Not at all satisfied, Slightly satisfied, Moderately satisfied, Very satisfied, Completely satisfied</td></tr></tbody></table><p><br>This list follows <a class="" mce_href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">Krosnick's advice to use 5-point unipolar scales and 7-point bipolar scales</a>.</p><p>Let me know any of your favorite scales that I omitted.</p> </article> <article> <h1>Customer-Service Survey Template using NPS</h1> <p>Jeffrey Henning — Fri, 22 May 2009 11:01:00 GMT</p> <p><a href="http://blog.vovici.com/Blog/bid/18253/Net-Promoter-Score-s-Angels-Demons" class="" mce_href="http://blog.vovici.com/Blog/bid/18253/Net-Promoter-Score-s-Angels-Demons">As I mentioned last Friday</a>, I attended a webinar, "The Ultimate Question: How to Measure & Build Customer Loyalty in the Support Center", presented by Fred Reichheld on the use of the Net Promoter Score® within customer support centers. Fred reviewed the thinking behind NPS and presented this questionnaire for use as a customer-service survey:<br></p> <blockquote> <p>Considering only your <b>most recent support</b> experience, how likely would you be to recommend our <b>customer support</b> to a friend or colleague? (0 is least likely, 10 is most likely)<br>( ) 0 ( ) 1 ( ) 2 ( ) 3 ( ) 4 ( ) 5 ( ) 6 ( ) 7 ( ) 8 ( ) 9 ( ) 10</p></blockquote> <blockquote> <p>Please give your reasons for the rating above.<br>____________________________________________________________________________<br>____________________________________________________________________________<br>____________________________________________________________________________</p></blockquote> <blockquote> <p>Considering your <b>complete</b> experience with our company, how likely would you be to recommend our <b>products</b> to a friend or colleague? (0 is least likely, 10 is most likely)<br>( ) 0 ( ) 1 ( ) 2 ( ) 3 ( ) 4 ( ) 5 ( ) 6 ( ) 7 ( ) 8 ( ) 9 ( ) 10</p></blockquote> <blockquote> <p>Please give your reasons for the rating above.<br>____________________________________________________________________________<br>____________________________________________________________________________<br>____________________________________________________________________________</p></blockquote> <p>Fred described this as a bottom-up measure of NPS. He pointed out two sources of bias: the most recent transaction biases the respondent's thinking about the overall relationship, and the fact that the survey is sponsored by the firm gives scores an upward bias as respondents are less negative than they would be if the survey was anonymous.</p> <p>In contrast to this, Fred described a top-down measure of NPS as a double-blind survey of customers of the sponsor and its competitors. This top-down average score will be different because of the biases inherent in the bottom-up method. Just as accounting reports can legitimately differ, these two measures, to Fred's mind, have valid, legitimate differences.</p> <p>Given Fred's advocacy of short surveys, I was disappointed that he didn't point out that <a href="http://www.vovici.com/survey-solutions/customer-support.aspx">customer-support surveys</a> really shine <a href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration" class="" mce_href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration">when additional data is integrated</a> behind the scenes into the survey. This preserves the survey experience for respondents while enabling the survey analyst to study performance across many additional factors that affect customer loyalty, such as demographics and product ownership. This provides a much richer source of information for the organization to use to adapt and grow.</p> <p>[Net Promoter Score is a registered trademark of Fred Reichheld, Bain & Company and Satmetrix.]</p> </article> <article> <h1>Hurdles in Race to Turn Recipients into Respondents</h1> <p>Jeffrey Henning — Thu, 21 May 2009 10:52:00 GMT</p> <p><a href="http://blog.vovici.com/Portals/60483/images/6a00d83548632853ef0115706cc096970b-pi.png" class="" target="_new" mce_href="http://blog.vovici.com/Portals/60483/images/6a00d83548632853ef0115706cc096970b-pi.png"></a>You've chosen the list of people you want to invite to your survey. You still have a lot of hurdles to jump to get each recipient to actually take your survey: </p> <ul> <li>Make sure their email address is not on the <a href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes" class="" mce_href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes">unsubscribe list</a></li> <li>Keep your <a href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam" class="" mce_href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam">survey invitation from being flagged as spam</a></li> <li>Craft a <a href="http://blog.vovici.com/Blog/bid/18205/Getting-Your-Survey-Invitation-Opened" class="" mce_href="http://blog.vovici.com/Blog/bid/18205/Getting-Your-Survey-Invitation-Opened">survey subject line and From field that help get the invitation opened</a> </li> <li>Cover the <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">six points important to respondents in an invitation</a></li> <li>Write an <a href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations" class="" mce_href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations">invitation that is compelling</a> enough that recipient clicks on the link to take the survey</li></ul> </article> <article> <h1>Order Effects: Early Choices in Long Choice Lists Are Selected More Often than Later Choices</h1> <p>Jeffrey Henning — Wed, 20 May 2009 10:42:00 GMT</p> <P>We last looked at respondent behavior with the post <A href="http://blog.vovici.com/Blog/bid/18225/Long-Surveys-Turn-Respondents-into-Liars" mce_href="http://blog.vovici.com/Blog/bid/18225/Long-Surveys-Turn-Respondents-into-Liars">Long Surveys Turn Respondents into Liars</A>. Well, similarly, long choice lists turn respondents into <A href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" mce_href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">satisficers</A>, selecting a satisfactory answer rather than the optimal answer.</P> <P>Jon Krosnick and Duane Alwin in the report "<A href="http://publicdata.norc.org:41000/gss/DOCUMENTS/REPORTS/Methodological_Reports/MR045.pdf" target=_blank>An evaluation of a cognitive theory of response order effects in survey measurement</A>" provide an excellent summary of the past research that documented this behavior: </P> <BLOCKQUOTE> <P>Studies of impression formation<SUP>1</SUP>, the impact of persuasive communications<SUP>2</SUP>, sequential processing of performance information<SUP>3</SUP>, and the serial position effect<SUP>4</SUP> all suggest that when items are presented visually on "show cards," primacy effects are to be expected. This occurs for two main reasons. </P> <OL> <LI>Items presented early may establish a cognitive framework or standard of comparison that guides interpretation of later items. Because of their role in establishing the framework, early items may be accorded special significance in subsequent judgments.</LI> <LI>Items presented early in a list are likely to be subjected to deeper cognitive processing; by the time a respondent considers the final alternative, his or her mind is likely to be cluttered with thoughts about previous alternatives that inhibit extensive consideration of it. Research on problem-solving suggests that the deeper processing accorded to early items is likely to be dominated by generation of cognitions that justify selection of these early items<SUP>5</SUP>. Later items are less likely to stimulate generation of such justifications (because they are less carefully considered) and may therefore be selected less frequently.</LI></OL></BLOCKQUOTE>So, now that we know that our respondents do this, how do we address this issue when <A href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" mce_href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">constructing choice lists</A>? <UL> <LI>If a long choice list can be structured into an outline, present the choices as a <A href="http://blog.vovici.com/Blog/bid/18154/Hierarchical-Questions-Enable-Respondents-To-Choose-from-Thousands-of-Choices" mce_href="http://blog.vovici.com/Blog/bid/18154/Hierarchical-Questions-Enable-Respondents-To-Choose-from-Thousands-of-Choices">hierarchical question</A> instead.</LI> <LI>Consolidate the long choice list into a shorter list that makes fewer distinctions.</LI> <LI>For long lists that can't be modified, use randomization. While it would be too costly in a paper survey to have multiple versions of the questionnaire, each presenting choice lists in different orders, for a web survey the ability to randomize choice lists is a built-in capability of most<A href="http://www.vovici.com/"> survey software</A> and has no added cost to use. Such randomization isn't needed for long lists that respondents don't have to read; for instance, alphabetized lists of states or countries, where the respondent knows the answer without reading the choice list and is simply finding the choice in the list. Nor is randomization appropriate for <A href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research" mce_href="http://blog.vovici.com/Blog/bid/18166/Rating-Scale-Comparison-Weighing-Different-Scales-for-Survey-Research">rating scales</A>. Instead, randomize the choices for any long list that lacks an inherent order.</LI> <LI>Finally, Krosnick and Alwin advise attempting to "to increase respondent motivation in order to increase concentration and decrease satisficing. Motivation may be increased by adding special instructions informing respondents that the question they are about to answer is relatively difficult and requires extra concentration." </LI></UL> <UL></UL><SUP>1</SUP> Asch, 1946;Nisbett & Ross, 1980, p. 172-175; Anderson & Hubert, 1963; Sherif, 1935; 1936; Lingle & Ostrom, 1981; Anderson L Barrios, 1961; Dreben, Fiske, & Hastie,1979. <SUP>2</SUP> Miller & Campbell, 1959; Ronis et al., 1977; Crano, 1977; Hovland et al., 1957; Insko, 1964. <SUP>3</SUP> Jones et al., 1968. <SUP>4</SUP> Bruce & Papay, 1970; Crowder, 1969; Rundus, 1971. <SUP>5</SUP> Koriat, Lichtenstein, & Fischhoff,1980; Hoch, 1984; Klayman & Ha, 1984; Tschirgi, 1980; Wason & Johnson-Laird,1972. </article> <article> <h1>Survey Software Success: Companion Presentation</h1> <p>Jeffrey Henning — Tue, 19 May 2009 10:40:00 GMT</p> <p>This presentation on success with <a href="http://www.slideshare.net/Vovici/survey-software-success?type=presentation" title="Survey Software Success" mce_href="http://www.slideshare.net/Vovici/survey-software-success?type=presentation">survey software</a> was the inspiration for <a href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx" mce_href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx">my ebook</a>:</p> <p> <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=surveysoftwaresuccess-090320075745-phpapp02&stripped_title=survey-software-success"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> </p> <p>View more <a href="http://www.slideshare.net/" mce_href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici" mce_href="http://www.slideshare.net/Vovici">Vovici</a>.</p> </article> <article> <h1>Community Platform for Market Research</h1> <p>Jeffrey Henning — Mon, 18 May 2009 10:21:00 GMT</p> <p>According to an <a href="http://www.slideshare.net/fgossieaux/early-peak-tribalization-of-business-2009-1374829" target="_blank">early peek at the 2009 Tribalization of Business survey</a>, the top two reasons for creating an online community are "market insights/research" and "idea generation". So market research (which includes ideation) has become the most popular reason for organizations to sponsor proprietary communities, yet rarely are organizations turning to <a href="http://www.vovici.com/survey-solutions/online-communities.aspx">online community software</a> built specifically for this purpose.</p> <p>The Vovici Community Builder is one of the few community platforms designed for gathering market research. A common database coordinates information for panelists and community members across major components:</p> <ul> <li><b>Panel Management</b> - By empanelling everyone your organization can email, you can reach beyond just those who participate in your online community to build samples that are more representative of all your customers and prospects.</li> <li><b>Online Surveys</b> - The ability to build and field sophisticated questionnaires to members and nonmembers allow you to conduct detailed, extensive survey research. Community members see the surveys they have been invited to from a pane within the community, while nonmembers receive only email invitations.</li> <li><b>Business Intelligence</b> - Integrating operational data and CRM information enables you to perform sophisticated analysis.</li></ul> <p align="center"><a onclick="window.open( this.href, '_blank', 'width=800,height=560,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef0115708212e5970b-800wi.png"></a> </p> <p>Vovici Community Builder also has the features you would expect in a community platform: </p> <ul> <li>Customize communities to maintain your corporate identity <ul> <li>Add, delete, and modify pages, content, and modules using our visual interface-no coding required</li> <li>IFrames embed dynamic content from other websites or documents inside your community</li></ul></li> <li>Use a variety of social tools to connect with your audience and understand perceptions, promote co-creation, and keep a pulse on sentiment. <ul> <li>Forums - collect ideas, opinions and comments with moderated or un-moderated forums</li> <li>Suggestion Boxes - gather sentiments and ideas directly from community members</li> <li>Multimedia - add images, audio and videos directly into your community</li> <li>Blogs - communicate with members and even enable members to post comments</li> <li>Events - provide a social calendar where key events can be posted for your members to view</li></ul></li></ul> <p>For those community platforms that lack built-in feedback management, the robust API of <a href="http://blog.vovici.com/Blog/bid/18091/Research-Magazine-Review-of-Vovici-v4" class="" mce_href="http://blog.vovici.com/Blog/bid/18091/Research-Magazine-Review-of-Vovici-v4">Vovici v4</a> can be used to integrate with standing communities to provide the robust market research capabilities that are driving the usage of proprietary communities.</p> </article> <article> <h1>Net Promoter Score’s Angels & Demons</h1> <p>Jeffrey Henning — Fri, 15 May 2009 10:05:00 GMT</p> <p>There’s something devilish in the air. Wednesday night my wife and I watched a <span style="font-style: italic;">Dr. Who</span> episode involving a demon, Thursday afternoon I attended a webinar where Fred Reichheld said market researchers thought he was the devil, and this morning my son complained that his camping trip meant he was going to miss tonight's premiere of <span style="font-style: italic;">Angels & Demons</span>.</p> <p>Since you probably don’t care about my family’s viewing habits, let’s turn to Fred’s presentation. Fred showed a slide with his face superimposed onto a cartoon devil, claiming that he has become the enemy of traditional market researchers. Among the criticisms he reported:</p> <div>Net Promoter Score® oversimplifies customer loyalty</div> <div>NPS is a score that doesn’t provide solutions</div> <div>Market researchers have proprietary indices with greater validity</div> <div>NPS is hazardous to your business</div> <div>NPS does not correlate to growth</div> <div>Fred is dumb or lying</div> <p>Well, presenting an oversimplification of <a href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" mce_href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">legitimate criticism of NPS</a> is one way to attempt to inoculate your listeners from paying attention to that criticism. I thought it was amusing, but I will play along. In fact, I will play devil’s advocate and give Fred halos instead of horns for these contributions to loyalty research:</p> <div> Discovering that customer retention is a key driver of profitability (q.v., Reichheld, Frederick F. (1993), “<a href="http://harvardbusinessonline.hbsp.harvard.edu/b01/en/common/item_detail.jhtml;jsessionid=4GCNVIIICJALSAKRGWCB5VQBKE0YOISW?id=93210&referral=7855" target="_blank" mce_href="http://harvardbusinessonline.hbsp.harvard.edu/b01/en/common/item_detail.jhtml;jsessionid=4GCNVIIICJALSAKRGWCB5VQBKE0YOISW?id=93210&referral=7855">Loyalty-Based Management</a>,” <span style="font-style: italic;">Harvard Business Review</span>, 71 (2), 64–73)</div> <div> Promoting the importance of very short surveys</div> <div> Championing the need for a common loyalty measure across the enterprise</div> <div> Pointing out that a 0-10 scale is an intuitive scale for telephone research, since—unlike a 1-10 scale—the 0 is unambiguously the worst score</div> <div> Advocating the firing of staff who cheat on their customer-feedback numbers and discouraging organizations from tying that feedback to compensation</div> <div> Making NPS free</div><br> <p>That last is important. As he said: “NPS is an 'open source' system: you are all welcome to use it! You don't have to pay me to use it.” (I could have done without his next statement: “Therefore its bad business for market research, and why they call me a liar.”) But if you’ve ever gone to implement the five questions of the <a href="http://www.burke.com/Library/WhitePapers/SCI_White_Paper_low_res_pages.pdf" target="_blank" mce_href="http://www.burke.com/Library/WhitePapers/SCI_White_Paper_low_res_pages.pdf">Secure Customer Index</a>® or the one question of the <a href="http://blog.vovici.com/Blog/bid/18229/Customer-Effort-Score-A-Loyalty-Predictor-for-Customer-Service-Interactions" mce_href="http://blog.vovici.com/Blog/bid/18229/Customer-Effort-Score-A-Loyalty-Predictor-for-Customer-Service-Interactions">Customer Effort Score</a><span style="font-weight: bold; font-size: 18px; line-height: normal; font-family: 'Trebuchet MS';"><span style="font-weight: bold; font-size: 13px; line-height: normal; font-family: 'Trebuchet MS';">™</span><span style="font-weight: normal; font-size: 13px; line-height: 15px; font-family: arial;"> and realized you couldn’t because they are proprietary, you’ll appreciate that there are no such restrictions on the Net Promoter Score.</span></span></p> <p>Oh, and even if you liked the book (<span style="font-style: italic;">Angels & Demons</span>, not <span style="font-style: italic;">The Ultimate Question</span>), my advice is to skip the movie. This <a href="http://featuresblogs.chicagotribune.com/talking_pictures/2009/05/angels-demons1-12-stars.html" target="_blank" mce_href="http://featuresblogs.chicagotribune.com/talking_pictures/2009/05/angels-demons1-12-stars.html">review of <span style="font-style: italic;">Angels & Demons</span></a> said it all for me: "Hanks returns to the dullest role of his career, under the direction of Howard, who takes the material as seriously as a kidney stone on the way out." As for Fred, I’ll leave it to you to decide whether he is an angel or demon.</p><br> <div><span style="font-size: 9px; font-family: Arial;">[All trademarks are properties of their respective holders. Vovici makes no claim to any of these registered or common-law marks.]</span></div> </article> <article> <h1>Departmental Use of Feedback Management</h1> <p>Jeffrey Henning — Thu, 14 May 2009 09:29:00 GMT</p> <p><a onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef01156f89d05a970c-800wi.png"></a>Most organizations get to the point where they are <a href="http://blog.vovici.com/Blog/bid/18224/Feedback-Management" class="" mce_href="http://blog.vovici.com/Blog/bid/18224/Feedback-Management">subjecting customers to too many surveys</a> through the bottom-up adoption of <a href="http://www.vovici.com/survey-software/online-survey-software.aspx">survey tools</a>. This frequently happens in an organization where no market-research function exists or where MR concentrates primarily on strategic or outsourced projects.<br><br>With MR uninvolved, surveying starts as an individual activity, where each individual may have selected a different <a href="http://www.vovici.com/">survey software</a> application to use. Over time, these individuals begin working together, and survey research within the department turns into a collaborative effort. The requirements become increasingly complex as users' knowledge of surveying increases.<br>Each department or division often evolves through four stages in its use of survey systems: </p> <ol> <li>Survey activity using simple survey tools is widely distributed.</li> <li><i>De facto</i> survey-domain experts emerge through experience with these tools and through peer recognition. Only rarely do these experts have any formal training in market research.</li> <li>Survey development, deployment and analysis changes from a solitary effort to a collaborative approach within the department and occasionally across departments.</li> <li>The need quickly arises for more sophisticated survey systems, greater collaborative capability and wider information sharing. </li></ol>Missing from this evolution, unfortunately, is a <a href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department" class="" mce_href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department">market-research department mentoring</a> the users of these survey tools.<br>While <a href="http://vovici.com" class="" mce_href="http://vovici.com">enterprise feedback management</a> systems are intended to provide cross-departmental standardization, they are adapted as frequently for standardization within a department. Departments often have all the same problems as the enterprise does when it comes to survey research, only smaller in scale. </article> <article> <h1>Representative Web Surveys Require Good Email Lists of Customers</h1> <p>Jeffrey Henning — Wed, 13 May 2009 20:17:00 GMT</p> <p>Some organizations, such as ecommerce sites like Amazon and EBay, are fortunate in that they have the email addresses of every single one of their customers. When they invite a <a class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">random selection</a> of customers to participate in a <a href="http://www.vovici.com/">web survey</a>, they can be assured that the results are truly representative of their customer base.<br><br>Most organizations are not so fortunate. A traditional B2B vendor might have the email addresses for perhaps half of its customers. A consumer brand might have email addresses for fewer than 10% of its customers. For these organizations, web surveys with random samples are only representative of the customers for whom they have email addresses. Such organizations must be very careful when presenting their results to not generalize them to all customers. Significant differences probably exist between the group of customers for which they have email addresses and the group for which they don't, especially since email addresses are typically collected as part of "loyalty" (frequent-buyer) programs. As a result, surveys of such email lists will overstate satisfaction, repurchase likelihood and willingness to recommend.</p><p>Organizations with unrepresentative email lists need to take the following steps for <a class="" mce_href="http://blog.vovici.com/Blog/bid/18188/Survey-Software-Success-Free-EBook" href="http://blog.vovici.com/Blog/bid/18188/Survey-Software-Success-Free-EBook">web-survey success</a>: </p><ul><li>Describe the survey not as representative of all customers but of the target population for which the organization has email addresses.</li><li>Work with the marketing department to increase the percent of customers and prospects who have provided email addresses.</li><li>Conduct a telephone study or paper survey of customers for which your organization doesn't have email addresses to determine their demographics, firmographics and attitudes. This can be used to contrast this group with the group accessible by email.</li><li>Conduct more-frequent surveys of members of the customer loyalty program. Such programs typically use email for most communication. </li><li>Don't put faith in polls placed on your corporate web site; <a class="" mce_href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win" href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win">such polls are unrepresentative</a>.</li></ul>What steps have you taken to make sure that your web surveys are representative of your customer base? </article> <article> <h1>Survey Monkey Feature Grid vs. Vovici</h1> <p>Jeffrey Henning — Tue, 12 May 2009 20:14:00 GMT</p> <table border="1"> <tbody> <tr> <th>Support Model</th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Email support</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Online knowledgebase support</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Phone support</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>SAS-70 Secure datacenter hosting</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr></tbody></table> <table border="1"> <tbody> <tr> <th>Questionnaire Design </th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Survey responses</td> <td align="middle">Unlimited</td> <td align="middle">Unlimited</td> <td align="middle">Unlimited</td></tr> <tr> <td>Max # of Users</td> <td align="middle">1 Only</td> <td align="middle">1 Only</td> <td align="middle">Unlimited</td></tr> <tr> <td>Brand your own thank you page</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Save, copy and edit surveys</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Add own logo to survey</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Spell check</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Basic skip logic</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Advanced branching based on Boolean Logic </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Screenout questions</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Pre-loaded design themes</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Build a mixed-type matrix/table question (ex. choose one + choose many + fillin or essay)</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Edit question HTML directly</td> <td align="middle">Basic</td> <td align="middle">Advanced</td> <td align="middle">Advanced</td></tr> <tr> <td>Export survey to Word</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Import survey from Word</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Import/export survey to XML</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Find and replace </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Include progress bar in survey </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Rank order question type</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Hidden Questions to augment response data set</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Email Triggers based on responses</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Scoring</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Design from provided library of template surveys</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Save questions to/Use questions from template library</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Use themes from template library</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Word Processor Questionnaire Design Environment </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Conditional Choices - make a set of responses dependent on a previous question</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Multiple End Page options </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Custom URL at end of survey</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Auto-next (proceed to next page automatically when possible) </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Support for JavaScript in survey</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>In-tool side-by-side survey translation capability</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>XLIFF support for XML-based extract, translation and re-import</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Randomize answer list</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Data Piping</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Customize text of Save, Next, Submit, Back, Reset and Jump survey buttons </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Set Advanced Table Properties: column width, borders, text wrapping, color bands on odd/even rows, anchors, repeat table headings, separate N/A and DK from choice list and more </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Page Rotation and Randomization </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Hierarchical Questions </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Answer required checks/Mandatory Question </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <th>Survey Mgmt, Publishing & Invitations </th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Folder categorization of surveys</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Open participation survey</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Personalize email invitations</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>HTML email editing</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Send reminders</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Automation of reminder scheduling</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Multilingual email variations</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Store email lists </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Thank you email</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Import/Export email lists</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Pre-schedule survey launch and close</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Project Collaboration - assign a survey to multiple people</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Workflow for survey approval</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Workgroup administrator role based permission </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Allow respondents to re-enter survey</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Survey test mode </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Quota management</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Random test data generator</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Allow respondents to change their original answers </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <th>Panel Management & Segmentation</th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Panel is integrated into invitation sending</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Capture profile information on each panelist </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Segment panelists for individual survey invitations</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Track panelist survey participation</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Surveys that add panelists to the panel </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Change panelist status </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Panel basic health report</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Random selection of panelists</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Panelist filtering</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Saving panelist filter rules</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Sharing saved panelist filters</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <th>Response Collection & Data Management </th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Password protect each survey respondent (unique password per respondent)</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Ensure only one response per email address</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Distinguishes between partials and full completes</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Export responses to Excel</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Ability to download responses to SPSS</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Ability to download responses to CSV or TSV</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Response validation - make sure all responses adhere to branching rules</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Respondent data editor (edit respondent data) </td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Filter response dataset with boolean logic</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Results can be exported - then cleaned -and re-imported to the system </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <th>Reporting & Analysis </th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Download editable report in PPT or Word or PDF</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Link to live, public reports on web</td> <td align="middle">Yes</td> <td align="middle">No</td> <td align="middle">No</td></tr> <tr> <td>Download editable report in Excel</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Filtering report source for one question</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Filtering report source for entire report</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Create custom report elements</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Saving custom reports</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Display summary statistics - avg, std dev, count, sum, min/max, mode, mean</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Tables and charts are supported -options include show rows with zero counts, show coded-value counts, show not answered counts, etc</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Trend report:Shows results (summary statistics) using week, month, quarter or year as banner. The report element uses the survey submission time stamp to determine breakpoints. </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Live Instant Results Shown to Respondents</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Pie chart (2D or 3D)</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Column or Line chart</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Stacked column</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Stacked bar</td> <td align="middle">Yes</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Banner (crossbreak) definitions for comprehensive cross-tabulations </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Report on top "n" and bottom "n" score </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Correlation analysis with variance, kurtosis, simple regression </td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>2-way Cross-tabbing</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>3-way Cross-tabbing</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Show/hide empty columns/rows control</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <td>Reports access control by workgroup administrator</td> <td align="middle">No</td> <td align="middle">Yes</td> <td align="middle">Yes</td></tr> <tr> <th>Architecture, Security and API</th> <th>SurveyMonkey</th> <th>Vovici, Individual Edition</th> <th>Vovici, Enterprise Edition </th></tr> <tr> <td>Full library of web services to put data into system </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Full library of web services to take data out of system</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Role-based Security </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>User control to response tables, groups, or individual surveys </td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Regular password change control</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Online Community add-on</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr> <tr> <td>Business Intelligence add-on</td> <td align="middle">No</td> <td align="middle">No</td> <td align="middle">Yes</td></tr></tbody></table> </article> <article> <h1>Writing Objective Survey Questions</h1> <p>Jeffrey Henning — Mon, 11 May 2009 20:08:00 GMT</p> <p>The saying <i>Garbage In, Gospel Out</i> reflects our willingness to believe computer output, even if it was generated from bad input. Survey researchers are no more immune to this tendency than computer scientists, as poorly worded questions can lead to suspect results and erroneous conclusions.<br><br>We recognize bad questions when we see them in other people's surveys, but we don't recognize them as easily when we write them ourselves. Each of the following examples are adapted from public surveys: </p> <ol> <li>"How likely is it that you will attend the 2009 Expo at our new, low entry prices?"</li> <li>"The 2009 Forum will be held July 24-26 in central Iowa. Please check all the reasons why you may choose not to attend."</li> <li>"What type of involvement would you like to have with the Celebration of Nations program?"</li> <li>"What do you think of those titles? Yes, we are clearly not creative, but that's why we are asking for your help."</li></ol>So how do you write objective questions that don't bias the results one way or the other? <ul> <li><b>Your questions should use nonjudgmental wording and neutral terms</b>. Respondents should not be able to determine where you stand on any topic. (See this post for an example of how to <a href="http://blog.vovici.com/Blog/bid/17978/Humility-and-Data-Analysis" class="" mce_href="http://blog.vovici.com/Blog/bid/17978/Humility-and-Data-Analysis">research attitudes towards abortion</a>.)</li> <li><b>Don't presuppose anything (one type of leading question)</b>. For the third example above, for instance, ask instead "What type of involvement, if any, would you like to have with the Celebration of Nations program?"</li> <li><b>Don't ask </b><a href="http://www.wisegeek.com/what-are-leading-questions.htm" target="_blank"><b>other types</b></a><b> of </b><a href="http://www.busreslab.com/tips/tip34.htm" target="_blank"><b>leading questions</b></a>. (See the links for details.)</li> <li><b>Avoid so-called "double barreled questions"</b> by splitting them into two. Instead of "How would you rate our price and service?" ask "How would you rate our price?... How would you rate our service?" </li> <li><b>Remove ambiguity</b> in use of words and grammatical structures. The question "Will a bimonthly schedule make you more or less likely to renew your subscription?" is useless, since bimonthly can mean either "twice a month" or "every other month".</li> <li><b>Avoid industry jargon and acronyms</b>; too often in my own surveys I've assumed people know what I mean by <a href="http://en.wikipedia.org/wiki/Customer_relationship_management" target="_blank">CRM</a> or <a href="http://en.wikipedia.org/wiki/Total_Quality_Management" target="_blank">TQM</a> when I should have defined and described them.</li> <li><b>Specify how you use general terms</b>. If you need to map results back to industry figures, make sure you are using industry definitions. Instead of "Have you purchased a new big-screen television in the past year?" make sure you define the cut-off screen size that maps to your data (e.g., "Since January 1, 2008, have you purchased a new big screen (40-inch or larger) television?").</li> <li><b>Open-ended questions should specify a unit of measure</b>. Instead of "How far do you live from the nearest Acme store?" ask "How many minutes does it typically take you to travel to the Acme store that you typically shop at?"</li></ul> <p>In general, try to write from the respondent's perspective rather than your perspective. Don't make subtle distinctions that would be obvious to a coworker but not a customer. You spend 40, 50, 60 or more hours a week thinking about your company and its products and services; your customers don't. Have others outside the organization proofread your questions for clarity. For strategic surveys, pre-test your survey with a segment of your audience.</p> <p>Bad questions are the Garbage In, with their analysis being the Garbage Out. A question like "How likely is it that you will attend the 2009 Expo at our new, low entry prices?" will end up overstating actual likelihood to attend, while a question like the following will understate the likelihood to attend: "The 2009 Forum will be held July 24-26 in central Iowa. Please check all the reasons why you may choose not to attend." </p> <p>If you don't get the question right, you won't get the analysis right.</p> </article> <article> <h1>Customer Effort Score™: A Loyalty Predictor for Customer Service Interactions</h1> <p>Jeffrey Henning — Fri, 08 May 2009 20:00:00 GMT</p> <p>Though it was little noted at the time, in December the <a href="http://www.executiveboard.com/" target="_blank">Corporate Executive Board</a> introduced <a href="http://ir.executiveboard.com/phoenix.zhtml?c=113226&p=irol-newsArticle_Print&ID=1236803&highlight=" target="_blank">a new loyalty metric, the Customer Effort Score<sup>TM</sup></a>: </p> <blockquote> <p>Exceeding customer expectations has long been the measure of success in customer service interactions-89% of customer service executives believe that "delighting the customer" will lead to increased loyalty. New research by the Corporate Executive Board's <a href="http://www.ccc.executiveboard.com/" target="_blank">Customer Contact Council</a>, however, reveals the alarming truth: exceeding customer expectations results in virtually no loyalty gains. In fact, service and support centers have little stake in building customer loyalty at all.</p></blockquote> <blockquote> <p>"The probability that a service interaction will drive disloyalty is approximately four times greater than the chance it will create any positive loyalty impression. In other words, as a function, customer service typically plays on the ‘negative side' of the loyalty field," said Matthew Dixon, Ph.D., Managing Director of the Customer Contact Council. "Most service executives are using traditional customer satisfaction (CSAT) or the more recently popularized <a href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" class="" mce_href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">Net Promoter® Score (NPS)</a> to gauge loyalty in service interactions, but we found these metrics fail to capture the most powerful driver of disloyalty-the amount of personal effort a customer has to put into the service experience." In light of this new understanding, the Customer Contact Council has developed an original metric that is far more predictive of loyalty than either CSAT or NPS. This new metric, the Customer Effort Score<sup>TM</sup> (CES<sup>TM</sup>), is based on a single question that determines the degree of required customer effort during a service request.</p></blockquote> <p>Unfortunately, the actual wording of the single question is only available to Corporate Executive Board subscribers. The Customer Contact Council has published a presentation, "<a href="http://www.executiveboard.com/businessweek/pdf/CCC1A8IGV5%20Essay.pdf" target="_blank">Shifting the Loyalty Curve: Mitigating Disloyalty by Reducing Customer Effort</a>", which provides an introduction to the research. In a blog post, the author of the study provides additional detail about <a href="http://customerworld.typepad.com/swami_weblog/2009/01/customer-disloyalty-metric-customer-effort-scoreces.html" target="_blank">customer effort by service category and industry</a>. The blog <a href="http://everyexperiencecounts.blogspot.com/2009/02/corporate-executive-board-introduces.html" target="_blank">Every Experience Counts</a> reports on the metric in a little more detail.</p> <p>The selection of the CES metric was derived from a survey using a convenience sample of almost 18,000 customers of CEB clients. CES suffers from <a href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices" class="" mce_href="http://blog.vovici.com/Blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices">similar problems to NPS</a>: it's proprietary and the data used for deriving it is not publicly available, making it difficult for third parties to verify the claims about the methodology. That said, I would encourage any contact center managers who are already CEB customers to check out the full report. It requires little effort (ahem) to integrate the Customer Effort Score into existing transactional surveys, which certainly makes it worth piloting.</p> <p>Most surprising to me was the finding that 89% of customer-service executives look to customer satisfaction as the primary driver of customer loyalty. After all, it was over 13 years ago, Jones and Sasser published their landmark paper "<a href="http://www.hbr.org/hbsp/hbr/articles/article.jsp?articleID=95606&ml_action=get-article&pageNumber=1&ml_subscriber=true&referral=2533" target="_blank">Why Satisfied Customers Defect</a>", introducing the <a href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model" class="" mce_href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model">Apostle Model</a> and demonstrating that customer satisfaction and loyalty were orthogonal. Clearly the word is not yet out that satisfaction is but one component that drives loyalty.</p> </article> <article> <h1>Ranking Questions vs. Rating Questions</h1> <p>Jeffrey Henning — Thu, 07 May 2009 19:50:00 GMT</p> <P>Sometimes I hear clients use the words <I>ranking and rating</I> interchangeably, even though there is a distinction. The difference is simple: a rating question asks you to compare different items using a common scale (e.g., "Please rate each of the following items on a scale of 1-10, where 1 is ‘not at all important' and 10 is ‘very important'") while a ranking question asks you to compare different items directly to one another (e.g., "Please rank each of the following items in order of importance, from the #1 most important item through the #10 least important item"). Both types of questions have their strengths and weaknesses.<BR><BR>Ranking questions: </P> <UL> <LI>Guarantee that each item ranked has a unique value</LI> <LI>Take on average three times longer to answer than rating questions (Munson and McIntyre, 1979)</LI> <LI>Mentally tax respondents, requiring them to compare multiple items against one another</LI> <LI>Increase the difficulty of answering disproportionately as choices are added</LI> <LI>Limit the range of statistical analysis available.</LI></UL>Rating questions: <UL> <LI>Lead to less differentiation among items, with the possibility that a respondent rates every item identically</LI> <LI>Often have a narrow distribution of ratings, which typically fall into an upper band</LI> <LI>Accept great personal variations in rating styles (e.g., respondents who never assign the highest rating)</LI> <LI>Produce possibly spurious positive correlations due to individuals' personal variations.</LI></UL> <P>(See "<A href="http://communication.stanford.edu/faculty/Krosnick/docs/measurement%20of%20values%20in%20surveys.pdf" target=_blank>The measurement of values in surveys: A comparison of ratings and rankings</A>" by Duane Alwin and Jon Krosnick for a more technical review.)</P> <P>The mental effort required to answer a rating question is linear: the same effort is involved per item. The mental effort for a rank-order question is almost exponential - N*(N-1)/2 - since each item has to be compared to every other item. Because the effort grows rapidly as more items are added, it is commonly advised to only use ranking questions when there are seven or fewer items to compare. For longer lists, use a rating question instead or ask respondents to <A href="http://blog.vovici.com/Blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices" mce_href="http://blog.vovici.com/Blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices">select the three most important items</A> from the list.<BR><BR><A onclick="window.open( this.href, '_blank', 'width=640,height=480,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef01157072d261970b-popup"> <DIV style="TEXT-ALIGN: center"></DIV></A><BR><BR>For other cases, think about whether the items should be similar in comparison or should be very different. For instance, when asking people to rate the importance of items about why they did business with your organization or why they purchased a product, many attributes are of similar importance, making a rating scale appropriate. Alternatively, when asking people what features you should work on next, where you need to build a priority list for your development team, a ranking question is more appropriate. <P>When do you prefer to use ranking questions instead of rating questions, or vice versa?</P> </article> <article> <h1>Top 10 Reasons for Building an Online Community</h1> <p>Jeffrey Henning — Wed, 06 May 2009 19:46:00 GMT</p> A presentation on the top 10 reasons to build an <a href="http://www.slideshare.net/Vovici/top-10-reasons-to-build-an-online-community?type=powerpoint">online community</a>: <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=top10reasonstobuildanonlinecommunity-090320121659-phpapp02&stripped_title=top-10-reasons-to-build-an-online-community"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> <br>View more <a href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici">Vovici</a>.<br>Also see this related post: <a href="http://blog.vovici.com/Blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009" class="" mce_href="http://blog.vovici.com/Blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009">The Top Ten Reasons for Building an Online Community in 2009</a>. </article> <article> <h1>Long Surveys Turn Respondents into Liars</h1> <p>Jeffrey Henning — Tue, 05 May 2009 19:36:00 GMT</p> <p>Add to the list of ways to corrupt people with surveys - <a href="http://blog.vovici.com/Blog/bid/17981/Encouraging-Honesty-from-Survey-Respondents-and-Community-Members" class="" mce_href="http://blog.vovici.com/Blog/bid/17981/Encouraging-Honesty-from-Survey-Respondents-and-Community-Members">offering incentives to respondents</a> and <a href="http://blog.vovici.com/Blog/bid/18109/Employee-Compensation-Plan-Incorporating-Survey-Results" class="" mce_href="http://blog.vovici.com/Blog/bid/18109/Employee-Compensation-Plan-Incorporating-Survey-Results">bonuses to employees</a> - boring people to death with long questionnaires. Actually, you just bore them to the point where they pay less and less attention to the quality of their answers, until finally they're cheating.<br><br><a href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef011570722141970b-pi.png"></a> <br><br>When respondents first begin the questionnaire, they're trying to give you the most appropriate answer to each question (<i>optimizing</i>). A little ways into the survey and they are engaging in what Krosnick calls <a href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" class="" mce_href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior"><i>weak satisficing</i></a>: selecting the first choice that appears reasonable and answering "Yes" to be agreeable. Further into the survey and they are strong satisficing: failing to differentiate between ratings and selecting "don't know" rather than giving an opinion. After subjecting them to many pages of a questionnaire you've lost them and their good will; if the questions are required, they are just randomly selecting responses to be done with the damn thing.<br>What can you do about this? </p> <ul> <li>Remind respondents in the introduction of the importance of accurate answers</li> <li>Keep the <a href="http://blog.vovici.com/Blog/bid/18133/Recommended-Survey-Length" class="" mce_href="http://blog.vovici.com/Blog/bid/18133/Recommended-Survey-Length">questionnaire length appropriate</a> to the purpose of the survey</li> <li>Follow our six <a href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" class="" mce_href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">strategies for shortening questionnaires</a></li> <li>Rotate the order of sections of the questionnaire, so that no section suffers disproportionately from having respondents satisfice</li> <li>Minimize the use of required questions, so that the truly uninterested respondent can skip the question without fabricating an answer</li> <li>Keep the questions engaging and relevant.</li></ul>What other tips can you think of to keep respondents engaged for the duration of a long questionnaire? </article> <article> <h1>Feedback Management</h1> <p>Jeffrey Henning — Mon, 04 May 2009 19:32:00 GMT</p> <p>Ironically, large organizations seeking to better research customer experience don't even understand their customer's experience with their research.<br><br>Let's look at the feedback process from the perspective of the typical customer of a large organization: the customer is surrounded on all sides by survey requests.</p> <p align="center"></p> <p>In this example, survey invitations come from the training department, the customer service department, the marketing department and the sales department. Each department has selected its own survey tool with no central coordination of how frequently customers are invited to take part in research. In fact, one Vovici client had subjected customers to 450 different survey initiatives in the prior year. The result? </p> <ul> <li>Customers stopped taking requests for surveys seriously and responded less often</li> <li>As response rates dropped, individuals invited even more customers to take surveys</li> <li>Survey content often overlapped with that of other surveys</li> <li>Significant time and energy was invested in gathering survey results, but those results were not even shared within departments, let alone between departments</li> <li>Information was gathered inconsistently by different authors, limiting: <ul> <li>the ability to study changes in customer attitudes over time</li> <li>the ability to compare results across departments and product lines</li></ul></li> <li>Wasted investment in redundant tools and surveys</li> <li>No one had a true, holistic picture of the customer experience.</li></ul> <p>Any organization serious about understanding its customer experience needs to start by analyzing and standardizing its own research into that experience. A <a href="http://www.vovici.com/products/feedback-assessment.aspx">feedback assessment</a> is a great place to start on the path to <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a>.</p> </article> <article> <h1>Good and Bad Sampling Techniques</h1> <p>Jeffrey Henning — Fri, 01 May 2009 19:23:00 GMT</p> <p>Quick post today, contrasting good and bad sampling techniques and linking to past posts on these topics. </p> <p><b>Good Sampling Techniques</b> </p> <ul> <li>Make sure there is an equal probability of selecting any member of the target population (the first requirement of <a href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">random samples</a>)</li> <li><a href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" class="" mce_href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">Send scheduled email reminders</a> to recipients you invited, so as to maximize external selection (the second requirement of random samples)</li> <li>If you have more than 1,000 potential respondents, do not invite them all in <a href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" class="" mce_href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">an attempted census</a>, but leave names available for future surveys</li> <li>Gather enough responses to meet the <a href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys">recommended sample size</a> </li> <li>Take care to describe your surveys results as representative of the population for which you have email addresses, rather than the entire population</li></ul> <p><b>Bad Sampling Techniques</b> </p> <ul> <li><a href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win" class="" mce_href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win">Posting a link to the survey from a web site</a> results in a sample that is not representative of any group</li> <li>Conducting <a href="http://blog.vovici.com/Blog/bid/17839/Quality-What-s-Love-Got-To-Do-With-It" class="" mce_href="http://blog.vovici.com/Blog/bid/17839/Quality-What-s-Love-Got-To-Do-With-It">a survey within a short time period</a> is unrepresentative</li> <li><a href="http://blog.vovici.com/Blog/bid/17985/Disconnected-Listening-An-EFM-Case-Study" class="" mce_href="http://blog.vovici.com/Blog/bid/17985/Disconnected-Listening-An-EFM-Case-Study">Surveying everyone just because you can</a> limits future opportunities for surveys</li></ul> </article> <article> <h1>Survey Monkey vs. Vovici Individual Edition</h1> <p>Jeffrey Henning — Thu, 30 Apr 2009 14:44:00 GMT</p> <p><a href="http://www.vovici.com/personal.aspx" class="" mce_href="http://www.vovici.com/personal.aspx"></a>SurveyMonkey lacks many of the capabilities of Vovici's Individual Edition to control, protect and support your survey research. These Individual Edition capabilities include:</p> <p><b>Control</b></p> <ul> <li>Promote your brand by managing custom domain names or subdomains (e.g., <i>www.yourcompanyhere.com</i> or <i>surveys.yourcompanyhere.com</i>)</li> <li>Enforce brand recognition and personalize the look and feel of your surveys through more flexible configurations</li> <li>Save time by importing surveys directly from Microsoft Word</li> <li>Generate professional reports through one-button export to Microsoft Word or PowerPoint</li> <li>Build smarter surveys by using existing responses to tailor follow-up questions</li> <li><a href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors" class="" mce_href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors">Leverage pre-existing information</a> (e.g. history, transaction) to enhance the value of response data through hidden fields</li> <li>Help respondents understand questions through customizable tool tips</li></ul> <p><b>Protect</b></p> <ul> <li>Lock out security threats to employee and customer data in <a href="http://blog.vovici.com/Blog/bid/17820/Survey-Data-Security-and-Protection-It-should-matter-to-you" class="" mce_href="http://blog.vovici.com/Blog/bid/17820/Survey-Data-Security-and-Protection-It-should-matter-to-you">Vovici's SAS-70 certified data center</a></li> <li>Protect surveys with passwords to ensure the identity of the intended respondent</li></ul> <p><b>Support</b></p> <ul> <li>Rely on Vovici's live customer support team to get the help you need</li> <li>Leverage Vovici's web-based training to get a fast start, or refresher, on your next survey project</li></ul> <p>To learn more about Vovici Individual Edition, check out the <a href="http://www.vovici.com/demo.html">three-minute demo</a> or the <a href="http://www.vovici.com/personal.aspx">product and features page</a>. Join the customers who moved up from <a href="http://blog.vovici.com/Blog/bid/18142/Survey-Monkey-Over-a-Barrel" class="" mce_href="http://blog.vovici.com/Blog/bid/18142/Survey-Monkey-Over-a-Barrel">SurveyMonkey to Vovici</a> in migrating to a <a href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business" class="" mce_href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business">survey business platform that you won't outgrow</a>.</p> </article> <article> <h1>Getting Your Survey Invitation Opened</h1> <p>Jeffrey Henning — Wed, 29 Apr 2009 14:37:00 GMT</p> <p>Your survey <a href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam" class="" mce_href="http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam">invitation gets past the spam filters</a> and shows up in someone's email inbox. Now the recipient will have two or three items at hand to decide whether to click Delete or to open your message:<br></p> <ul> <li><b>From field</b> - The From field consists of the sending email address and the alias for that address. Nothing will get your email deleted faster than having an impersonal email address without an alias, such as - a particular favorite! - <a href="mailto:noreply@example.com">noreply@example.com</a>. Always include the alias. What type of alias works best varies by organization; examples to experiment with for your invitations include: <ul> <li>The name of an individual - The name of an employee managing the survey or commissioning the survey can be very effective.</li> <li>The name of the brand or company - While this can be impersonal, it may outperform the names of individuals for B2B vendors.</li> <li>Brand name + "Surveys" - For some organizations a From field such as "Acme Surveys" works best.</li> <li>Hybrid - Some panel companies test combinations such as "Acme Surveys & Jane Doe".</li></ul></li> <li><b>Subject line</b> - The text of the subject line will have the biggest impact: "Your Feedback Needed", "Some Quick Questions for You", "Help Us Serve You Better", "Learn What Your Peers Think About -----" are all subject lines that have been successful for survey invitations. Your organization will need to experiment to determine what works best for your audience.</li> <li><b>First line of the invitation</b> - Some email programs display the first line of the invitation in grayed-out text immediately after a bold subject line. All the more reason to pay special attention to the copy you use in <a href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations" class="" mce_href="http://blog.vovici.com/Blog/bid/18192/Compelling-Survey-Invitations">the body of your invitation</a>.</li></ul>What tactics have improved your organization's open rate on surveys? </article> <article> <h1>Net Promoter Score (NPS) Criticisms and Best Practices</h1> <p>Jeffrey Henning — Tue, 28 Apr 2009 14:27:00 GMT</p> <P>The Net Promoter Score®, popularized by Fred Reichheld in his book <I>The Ultimate Question: Driving Good Profits and True Growth</I>, is one of the simplest loyalty measures. Customers are asked "How likely is it that you would recommend us to a friend or colleague?" and then provide a rating from 0 ("Not at all likely") to 10 ("Extremely likely").</P> <P>The measure is called the "net promoter" score, because detractors are subtracted from promoters, to provide the estimate of how many more promoters than detractors the organization has. Detractors are defined as respondents rating their likelihood to recommend 6 or less, with promoters only those who rated their likelihood a 9 or 10 (respondents who selected 7 or 8 are considered neutral). The NPS measure can run from -100% (0% promoters, 100% detractors) to 100% (100% promoters, 0% detractors), with typical measures in the 30-40% range.</P> <P> </P> <P>Traditional customer-satisfaction measures typically omitted willingness to recommend, instead focusing on aspects like perceived value, customer satisfaction, corporate image, and rational and emotional commitment (see the <A href="http://blog.vovici.com/Blog/bid/18165/ACSI-American-Customer-Satisfaction-Index-Model-Strengths-and-Weaknesses" mce_href="http://blog.vovici.com/Blog/bid/18165/ACSI-American-Customer-Satisfaction-Index-Model-Strengths-and-Weaknesses">ACSI model</A>).</P> <P>Avis, HP and IBM are among the many prominent adopters of NPS. The benchmark is popular for its simplicity, and Reichheld claims it correlates to company growth. Critics contend that it doesn't, that its 11-point scale has lower predictive validity than other scales, that the segmentation of promoters/neutrals/detractors is arbitrary and that other questions may be better predictors of growth rates:</P> <UL> <LI><B>Not the Most Important Customer-Satisfaction Question</B> - "We find no support for the claim that Net Promoter is the 'single most reliable indicator of a company's ability to grow' (Netpromoter.com 2006; Nicks 2006). Although we do not have access to the raw data from which these claims were made, we were able to compare some of the exemplar cases of Net Promoter with the ACSI, which Reichheld (2004) reports does not correlate with growth. Instead, we found that when making "apples-to-apples" comparisons, Net Promoter does not perform better than the ACSI for the data under investigation... The clear implication is that managers have adopted the Net Promoter metric for tracking growth on the basis of the belief that solid science underpins the findings and that it is superior to other metrics. However, our research suggests that such presumptions are erroneous. The consequences are the potential misallocation of resources as a function of erroneous strategies guided by Net Promoter on firm performance, company value, and shareholder wealth." - Timothy L. Keiningham, Bruce Cooil, Tor Wallin Andreassen, & Lerzan Aksoy, "<A href="http://www.atypon-link.com/AMA/doi/pdf/10.1509/jmkg.71.3.39" target=_blank mce_href="http://www.atypon-link.com/AMA/doi/pdf/10.1509/jmkg.71.3.39">A Longitudinal Examination of Net Promoter and Firm Revenue Growth"</A></LI> <LI><B>Doesn't Accurately Differentiate Promoters and Detractors</B> - "The rule-of-thumb score classes proposed by Reichheld (promoters are those respondents who give a likelihood of recommendation of 9 or 10 while the detractors give 6 or less) are not supported statistically, mask important changes and potentially mislead management that there is negative NPS when this may not be the case." - <A href="http://www.marketingpower2.com/blog/journalofmarketing/2007/07/a_longitudinal_examination_of.html" target=_blank mce_href="http://www.marketingpower2.com/blog/journalofmarketing/2007/07/a_longitudinal_examination_of.html">Ken Roberts, Forethought Research Australia</A>. Further, the standard NPS question itself is unipolar (willingness to recommend) but Reichheld's analysis treats it as bipolar (willing to detract vs. willingness to promote).</LI> <LI><B>Less Accurate than Composite Index of 3 Questions</B> - "In his <I>Harvard Business Review</I> article ‘The One Number You Need to Grow', Reichheld maintained that since his tests showed propensity to recommend to be the single question that had the strongest statistical relationship to future company performance, there was no point asking any other questions in customer surveys... a single item question is much less reliable and more volatile than a composite index." - <I><A href="http://www.amazon.com/Customer-Satisfaction-customer-experience-customers/dp/0955416116" target=_blank mce_href="http://www.amazon.com/Customer-Satisfaction-customer-experience-customers/dp/0955416116">Customer Satisfaction - The customer experience through the customer's eyes</A></I>, Nigel Hill, Greg Roche and Rachel Allen, p. 7</LI> <LI><B>Fails to Predict Loyalty Behaviors</B> - "This research examines different customer satisfaction and loyalty metrics and tests their relationship to customer loyalty behaviors. The goal was to test the robustness of the customer-level analysis conducted by Reichheld and Satmetrix, which served as the foundation of their Net Promoter research. Contrary to Reichheld's assertions, the results indicate that recommend intention alone will not suffice as a single predictor of customers' future loyalty behaviors. Use of multiple indicators instead of a single predictor model performs significantly better in predicting customer recommendations and retention." - "<A href="http://www.trainingmag.com/msg/content_display/incentive/e3i0e6c70cad3fd5a6e135ff62b2633a34d" target=_blank mce_href="http://www.trainingmag.com/msg/content_display/incentive/e3i0e6c70cad3fd5a6e135ff62b2633a34d">The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Customer Retention, Recommendation, and Share-of-Wallet</A>" (Timothy L. Keiningham, Bruce Cooil, Lerzan Aksoy, Tor W. Andreassen, Jay Weiner). NPS is attitudinal rather than behavioral, measuring how many people say they would be likely to recommend, rather than how many are doing so. A large body of research indicates that claimed intention is a better reflection of present attitudes than future behavior (Bird, Ehrenberg and Barnard).</LI> <LI><B>Performs Worse than Satisfaction & Liking Questions</B> - The paper "Measuring Customer Satisfaction and Loyalty: Improving the ‘Net-Promoter' Score" by Daniel Schneider, Matt Berent, Randall Thomas and Jon Krosnick counterintuitively demonstrates that "satisfaction" and "liking" are better predictors of recommendations than "likelihood to recommend".</LI> <LI><B>Performs Worse than Other Scales</B> - Schneider et al also demonstrate that the 11-point scale has the lowest predictive value of any of the scales tested. The authors recommend a 7-point scale with labeled ends and midpoint for the willingness-to-recommend question but also recommend a bipolar scale for a reworded variant.</LI></UL> <P>As a result of this mounting criticism, only 19% of customer-feedback professionals agreed in a March 2008 survey that the NPS is a better predictor of growth than other loyalty questions and indices; 40% were neutral ("<A href="http://www.customerthink.com/blog/feedback_professionals_dont_believe_nps" target=_blank mce_href="http://www.customerthink.com/blog/feedback_professionals_dont_believe_nps">Customer Feedback Professionals Do Not Believe the NPS Claims</A>", Hayes).</P> <P>Despite these criticisms, NPS remains popular because it is well marketed, easy to understand and its model makes intuitive sense: every organization wants more promoters than detractors. </P> <P>Some suggested best practices for using the Net Promoter Score in your organization:</P> <UL> <LI>By all means, include the willingness-to-recommend question in your surveys.</LI> <LI>Use a customer loyalty index consisting of the willingness-to-recommend question with other questions relevant to your business (see <A href="http://blog.vovici.com/Blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template" mce_href="http://blog.vovici.com/Blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template">Apostle Model Best Practices</A> for one suggested index).</LI> <LI>Do not use the 11-point scale advocated by Reichheld, which is arbitrary in its assignment of promoters and detractors and has lower predictive validity than other scales. Reichheld shows flexibility as to the actual scale used; his original work showcases Enterprise Rent-A-Car, which uses a five-point scale in their research (treating 5 as promoters). Instead, use the seven-point bipolar scale recommended by Schneider, Berent, Thomas and Krosnick :</LI></UL> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote> <BLOCKQUOTE style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0px; MARGIN: 0px 0px 0px 40px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0px" class=webkit-indent-blockquote>How likely is it that you would recommend us or recommend against us to a friend or colleague?</BLOCKQUOTE></BLOCKQUOTE> <UL> <UL> <UL> <LI>Extremely likely to recommend against</LI> <LI>Moderately likely to recommend against</LI> <LI>Slightly likely to recommend against</LI> <LI>Neither likely to recommend nor recommend against</LI> <LI>Slightly likely to recommend</LI> <LI>Moderately likely to recommend</LI> <LI>Extremely likely to recommend</LI></UL></UL></UL> <UL> <LI>As Reichheld suggests, use a follow-up open-ended question to probe why a respondent selected the choice that they did.</LI> <LI>Calculate the Net Promoter Score primarily for comparison to other firms, but use an arithmetic mean of your responses for internal tracking and benchmarking, as this will provide a more stable measure over time.</LI></UL> <P>If you have time to read one white paper on the Net Promoter Score, read "<A href="http://mattberent.net/Netpromoter_-_AAPOR.pdf" target=_new mce_href="http://mattberent.net/Netpromoter_-_AAPOR.pdf">Measuring Customer Satisfaction and Loyalty: Improving the ‘Net-Promoter' Score</A>". Its authors include <A href="http://communication.stanford.edu/faculty/krosnick.html" target=_blank mce_href="http://communication.stanford.edu/faculty/krosnick.html">Jon Krosnick</A>, the foremost researcher on questionnaire design, and <A href="http://mattberent.net/" target=_blank mce_href="http://mattberent.net/">Matthew Berent</A>, Staff Survey Researcher with Intuit, which is one of the firms often cited for its use of NPS.</P> <P><B><I>Update (9/09/09): </I></B><A href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer" mce_href="http://blog.vovici.com/blog/bid/21057/Net-Promoter-Score-is-a-Misnomer">Net Promoter Score is a Misnomer</A> </P> </article> <article> <h1>Screening Questions May Indicate Need for Better Survey Profiles</h1> <p>Jeffrey Henning — Mon, 27 Apr 2009 14:20:00 GMT</p> <p>If you have ever been invited to take a <a href="http://www.vovici.com/">web survey</a>, answered two or three questions about who you are or what your purchase intent is, then had the survey end abruptly, you've encountered a screener: a set of initial questions written to screen out respondents who don't qualify for the main purpose of the study. Perhaps the survey was looking for people considering buying a Pontiac to get their reaction to today's news that GM is discontinuing the brand, or perhaps the survey was looking for reactions of NFL fans who watched the draft this weekend.<br><br>A screener is great for asking about recent actions and attitudes-information that can't be kept current in a database. Conversely, a screener is lousy if it asks basic demographic or firmographic information, since such data can and should be used to profile email lists. </p> <p>When writing your own questionnaires to survey your house list, if you find yourself asking respondents in a screener about such basic information, consider it a sign that you need to develop good profiles of your potential respondents. If you haven't before, then invite everyone on the house list to complete a profile questionnaire that you can use to target future research. If you do have a profile, but it is not meeting your current research needs, expand the profile: see <a href="http://blog.vovici.com/Blog/bid/17932/Tips-for-Defining-Community-Member-Profiles" class="" mce_href="http://blog.vovici.com/Blog/bid/17932/Tips-for-Defining-Community-Member-Profiles">Tips for Defining Community-Member Profiles</a> (its advice is as relevant to panelist profiles as it is to community members).</p> <p>Take care with using screeners when surveying customers. They've clicked through to the survey; you have their attention: skip them out to another survey instead or think about turning the screener into a <a href="http://blog.vovici.com/Blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18190/Skip-Logic-Conditional-Branches-in-Surveys">branching pattern</a> and asking other questions. Otherwise your respondents who don't pass the screener may get the impression their feedback isn't valuable.</p> </article> <article> <h1>Employee Community Case Studies</h1> <p>Jeffrey Henning — Fri, 24 Apr 2009 14:18:00 GMT</p> <p>Most discussions of online communities for feedback involve communities of customers and prospects. Just as important are online communities of employees. Such closed, private communities can offer employers rich insights about workplace conditions and changes in markets the employer serves.<br><br>One Vovici client, an international bank, has been experiencing tremendous growth in an emerging market, taking advantage of financially weakened competitors. As a result of this growth, the bank is facing a number of new challenges, chief of which are increased competition for local sales talent and increased competition for customers. The organization implemented an employee community to help address both of these issues. Employee feedback gathered from the online community is used to help position the bank as the "employer of choice" in this region's financial market. The community is also used to provide a unique window on customers: employees share customer feedback from the field and submit product and service ideas of their own and those suggested by customers.</p> <p>Another user of Vovici Community Builder has created an employee community that goes far beyond feedback. While employee satisfaction research is fielded through the online community, employees typically log in for other reasons: </p> <ul> <li>To view weekly sales numbers</li> <li>To participate in discussion forums</li> <li>To submit questions to senior management</li> <li>To view status reports for each department</li> <li>To see company news</li> <li>To access other employee portals, such as the benefits portal</li></ul>What innovative uses of online employee communities have you seen? </article> <article> <h1>Matrix Questions: Powerful but Perilous Technique</h1> <p>Jeffrey Henning — Thu, 23 Apr 2009 14:16:00 GMT</p> <P>The four <A href="http://blog.vovici.com/Blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires" mce_href="http://blog.vovici.com/Blog/bid/18197/Types-of-Questions-Four-Building-Blocks-for-Constructing-Questionnaires">basic question types</A> can be combined into matrix questions, as in the following example, which asks the respondent to rate satisfaction with their current electronics, their likelihood to purchase a replacement in the next half year and the brands they are considering for a replacement:</P> <P align=center></P> <P>This is a concise technique for combining questions with common topics and can be 50% faster for the respondent to complete than having to answer each question separately. No wonder survey authors love them.</P> <P>Unfortunately, matrix questions do raise a number of issues: </P> <UL> <LI>The faster speed of completion may lead to errors.</LI> <LI>Respondents can be prone to <A href="http://lovestats.wordpress.com/2009/03/25/survey-design-tip-3-do-you-encourage-straightlining/" target=_blank>"straightline" their answers to matrix questions</A>, selecting the same choice for each respondent, in a virtual straight line down the grid.</LI> <LI>Splitting the matrix into its component questions has greater predictive validity according to SSI's report "<A href="http://www.surveysampling.com/files/imce/SSI_Question_Design.pdf" target=_blank>Grid Test Summary</A>" (March 2009), perhaps precisely because doing so does take respondents more time to answer each question.</LI></UL> <P>Use care when using matrix questions in your surveys.</P> <P>When do you think matrices are the most appropriate way to ask for information?</P> </article> <article> <h1>Panel Management: Empaneling Respondents Across the Organization</h1> <p>Jeffrey Henning — Wed, 22 Apr 2009 13:47:00 GMT</p> <div id="__ss_1175053" style="width: 425px; text-align: left;">A <a href="http://www.slideshare.net/Vovici/panel-management?type=powerpoint" title="Panel Management" mce_href="http://www.slideshare.net/Vovici/panel-management?type=powerpoint">Panel Management</a> presentation that Brian Koma and I recently delivered:<br> <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=panelmanagement-090320112138-phpapp02&stripped_title=panel-management"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> <div style="font-size: 11px; padding-top: 2px; font-family: tahoma,arial; height: 26px;">View more <a href="http://www.slideshare.net/" style="text-decoration: underline;" mce_href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici" style="text-decoration: underline;" mce_href="http://www.slideshare.net/Vovici">Vovici</a>.</div></div> </article> <article> <h1>Types of Questions: Four Building Blocks for Constructing Questionnaires</h1> <p>Jeffrey Henning — Tue, 21 Apr 2009 13:29:00 GMT</p> <p>Sift out the finer distinctions and most questionnaires are constructed from four basic building blocks. These four, fundamental question types are: <br></p><table border="1"><tbody><tr><th>Open-ended questions</th><td>Essay Questions</td><td>Long text responses</td></tr><tr><td><br></td><td>Fill-in-the-Blank Questions</td><td>Short text responses</td></tr><tr><th>Closed-ended questions</th><td>Choose-One Questions</td><td>Single choice selected</td></tr><tr><td><br></td><td>Choose-Many Questions</td><td>Multiple choices selected</td></tr></tbody></table><h3>Essay Question</h3><p>The essay question, in a web survey, is displayed as a multi-line TEXTAREA: </p><p align="center"></p><p>Respondents can enter a few words, a few paragraphs, and even a few pages. Typically essay questions store 32,000 to 64,000 characters of text. We've had respondents copy and paste dozens of pages of text into essay questions.<br><br>This question type is best used for understanding in detail what a respondent believes in their own words. </p><h3><a name="fillintheblank" title="fillintheblank" class=""></a>Fill-in-the-Blank Question</h3>A fill-in-the-blank question is displayed as one or more text boxes with short labels, and is designed for gathering short responses: <p><br></p><p><br>It can also be used for contact information or address information: </p><p>Responses to text boxes are often validated to follow a common pattern. Common validations include:<br></p><ul><li>Email address</li><li>Whole number within a range</li><li>Real number within a range</li><li>Date</li><li>Telephone number - U.S./Canadian format or international format</li><li>Zip codes and postal codes</li></ul><h3>Choose-One Question</h3>The most common question type is that of the single-select multiple-choice question, where a respondent chooses one and only one of the available options. Choose-one questions are typically shown with radio buttons or dropdown boxes. <br>While choose-one questions constrain the choices of the respondent, they are much quicker to answer and much easier to analyze. <h3>Choose-Many Questions</h3>The multiple-select multiple-choice question allows the respondent to check all the choices that are applicable to the question. Most survey software uses the standard checkboxes of HTML forms to show these questions: <br>These four questions can then be combined together in matrix questions.<br>For best practices for each type of question, see these posts: <ul><li><a class="" mce_href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions" href="http://blog.vovici.com/Blog/bid/18095/Open-Ended-Questions">Essay-question best practices</a></li><li><a class="" mce_href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">Choose-one question best practices</a></li><li><a class="" mce_href="http://blog.vovici.com/Blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices" href="http://blog.vovici.com/Blog/bid/18170/Multiple-Answer-Questions-Select-All-That-Apply-Best-Practices">Choose-many questions</a> </li></ul> </article> <article> <h1>Market Research Departments Struggle for Strategic Relevance</h1> <p>Jeffrey Henning — Mon, 20 Apr 2009 13:20:00 GMT</p> <p><a href="http://www.linkedin.com/pub/2/559/809">Heidi Lo</a> and <a href="http://www.forrester.com/rb/analyst/brad_bortner" target="_blank">Brad Bortner</a> of Forrester have written a new independent report, "<a href="http://www.forrester.com/Research/Document/Excerpt/0,7211,48004,00.html" target="_blank">Role Insights: Market Researchers Struggle For Strategic Relevance</a>":<br></p> <blockquote><br></blockquote> <blockquote> <p>Forrester surveyed 40 market researchers from business-to-business (B2B) and business-to-consumer (B2C) companies with annual revenues of more than $1 billion and annual market research budgets ranging from $30,000 to $8 million.... Results from Forrester's Q2 2008 Global Market Research Organizing For Influence Online Survey show that many market researchers have not been able to position their market research groups as strategic partners within their organizations. Market research leaders report finding themselves sidelined because they are often viewed more as service bureaus focused on tactical, not strategic, questions and research. To pull out of this rut and get on the road to strategic relevance, it is imperative that market research groups create processes to prove value, enhance their own operational efficiency, and improve alignment with their internal clients. </p></blockquote> <blockquote> <p><b>Table of Contents</b></p> <ul> <li>Most Market Research Groups Are Not Strategic Partners Yet </li> <li>Challenges In Funding And Operations Are Key Obstacles </li> <li>Many Market Research Groups Are Lean Extensions Of Marketing - Minus The Budget </li> <li>Recommendations </li> <li>Get Organized To Start Attaining Strategic Relevance </li></ul></blockquote> <p>Forrester has prepared a sobering assessment of the state of market research within corporations. Given the strategic importance of customer research, it is surprising how project oriented most departments are. Yet MR departments are only addressing major projects-the minor projects are being addressed by line staff across the organization. Missing from Forrester's analysis is a discussion of the disintermediation of market researchers due to the widespread use of <a href="http://www.vovici.com/">survey software</a>, and the power of <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a> to help. Check out my post <a href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department" class="" mce_href="http://blog.vovici.com/Blog/bid/18087/Rethinking-the-Role-of-the-Market-Research-Department">Rethinking the Role of the Market Research Department</a> for more on this possible path for internal MR.</p> <p>What do you see as the way forward for structuring market research within organizations?</p> </article> <article> <h1>Compelling Survey Invitations</h1> <p>Jeffrey Henning — Fri, 17 Apr 2009 13:09:00 GMT</p> <p><a onclick="window.open( this.href, '_blank', 'width=800,height=412,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef0115700f56ca970b-800wi.png"></a>You've gotten the recipient to jump all the hurdles to get to the survey except for the last one - reading the invitation and clicking on the survey link.</p> <p>You now have the challenge of writing a short but motivating invitation that will prompt your recipient to become a respondent.</p> <p>As with any attempt at marketing, you have less than eight seconds to make a first impression. A clever use of images can help catch people's attention. You need to tell recipients what you want them to do as soon as possible in the invitation. Know your audience and write the invitation with this knowledge in mind. Keep the invitation short-but cover the key points.</p> <p>Patrick Glaser's "A Playbook for Creating Survey Introductions for Online Panels" provides a useful <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">outline for a survey invitation</a>, along with example phrases.</p> <p>I wish I could give you some proven catchphrases for subject lines that work with every survey, but what worked once--"Help us improve our products" or "Share your opinion"--might not work on the next round of survey invitations to the same audience. Experimentation and inventiveness are essential.</p> <p>Do not make privacy claims that you may not keep. This tends to happen by mistake rather than from any intention to mislead recipients. Often you are surveying your customer base and will want to set up <a href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" class="" mce_href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">survey alerts and email triggers</a> so that you can respond directly to dissatisfied customers to address their problems; don't then tell them in the invitation that their survey responses will only be used when aggregated with other responses.</p> <p>Similarly, do not offer incentives that you can't deliver. Again, this tends to happen by mistake, when an <a href="http://blog.vovici.com/Blog/bid/18078/When-Survey-Incentives-Run-Amuck" class="" mce_href="http://blog.vovici.com/Blog/bid/18078/When-Survey-Incentives-Run-Amuck">organization promises every respondent a reward</a>, then has far more awards than imagined. See last week's post for <a href="http://blog.vovici.com/Blog/bid/18167/Survey-Incentive-Strategies" class="" mce_href="http://blog.vovici.com/Blog/bid/18167/Survey-Incentive-Strategies">strategies for survey incentives</a>.</p> </article> <article> <h1>Customer Experience Excellence: Why, What and How</h1> <p>Jeffrey Henning — Thu, 16 Apr 2009 12:58:00 GMT</p> <p>Yesterday, at the <a href="http://www.omegascoreboard.com/score_2009.shtml" target="_blank">SCORE 2009 conference</a>, I had the good fortune to hear <a href="http://www.forrester.com/rb/analyst/bruce_temkin" target="_blank">Bruce Temkin</a> of Forrester Research present "The Path to Customer Experience Excellence: Why, What and How". I regularly read his blog, <a href="http://experiencematters.wordpress.com/" target="_blank">Customer Experience Matters</a>, so I was familiar with much of the material, but where the blog provides bite-size snack, Bruce's presentation was dinner and a show. (Complete with corny-enough jokes that I immediately reused one later in the day!)</p> <p>Bruce began by walking the audience through a customer interaction that a Forrester analyst Adele had with a consumer-electronics retailer. His framing analogy was that too many organizations make customers roll a rock up the hill, like Sisyphus, only to have the rock roll back down. Adele used the site, but didn't get what she needed. The rock rolls back down the hill. Adele called customer service, but didn't get what she needed. The rock rolls back down the hill. Adele called technical support, but didn't get what she needed. The rock rolls back down the hill. Adele went back to the site and sent an email, but the responding email didn't give her what she needed. The rock rolls back down the hill. </p> <p>As a guy who once named his company after a Greek demigod, I appreciated Bruce's take on this story. The two lessons he quoted: </p> <ol> <li>"Making customers push rocks up hills does not build loyalty."</li> <li>"Don't mess around with Zeus."</li></ol>Here is my quick recap of the "Why, What and How" to customer experience excellence, with links to related blog posts from Bruce: <ul> <li><b>Why?</b> You could have safely ignored customer-experience management three years ago, but not now. <a href="http://experiencematters.wordpress.com/2009/03/13/customer-experience-grows-up/" target="_blank">Customer experience is growing up</a>; it's been important to firms for long enough that your competitors are differentiating themselves on customer experience. The recession is strengthening the <a href="http://experiencematters.wordpress.com/2009/02/18/customer-experience-correlates-to-loyalty/" target="_blank">correlation between customer experience and customer loyalty</a>, across all 12 B2C industries Forrester studied. Perhaps as a result, most businesses will cut other areas disproportionately more than they will <a href="http://experiencematters.wordpress.com/2009/02/17/the-state-of-customer-experience/" target="_blank">cut areas that affect customer experience</a>.</li> <li><b>What?</b> One of the top firms in Forrester's <a href="http://experiencematters.wordpress.com/2008/03/14/the-most-useful-firms-costco-and-sams-club/" target="_blank">customer experience rankings is CostCo</a>, which tied for #3 after Barnes & Noble and USAA. "<a href="http://experiencematters.wordpress.com/2008/02/18/what-is-the-perfect-customer-experience/" target="_blank">Great customer experience</a> is not about Rainforest Café with a dazzling impression: it's about consistently meeting the needs and beating the expectations of customers. Unlike when going to a 7-11, you don't even expect to be able to park near the building at CostCo. But you do expect a broad selection at a great price." [quote from my notes; not verbatim]</li> <li><b>How?</b> To build great customer experiences, your organization needs to follow the three principals of <a href="http://experiencematters.wordpress.com/2007/06/17/experience-based-differentiation/" class="" target="_new" mce_href="http://experiencematters.wordpress.com/2007/06/17/experience-based-differentiation/">Experience-Based Differentiation</a>:</li> <ol> <li>Obsess about customer needs, not product features.</li> <li>Reinforce brands with every interaction, not just communications.</li> <li>Treat customer experience as a competence, not a function.</li></ol></ul>Bruce wrapped up by pointing attendees to his free booklet, <a href="http://experiencematters.wordpress.com/2008/07/22/free-book-the-6-laws-of-customer-experience/" target="_blank">The 6 Laws of Customer Experience</a>: The Fundamental Truths That Define How Organizations Treat Customers. Bruce is one of the leading authorities researching CE today; you'd have to have rocks in your head not to review his findings and work to make life easier for your customers. </article> <article> <h1>Skip Logic & Conditional Branches in Surveys</h1> <p>Jeffrey Henning — Wed, 15 Apr 2009 12:51:00 GMT</p> <p>Survey respondents can be routed through questionnaires using three types of branching logic: </p> <ul> <li><b>Skipping specific questions</b>: A skip pattern will jump a respondent over a group of questions that isn't relevant to them. For instance, a common skip pattern has a respondent rate a particular attribute on a scale and, if the rating is high, skips the respondent over one or two follow-up questions designed to probe why other respondents gave the item a low rating.</li> <li><b>Conditional branching</b>: A branch pattern will route a respondent to the appropriate section of the questionnaire: each respondent follows one of the branches. A common branch pattern has a respondent classify a product purchase or a type of service interaction and then asks follow-up questions specific to that answer.</li> <li><b>Unconditional destination</b>: Often a branching pattern is terminated with an unconditional destination, which returns respondents back to a common path. This has the effect of jumping respondents over other branches.</li></ul>When you find yourself writing a question that starts with something like "If you answered ‘No' to the previous question", that is a sure sign that you should set up a skip or branch pattern. Making people do this logic in their head is fine in a paper survey, but it is in appropriate for online surveys. Such manual skip patterns slow down the respondent, increase the amount of reading that they must do and make completing the survey tedious. Conversely, when skip and branching patterns are well implemented, they make the survey highly relevant and engaging.<br>A great example of well-implemented branching is a web-site feedback survey created by one Vovici client. With over 15 million unique visitors a month to their web site, the client determined that visitors typically came to the site with one of 17 different purposes in mind. Visitors to the site were <a href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">randomly invited</a> to take the feedback survey, which began with five standard questions that all respondents were asked, ending with a question asking "So what are you here to do today?" The answer to this gating question would then branch respondents to one of 17 different paths, each asking an average of 8 questions unique to that action. The result was that no respondent answered more than 20 questions of a 148-question survey, and the questions they did answer were very specific to their experience.<br>Your own skip and branching patterns don't need to be as complex. Some best practices for skip and branching logic: <ul> <li>Remember that page breaks indicate logical jumping-off points. Sometimes survey authors like to have one question per page; other authors like to have related questions together on a page. Skip logic overrides the preference for how to group questions, as it is only applied at the end of a page.</li> <li>Skip logic routes respondents forward through the survey, never backward. Think of the question flow like a river, going downstream, with different sluice gates channeling the water into different canals and sometimes back into the main watercourse.</li> <li>Due to the wide variety of possible synonymous answers to <a href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">open-ended questions</a>, skip logic is primarily used with closed-ended questions.</li> <li>Before inviting respondents to take your survey, make sure you test that each path through the survey matches the logic you intended.</li> <li>Some survey software applications, include Vovici v4, support Boolean logic ("and", "or" and "not" and nested parentheses). This makes programmers happy, but can be confusing for first-time survey authors. For instance, "and" doesn't have its English meaning: a survey author who wants the system to skip to Q9 when Q1 is answered "blue" and "green" needs to write "Q1=blue or Q1=green" not "Q1=blue and Q1=green" (which is a logical impossibility from a Boolean perspective). If you are using advanced branching for the first time, get some help from technical support or a programmer and test your survey logic carefully.</li> <li>For a questionnaire already collecting responses, you sometimes might find that you should have added a skip pattern. Good <a href="http://www.vovici.com/">survey software</a> will let you do this, then provide a utility to validate all previous answers against this new skip pattern, setting any answers back to unanswered for questions that shouldn't have been asked.</li> <li>If you are displaying progress bars, respondents may suddenly find themselves very far through the survey, because of a skip pattern. Curious respondents may hit the Back button, then answer differently to see where they go next. Respondents are more likely to do this in the screener if the survey offers a financial incentive to qualifying respondents. In such cases, configure the survey to not publish a Back button beside the Next button.</li></ul>What other best practices do you follow when using skip logic? </article> <article> <h1>Closed Communities for B2B Research</h1> <p>Jeffrey Henning — Tue, 14 Apr 2009 12:48:00 GMT</p> <p>Open online communities are accessible to anyone. Often the content can be reviewed without creating a user account, though typically an account is required for commenting and starting new discussion threads. Closed online communities, in contrast, are by invitation only. Both types of communities have their place. While many B2C communities are open and public, many B2B communities are closed and private.</p> <p>These private B2B communities often take the form of product advisory councils or key-account advisory boards. The content produced by such communities is sensitive and competitive, often involving feedback that will be used to shape future products, services and promotions. Accordingly, organizations do not want their competitors to be able to access and review this information, and a closed community is the preferable approach.</p> <p>Once you decide a community will be closed, however, you dramatically increase the workload of the community manager: he or she needs to promote the community to potential recruits, they need to get new members engaged in the community, they need to encourage members to return to the community frequently, and so on. Further, because private communities are typically smaller than public communities, sponsoring organizations need to generate more content themselves at the beginning, and it is harder to get the community to the point where it is thriving and self-sustaining. That said, the results are often rich and instructive, and therefore worthy of the extra labor involved.</p> <p>For a full contrast between the two types of communities, check out this <a href="http://blog.vovici.com/Blog/bid/18086/Public-Communities-vs-Private-Communities" class="" mce_href="http://blog.vovici.com/Blog/bid/18086/Public-Communities-vs-Private-Communities">Public vs. Private Community</a> matrix.</p> </article> <article> <h1>Survey Software Success: Free EBook</h1> <p>Jeffrey Henning — Mon, 13 Apr 2009 12:35:00 GMT</p> <p><a href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx" class="" mce_href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx"></a>The hypertext nature of this blog is great for when you want to dive in on one topic and follow links to related posts. However, it's not well suited to those times when you want to read past posts in detail.</p> <p>To address this, I've compiled much of the blog into a free, 73-page ebook, <i>Survey Software Success</i>. The structure of the book was inspired by a presentation we did and outlines seven best practices for survey success: </p> <ol> <li>Focus on a Goal</li> <li>Survey the Right Number of People</li> <li>Plan Your Mail Campaign with Care</li> <li>Craft Your Questions Well</li> <li>Order Questions Logically</li> <li>Get the Survey Length Right</li> <li>Close the Feedback Loop</li></ol>Each of the above is its own chapter, and the book is about 73 pages long. You can <a href="http://www.vovici.com/forms/signup-ebook-surveysoftware.aspx" target="_blank">use this form</a> to get your free copy. </article> <article> <h1>Employee Satisfaction Presentation: Five Reasons You Must Measure Employee Loyalty During a Recession</h1> <p>Jeffrey Henning — Fri, 10 Apr 2009 12:30:00 GMT</p> <div id="__ss_1175400" style="width: 425px; text-align: left;">This is a presentation that <a href="http://blog.walkerinfo.com/blog/loyalty-in-the-workplace" mce_href="http://blog.walkerinfo.com/blog/loyalty-in-the-workplace">Chris Woolard</a>, Brian Koma, and I delivered on <a href="http://www.slideshare.net/Vovici/employee-satisfaction?type=presentation" title="Employee Satisfaction" mce_href="http://www.slideshare.net/Vovici/employee-satisfaction?type=presentation">Employee Satisfaction</a>:<br> <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=employeesatisfaction-090320125241-phpapp01&stripped_title=employee-satisfaction"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> <div style="font-size: 11px; padding-top: 2px; font-family: tahoma,arial; height: 26px;">View more <a href="http://www.slideshare.net/" style="text-decoration: underline;" mce_href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici" style="text-decoration: underline;" mce_href="http://www.slideshare.net/Vovici">Vovici</a>.</div></div>See this <a href="http://blog.vovici.com/Blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession" class="" mce_href="http://blog.vovici.com/Blog/bid/18073/Five-Reasons-You-Must-Measure-Employee-Loyalty-During-a-Recession">write-up of the presentation</a>. <div></div> </article> <article> <h1>Good Surveys start with Good Goals</h1> <p>Jeffrey Henning — Thu, 09 Apr 2009 12:27:00 GMT</p> <p>Don't let your survey project go astray from the start: be certain to focus on a specific goal. Be precise about what information you need to gather and what you plan on doing with it. </p><p>If your organization hasn't done a survey in a while, the tendency is for every department to chime in with questions they want you to ask. The result: questionnaire design by committee, taking up more of your and your coworkers' time to prepare, then producing a survey that is too long and tedious for respondents to complete quickly.</p><p>By setting a narrow goal you will be able to relentlessly simplify the survey and keep it on target. To do this, talk to the stakeholders who will use the survey data. Develop an understanding of their wants and needs. What are the specific decisions they need to make? What information do they currently have? Where are they uncertain about what the target audience believes?</p><p>Once you have come up with a list of goals, ask yourself whether a survey is really the best way to gather this information. </p><ul><li>Can you meet the goals without doing a survey at all?</li><li>Are your potential respondents the only source of this information?</li><li>If you work for a large organization, is a coworker in another department doing a survey on this topic or researching this issue?</li><li>Do your CRM, web analytics or other systems hold data that would help you reach your goal? You can ask web-site visitors their favorite sections of your web site, but using web analytics to study the traffic volume for the different sections will provide you better data.</li></ul><p>To help you, here are some examples of bad goals and better goals.</p><p><br></p><table padding="0" border="1"><tbody><tr><th>Bad Survey Goal</th><th>Better Approach</th><th>Improvement</th></tr><tr><td width="25%">Maybe customers would like it if we provided support after 6 p.m.</td><td>Prioritize ideas for expanding customer service.</td><td>Don't ask about one product or service improvement in isolation. You need to compare it against other possible ideas to determine the best initiative for your organization.</td></tr><tr><td>Find out how satisfied customers are with customer service.</td><td>Find out how satisfied customers are with customer service and intervene with unsatisfied customers to address their issues.</td><td>Don't approach customer satisfaction as simply a strategic concern, a matter of measurement, but use it tactically as well to identify individuals who are unsatisfied in an attempt to fix their problems and satisfy them.</td></tr><tr><td>What are our customers thinking right now?</td><td>Instead of doing a survey, conduct one-on-one unstructured interviews with customers to determine their current attitudes. Use the results of those interviews to determine other research needs.</td><td>Surveys are great for structured information gathering that you can extrapolate to your target audience (quantitative insights-see sidebar). When you are looking for qualitative insights instead, just talk to customers.</td></tr><tr><td>Let's create a committee and see what they want to find out from a customer survey.</td><td>Instead survey the employees about the customer information they are currently lacking and use that to determine the priorities for your customer research.</td><td>Committees can be driven by individuals with pet-projects; a survey of employees will help you develop an assessment of how many people need to gather specific types of information.</td></tr><tr><td>Determine how often callers into help desk check the customer-service portal before calling.</td><td>Record customer-service portal log-ins and knowledge-base searches into the CRM system, so that representatives can see the searches as soon as they identify the caller. Run reports on the percent of knowledge-base lookups that lead to calls and track that over time.</td><td>Actual usage is more reliable data than self-reported assessments of usage through a survey, and customer do not need to be interrupted to take a survey.</td></tr></tbody></table><br>I'd love your examples of bad goals you've seen for survey projects over the years. </article> <article> <h1>Survey Monkey Wrench</h1> <p>Jeffrey Henning — Wed, 08 Apr 2009 12:20:00 GMT</p> <p>Successful <a href="http://www.vovici.com/">online surveys</a> often grow from projects to processes:</p> <ul> <li>That one-off survey project to determine satisfaction with the help desk about a particular upgrade is now a regular process used each time IT rolls out a new application.</li> <li>That annual corporate climate study is now a monthly employee-feedback process with the results distributed to over 1000 managers in personalized <a href="http://blog.vovici.com/Blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day" class="" mce_href="http://blog.vovici.com/Blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day">hierarchical reports</a>.</li> <li>That occasional survey to measure satisfaction with the call center is now a weekly census used to measure the satisfaction with each of 500 customer-service representatives.</li></ul> <p>When you've outgrown that <a href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business" class="" mce_href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business">Survey Monkey</a> survey and need to convert your survey projects into feedback processes, check out our robust, scalable and high-performing API (Application Programming Interface) for integration. When surveys are no longer tactical, but strategic, you want a strategic vendor as your partner. When call-center representatives are fired or promoted based on their rolling six-week satisfaction rating, you want iron-clad and litigation-proof data to rely on. </p> <p>And if you think with Survey Monkey and a wrench that you can build what you need, then by all means give them a call first. Then give us a call at +1.703.481.9326 in the United States and +44 (0)208.547.4040 in Europe to discuss your unique needs.</p> </article> <article> <h1>Ensuring Your Survey Invitation Isn’t Flagged as Spam</h1> <p>Jeffrey Henning — Tue, 07 Apr 2009 12:11:00 GMT</p> <p>With the rise of <a href="http://en.wikipedia.org/wiki/Uniform_Resource_Locator" target="_blank">Bayesian filters</a>, most email users are seeing fewer spam messages in their Inboxes. You want to make sure that your <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">survey invitation</a> and <a href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" class="" mce_href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">reminder emails</a> aren't flagged as spam, which would artificially lower your <a href="http://blog.vovici.com/Blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship" class="" mce_href="http://blog.vovici.com/Blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship">response rate</a>.</p> <p>Because spam filtering programs are constantly changing, it is best to send your draft email invitation to the personal email addresses of friends and coworkers simply to make sure all receive it. Be careful when discussing survey incentives in the email, as words and phrases such as "free" and "act now" are more likely to get your invitation classified as spam. Hectic punctuation - <i>ALL CAPS, lots of exclamations!!!, dollar $ign$, L33t sp34k</i> - are also more likely to get your message flagged.</p> <p>Another important aspect of spam for U.S. organizations is the CAN-SPAM Act, which requires three types of compliance for organizations sending commercial email messages to organizations with which they have no relationship. (In other words, messages sent to customers do not fall under the jurisdiction of this act.) The three types: </p> <ol> <li><b>Unsubscribe compliance</b> - Each email contains an obvious and <a href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes" class="" mce_href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes">working method for the recipient to unsubscribe</a>. All opt-out requests are processed within 10 days. The suppression lists of opt-outs are not mailed to.</li> <li><b>Content compliance</b> - The email From field is accurate, the subject field relates to the body of the email message and is not misleading, and the body contains a valid physical mailing address for the organization sending the message. (Surveys on sexual topics must be labeled in the subject header as containing sexually-explicit content.)</li> <li><b>Sending-behavior compliance</b> - Your email messages cannot be sent through an open relay and cannot contain a false header. Almost all hosted survey software applications comply with this. Additionally, your message cannot be sent to an email address that was harvested (downloaded and parsed) from web pages.</li></ol> <p>The penalty for failure to comply with these items ranges from a misdemeanor charge to an aggravated offense, depending on the size and scope of the offense. Where appropriate, you should consult a lawyer or review the <a href="http://www.ftc.gov/spam/">CAN-SPAM Act in more detail</a> to guarantee your email is in compliance.</p> <p>What tools have you found helpful for writing email messages that will get past spam filters?</p> </article> <article> <h1>Voice of the Customer (VOC) Techniques & Technologies</h1> <p>Jeffrey Henning — Mon, 06 Apr 2009 09:38:00 GMT</p> <p><a target="_blank" href="http://www.forrester.com/rb/analyst/bruce_temkin">Bruce Temkin</a>, a principal analyst at Forrester, defines "voice of the customer" as "a systematic approach for incorporating the needs of customers into the design of customer experiences." His blog post "<a target="_blank" href="http://experiencematters.wordpress.com/2007/07/03/are-you-listening-to-the-voice-of-the-customer/">Are you listening to the voice of the customer?</a>" outlines five levels of activities in a VOC program:<br></p><ol><li><b>Relationship tracking</b>. Organizations need to track the health of customer relationships over time...</li><li><b>Interaction monitoring</b>. Every customer interaction - from an online transaction to a call into the call center - is important. Firms need a way to monitor how effectively they handle these customer touches...</li><li><b>Continuous listening</b>. ...There are many opportunities to hear what customers are saying, such as listening to calls in the call center, reading blogs, reading inbound emails, and visiting retail outlets.</li><li><b>Project infusion</b>. Projects that affect customers should incorporate insights about customers. Despite the clear need for this type of effort, many companies lack a formalized approach for infusing customer insights into projects...</li><li><b>Periodic immersion</b>. Every so often, it's valuable for all employees - especially executives - to spend a significant amount of time interacting directly with customers or working alongside frontline employees...</li></ol>Surveys, online communities and text analytics can help you listen to the voice of the customer at each of the five levels that Bruce describes: <ol><li><b>Relationship tracking</b>. Periodic surveys are an excellent way to keep your finger on the pulse of customers. Many of our users conduct a full <a class="" mce_href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">census of customers</a> annually or biannually, but to truly listen to the voice of the customer we advocate monthly or quarterly surveys of a <a class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">random sample</a> of customers. That way you always have a fresh perspective on current customer attitudes.</li><li><b>Interaction monitoring</b>. Automated surveys can follow up each purchase, customer-service interaction and renewal. By <a class="" mce_href="http://blog.vovici.com/Blog/bid/17907/Announcing-Vovici-EFM-CRM-Connector-for-Oracle" href="http://blog.vovici.com/Blog/bid/17907/Announcing-Vovici-EFM-CRM-Connector-for-Oracle">integrating your CRM system</a> with your <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback</a> platform, you can constantly listen for the voice of the customer. Coupling this interaction with <a class="" mce_href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">survey alerts/email triggers</a> is a great way to act on the voice of the customer, too.</li><li><b>Continuous listening</b>. <a target="_blank" href="http://www.clarabridge.com/">Text analytics</a> enable you to eavesdrop on thousands of customers as they comment on your surveys, email your organization and blog and <a target="_blank" href="http://twitter.com/jhenning">tweet</a> about you.</li><li><b>Project infusion</b>. <a class="" mce_href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR" href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR">Qualitative and quantitative research</a> into the voice of customer needs to be infused throughout the <a class="" mce_href="http://blog.vovici.com/Blog/bid/18065/Online-Communities-in-the-Product-Development-Lifecycle" href="http://blog.vovici.com/Blog/bid/18065/Online-Communities-in-the-Product-Development-Lifecycle">product lifecycle</a>: integrate such research into the planning, development, implementation and marketing of each product or service.</li><li><b>Periodic immersion</b>. <a href="http://www.vovici.com/survey-solutions/online-communities.aspx">Online communities</a> can immerse your employees in the thoughts and attitudes of your customers. To truly become a customer-driven organization, make sure that employee participation in your customer community is broad and deep.</li></ol>Modern technologies make it easier than ever to listen to customers. Make sure your organization is evaluating such tools, and be sure to check out <a href="http://www.vovici.com/about/news-and-press/2009/VOC.aspx">Vovici's new</a> <a href="http://www.vovici.com/survey-solutions/Voice-of-the-Customer.aspx">Voice of the Customer</a> Success Package that can help your organization better listen to customers. </article> <article> <h1>Demographic Questions: Sample Survey Template</h1> <p>Jeffrey Henning — Fri, 03 Apr 2009 09:29:00 GMT</p> <p>In your <a href="http://www.vovici.com/">online surveys</a>, you should use demographic and <a href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template" class="" mce_href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template">firmographic</a> questions to profile respondents and their organizations. This will enable you to cross-tabulate and compare subgroups to see how opinions vary between these groups.</p> <p>When possible, place these sections near the end of the questionnaire as they are tedious and intrusive. Sometimes, though, one or two demographic questions must be moved to the screener in order to route respondents appropriately.</p> <p>Pre-populate from CRM systems where possible, so that information can be updated rather than re-entered. When <a href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors" class="" mce_href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors">integrating with CRM data</a>, make sure to ask the question in your survey the same way it is reported by your CRM system.</p> <p>When trying to compare your survey results to other data, such as <a href="http://www.census.gov/dmd/www/2000quest.html">U.S. Census estimates</a>, make sure to use similar categories and questions as your sources.</p> <p>Here are sample demographic questions, many adapted from the 2000 U.S. Census: </p> <blockquote>Q. Gender<br>What is your sex?<br>o Male<br>o Female<br><br>Q. Age<br>In what year were you born? ____<br><br>Q. Marital Status<br>What is your marital status?<br>o Now married<br>o Widowed<br>o Divorced<br>o Separated<br>o Never married<br><br>Q. Education<br>What is the highest degree or level of school you have completed? If currently enrolled, mark the previous grade or highest degree received.<br>o No schooling completed<br>o Nursery school to 8th grade<br>o 9th, 10th or 11th grade<br>o 12th grade, no diploma<br>o High school graduate - high school diploma or the equivalent (for example: GED)<br>o Some college credit, but less than 1 year<br>o 1 or more years of college, no degree<br>o Associate degree (for example: AA, AS)<br>o Bachelor's degree (for example: BA, AB, BS)<br>o Master's degree (for example: MA, MS, MEng, MEd, MSW, MBA)<br>o Professional degree (for example: MD, DDS, DVM, LLB, JD)<br>o Doctorate degree (for example: PhD, EdD)<br><br>Q. Employment Status<br>Are you currently...?<br>o Employed for wages<br>o Self-employed<br>o Out of work and looking for work<br>o Out of work but not currently looking for work<br>o A homemaker<br>o A student<br>o Retired<br>o Unable to work<br><br>Q. Employer Type<br>Please describe your work.<br>o Employee of a for-profit company or business or of an individual, for wages, salary, or commissions<br>o Employee of a not-for-profit, tax-exempt, or charitable organization<br>o Local government employee (city, county, etc.)<br>o State government employee<br>o Federal government employee<br>o Self-employed in own not-incorporated business, professional practice, or farm<br>o Self-employed in own incorporated business, professional practice, or farm<br>o Working without pay in family business or farm<br><br>Q. Housing<br>Is this house, apartment, or mobile home -<br>o Owned by you or someone in this household with a mortgage or loan?<br>o Owned by you or someone in this household free and clear (without a mortgage or loan)?<br>o Rented for cash rent?<br>o Occupied without payment of cash rent?<br><br>Q. Household Income<br>What is your total household income?<br>o Less than $10,000<br>o $10,000 to $19,999<br>o $20,000 to $29,999<br>o $30,000 to $39,999<br>o $40,000 to $49,999<br>o $50,000 to $59,999<br>o $60,000 to $69,999<br>o $70,000 to $79,999<br>o $80,000 to $89,999<br>o $90,000 to $99,999<br>o $100,000 to $149,999<br>o $150,000 or more<br><br>Q. Ethnicity<br>Please specify your ethnicity.<br>o Hispanic or Latino<br>o Not Hispanic or Latino<br><br>Q. Race<br>Please specify your race.<br>o American Indian or Alaska Native<br>o Asian<br>o Black or African American <br>o Native Hawaiian or Other Pacific Islander <br>o White<br></blockquote>For related posts, see: <ul> <li><a href="http://blog.vovici.com/Blog/bid/18106/Race-Ethnicity-Survey-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18106/Race-Ethnicity-Survey-Questions">Race & Ethnicity Survey Questions</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18077/How-to-Ask-Respondents-Their-Age" class="" mce_href="http://blog.vovici.com/Blog/bid/18077/How-to-Ask-Respondents-Their-Age">How to Ask Respondents Their Age</a></li> <li><a href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template" class="" mce_href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template">Firmographic - Definition & Template</a></li></ul> </article> <article> <h1>Closing the Feedback Loop: Sharing Results with Online Community Members & Respondents</h1> <p>Jeffrey Henning — Thu, 02 Apr 2009 09:23:00 GMT</p> <p>I wanted to elaborate on one of the <a href="http://blog.vovici.com/Blog/bid/18121/The-Seven-Deadly-Sins-of-Online-Community-Management" class="" mce_href="http://blog.vovici.com/Blog/bid/18121/The-Seven-Deadly-Sins-of-Online-Community-Management">Seven Deadly Sins of Online Community Management</a>: "Greed: Failing to share with community members the results of feedback and the changes inspired by the community."</p> <p>When surveying customers, employees, community members and other key constituencies, your respondents complete the survey because they value their relationship with you, and they want to see you improve. Implicit in the fact that you sent them a survey is your intention to learn, adapt and change based on the results. Accordingly, to close the feedback loop with respondents and community members, you should:<br></p> <ul> <li>Explain what you're using the data for</li> <li>Share summaries and slices of the data</li> <li>Identify actions you're taking as a result of survey</li> <li>Consider using web seminars, video conferences and meetings to share data</li></ul> <p>Many <a href="http://www.vovici.com/">survey-software</a> applications include the ability to send a thank-you notice to all respondents. This is an excellent way to point out some of the above.</p> <p>With the survey completed, use the occasion to stimulate additional participation and tease out feedback that elaborates on some of the points raised. For the long-term success of your online community or survey panel, you have to demonstrate to participants that you value their participation and will use it to serve them better in the future. Closing the feedback loop is the best way to achieve steady participation and <a href="http://blog.vovici.com/Blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship" class="" mce_href="http://blog.vovici.com/Blog/bid/18134/Survey-Response-Rate-Directly-Proportional-to-Strength-of-Relationship">response rates</a> over time.</p> </article> <article> <h1>First Commercial Software for TRS-80 in a Decade: Vovici FEEDBACK/80</h1> <p>Jeffrey Henning — Wed, 01 Apr 2009 09:14:00 GMT</p> <p>ROCKLAND, MA-April 1, 2009- Vovici today announced the immediate availability of a new version of EFM Feedback written specifically for the Tandy/Radio Shack TRS-80 Model I. "When we realized that no new software had been developed for the TRS-80 Model I in eleven years, we felt compelled to re-open the market window," said Jeffrey Henning, chief strategy officer of Vovici. "The TRS-80 is an exciting platform for us, given its installed base. While some software companies were started in the garage, we're the first to develop software for computers that are often left in the garage." <br><br>Vovici FEEDBACK/80 is available directly from Vovici as a bundle with any edition of a Vovici application. For a limited time only, the new product comes with <a href="http://blog.vovici.com/Blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise" class="" mce_href="http://blog.vovici.com/Blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise">Vovici v4</a>, the leading enterprise-feedback-management platform and the only <a href="http://www.vovici.com/" mce_href="http://www.vovici.com/">survey software</a> that uses a word-processor interface. Available as an annual subscription, Vovici v4 allows you to design, administer, collect and analyze web surveys.</p> <p>Unlike the TRS-80 edition, the web edition offers full charting and presentations, report writing and questionnaire libraries. Add-on options include <a href="http://www.vovici.com/survey-analytics/survey-data-analysis.aspx" mce_href="http://www.vovici.com/survey-analytics/survey-data-analysis.aspx">Feedback Intelligence</a>, which brings business intelligence capabilities to survey analysis, and <a href="http://www.vovici.com/survey-software/mobile-survey-pc-edition.aspx" mce_href="http://www.vovici.com/survey-software/mobile-survey-pc-edition.aspx">Mobile Survey</a>, which provides for offline survey administration.</p> <p>Vovici FEEDBACK/80 supports a nostalgic command-line interface and can be used to enter and tabulate survey results, with the number of questionnaires you can process limited only by RAM. You can enter 30 questionnaires for a system with 4K RAM and several hundred questionnaires for 16K RAM. FEEDBACK/80 can support up to 50 questions per survey, where every question is a multiple-choice, single-response style question. To conserve RAM, you do not enter the text of the questionnaire into the system. </p> <p>"We're taking backward compatibility to a new level," commented <a href="http://bit.ly/BEGQ" target="_blank" mce_href="http://bit.ly/BEGQ">Jody Bailey</a>, vice president of development and network operations for Vovici. "Supporting the TRS-80 emphasizes the Vovici commitment to integrating state-of-the-art feedback management with legacy systems."</p> <p><b>About Vovici</b> </p> <p>Vovici is the pioneer of <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx" mce_href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">Enterprise Feedback Management</a> (EFM), providing comprehensive survey software, panel management and online community solutions. Our survey tools enable organizations to centralize feedback data collection, build and manage proprietary panels, leverage corporate social networking, and utilize robust survey analytics and reporting. Our solutions increase customer loyalty, facilitate collaboration and innovation, influence critical business decisions and provide voice to online communities. Organizations worldwide, including more than half of the Fortune 500, rely on Vovici to gather feedback on customer satisfaction, perform market research and gauge employee satisfaction. Visit <a href="http://www.vovici.com/" mce_href="http://www.vovici.com/">http://www.vovici.com/</a> for more information.</p> <p>CONTACT: </p> <blockquote> <p>Vovici<br>Jeffrey Henning, 781-848-8100 x 550 <br><a href="mailto:jhenning@vovici.com?subject=TRS-80&body=I%20enjoyed%20your%20April%20Fool%27s%20joke%21%20%20But%20I%20have%20a%20question%20for%20you...">mailto:jhenning@vovici.com?subject=TRS-80&body=I%20enjoyed%20your%20April%20Fool%27s%20joke%21%20%20But%20I%20have%20a%20question%20for%20you...</a> </p></blockquote> </article> <article> <h1>Enterprise Feedback Management: Using Feedback across the Enterprise</h1> <p>Jeffrey Henning — Tue, 31 Mar 2009 09:10:00 GMT</p> <br> <div id="__ss_1174981" style="width: 425px; text-align: left;">A presentation Brian Koma and I recently delivered on <a href="http://www.slideshare.net/Vovici/enterprise-feedback-management?type=presentation" title="Enterprise Feedback Management" mce_href="http://www.slideshare.net/Vovici/enterprise-feedback-management?type=presentation">Enterprise Feedback Management</a>: <param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=enterprisefeedbackmanagement-090320110956-phpapp01&stripped_title=enterprise-feedback-management"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"> <div style="font-size: 11px; padding-top: 2px; font-family: tahoma,arial; height: 26px;">View more <a href="http://www.slideshare.net/" style="text-decoration: underline;" mce_href="http://www.slideshare.net/">presentations</a> from <a href="http://www.slideshare.net/Vovici" style="text-decoration: underline;" mce_href="http://www.slideshare.net/Vovici">Vovici</a>.</div></div> </article> <article> <h1>Multiple-Answer Questions (“Select All That Apply”): Best Practices</h1> <p>Jeffrey Henning — Mon, 30 Mar 2009 08:59:00 GMT</p> <p>A common question type in most <a href="http://www.vovici.com/">survey software</a> applications is the "Choose Many":</p> <p mce_keep="true" align="center"></p> <p mce_keep="true">The best practices for using this question type are straightforward:<br></p> <ul> <li><b>Always include a "None of the above" as an exclusive choice</b> - If you omit this option, then a valid response is to not select any of the choices, because none of them apply. Nothing is more maddening for a respondent then to purposefully not answer this question only to have it be required, with the survey system prompting for an answer. To handle this case, set the last option to "None of the above" or similar wording and set it to be exclusive: if the respondent clicks this choice, the other checkboxes will be automatically unchecked. If you do not require an answer, failure to use this option will give you an ambiguity in the data: did the respondent skip the question or "answer" it by not checking anything? (If your survey software doesn't include the ability to make this exclusive, include the choice but manually clean the data beforehand, removing any other choices selected when "None of the above" was selected.)</li> <li><b>If the choice list has no natural order, randomize the order</b> - Respondents are in a hurry and have a tendency to skim the later choices of a long choice list (see <a href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior" class="" mce_href="http://blog.vovici.com/Blog/bid/18156/Satisficing-and-Survey-Respondent-Behavior">Satisficing and Survey Respondent Behavior</a>). If they have checked a few choices early on, then they feel like they have answered this question sufficiently and will move on to the next. By randomizing the list, you avoid giving an artificial bias to early options. Of course, make sure to anchor the position of "None of the above" choice to the bottom of the list. </li> <li><b>Avoid providing choices that overlap in meaning</b> - If one choice overlaps another, then respondents will be inconsistent about selecting both choices when appropriate.</li> <li><b>Explicitly remind respondents to select all that apply</b> - Include instructions such as "Please check all that apply" at the end of your question text [suggested by <a href="http://twitter.com/kschulman14/status/1417996577" target="_blank">Kevin Schulman</a>]</li> <li><a href="http://blog.vovici.com/.a/6a00d83548632853ef01156f7fdeae970b-pi"></a><b>Don't use a listbox to show the choices</b> - While the HTML <SELECT> tag has long supported the ability to use this user-interface control (see illustration), it is confusing to respondents and rarely used today. Checkboxes are the better choice.</li> <li><b>Consider using a Yes/No matrix</b> - As discussed in my recent post, "<a href="http://blog.vovici.com/Blog/bid/18139/Yes-No-Matrix-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18139/Yes-No-Matrix-Questions">Yes/No Matrix Questions</a>", sometimes a list of Yes/No questions is more appropriate to your research than a choose-many question.</li></ul> <p mce_keep="true">Two powerful applications of the choose-many question: </p> <ul> <li><b>As a preface to a matrix question</b> - Ask the respondent all the brands that they are familiar with, then - on the next page - show a matrix question asking them to provide additional information about each brand they selected. See "<a href="http://blog.vovici.com/Blog/bid/18118/Questionnaire-Workshop-Refactoring-One-Question-into-Three-Parts" class="" mce_href="http://blog.vovici.com/Blog/bid/18118/Questionnaire-Workshop-Refactoring-One-Question-into-Three-Parts">Questionnaire Workshop: Refactoring One Question into Three Parts</a>" for an example of this approach. In addition to piping selected choices into a matrix, <a href="http://blog.vovici.com/Blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise" class="" mce_href="http://blog.vovici.com/Blog/bid/18072/Vovici-v4-Extends-Survey-Authoring-Analysis-throughout-Enterprise">Vovici v4</a> survey software supports the ability to pipe unselected choices into a matrix.</li> <li><b>To select the most important items</b> - When you have more choices than a respondent can comfortably rank (8 or more), instead ask the respondent to "choose at most 3 choices". While you can ask the respondent to "choose exactly 3 choices", and set the validation to require this, it is better to let them select up to 3 choices, rather than have to include a lower priority choice in the list.</li></ul> <p mce_keep="true">Ironically, I picked the choose-many topic as the subject of this morning's blog post because I didn't think I had much to say about it! And now I feel like I am leaving something out. What other aspects of this question type are important to consider?</p> </article> <article> <h1>Survey Monkey in Silk</h1> <p>Jeffrey Henning — Fri, 27 Mar 2009 08:51:00 GMT</p> <p>In "<a href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business" class="" mce_href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business">Survey Monkey</a> Business", I wrote that "Whenever a prospect tells us that the competition for their business is between SurveyMonkey and Vovici, we know that one of us is in the wrong place." Last week I gave some examples of <a href="http://blog.vovici.com/Blog/bid/18142/Survey-Monkey-Over-a-Barrel" class="" mce_href="http://blog.vovici.com/Blog/bid/18142/Survey-Monkey-Over-a-Barrel">customers who outgrew SurveyMonkey</a>, some of whom moved up to <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a>, with <a href="http://www.vovici.com/survey-software/panel-management.aspx">panel management</a> and CRM integration.</p> <p>But many customers just need a more powerful survey tool. The three biggest reasons such customers move up to a Vovici v4 Individual or Professional Edition account are: </p> <ol> <li><b>Reporting</b> - The prospects spend an inordinate amount of time creating reports in Excel and want the survey tool to do the heavy lifting. The user knows their life can be better, and Vovici v4 dramatically decreases their workload.</li> <li><b>Branding</b> - The prospects want more control over the look and feel of the survey and reports. They want all surveys to be driven from their own domain name (e.g., <u>acme_surveys.foo</u>) or subdomain (e.g., <u>surveys.example.com</u>).</li> <li><b>Advanced Features</b> - Power users of 'Monkey quickly realize the limitations of the tool, and its inability to do advanced branching, data piping and multi-language surveys.</li></ol> <p>You can dress a monkey in silk, but...</p> <p>If you're interested in learning more, <a href="http://www.vovici.com/forms/signup-demo.aspx">schedule a live demo</a> or give us a call at +1.703.481.9326.</p> </article> <article> <h1>Survey Incentive Strategies</h1> <p>Jeffrey Henning — Thu, 26 Mar 2009 08:44:00 GMT</p> <p>For your first survey mail campaign to a target audience, avoid offering incentives. You need to establish a baseline response rate for this target group. If you are like many organizations, you will not need to offer incentives to boost your response rate. If you are disappointed in the response rate for this survey, at least you will now have a response rate to compare incentive-based campaigns against, to determine their effectiveness.<br><br>Also, offering incentives can have a corrosive effect on the integrity of the responses (see <a href="http://blog.vovici.com/Blog/bid/17981/Encouraging-Honesty-from-Survey-Respondents-and-Community-Members" class="" mce_href="http://blog.vovici.com/Blog/bid/17981/Encouraging-Honesty-from-Survey-Respondents-and-Community-Members">Encouraging Honesty from Survey Respondents and Community Members</a>).</p> <p>When crafting an offer, use a combination of incentives and deadlines to gain rapid response: "The first 100 people to submit a completed survey by April 2 will receive..." The deadline creates a sense of urgency, while the limit on the number of incentives bounds your financial liability.</p> <p><i>Koma's First Law of Incentives</i>: A large number of small incentives typically gets better response than a small number of large incentives. Besides getting a higher response, another benefit to following Koma's First Law is that it is easier than the alternative: that "small number of large incentives" has to be awarded through a sweepstakes-and sweepstakes administration is complex. The laws around sweepstakes vary not only by state, but even by municipality, in some cases. </p> <p>When offering incentives to each early respondent, make certain to prompt them for an electronic or physical address to deliver the incentive to. Electronic incentives are easier to administer, but not all are created equal: in one study, receiving a $5 credit on the credit card of a respondent's choice provided a 30% higher return than crediting frequent flier miles.</p> <p>Incentives, like subject titles and the body of survey invitations, requires a bit of creativity. Whoever first offered an iPod as an incentive was a genius; whoever offered it second was clever, and whoever offered it last week was disappointed by the response it generated. Relevant incentives motivate invitees.</p> <p>What's your favorite example of a good survey incentive?</p> </article> <article> <h1>Rating Scale Comparison: Weighing Different Scales for Survey Research</h1> <p>Jeffrey Henning — Wed, 25 Mar 2009 08:40:00 GMT</p> <p>If there is one thing I dislike, it is giving people advice about scales. As Brian Koma, our vice president of marketing puts it, "People will defend which scale they use like they're fighting a holy war."</p><p>So here's some advice on why the three-point scale, the five-point scale, the seven-point scale, the ten-point scale and the eleven-point scale are all the absolute best rating scale for you to use! </p><ul><li><b>Three-item scale</b>: "Using only three answer options takes up a lot less visual space and is far less daunting than five options. Thus, we always try to limit the response options to three, as that number offers the robustness to capture what we need while still remaining visually inviting." - Ziggy Zubric, owner of Marketing Endeavors, "<a class="" mce_href="http://blog.vovici.com/Blog/bid/18112/Less-Really-is-More-When-it-Comes-to-Response-Scales" href="http://blog.vovici.com/Blog/bid/18112/Less-Really-is-More-When-it-Comes-to-Response-Scales">Less Really is More When it Comes to Response Scales</a>"</li><li><b>Five-point scale & seven-point scale</b>: "To explore the relation between scale length and reliability, we conducted a meta-analysis of the results of many past studies. Our data consist of results from 706 tests of reliability taken from thirty different between-subject studies. We combined various measures of reliability and various sample sizes, controlling for these and other factors in determining the relation of scale length to reliability. In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five." - Jon Krosnick, professor of communication at Stanford, "<a target="_blank" href="http://communication.stanford.edu/faculty/krosnick.html">The Optimal Length of Rating Scales to Maximize Reliability and Validity</a>"</li><li><b>Ten-point scale</b>: "A five-point scale is totally inappropriate for customer satisfaction studies. Why? It lacks enough granularity and robs companies of a burning desire to take corrective action. It commonly leads executives to believe that ‘80% rate us four or five; that's great, let's move on,' without realizing that it simply means that 80% are at least somewhat satisfied. Further, many people will never rate anything a ‘five,' resulting in ‘four' including those who are really very satisfied and those who are only somewhat satisfied. To avoid this topping effect, use at least a 10-point scale and count nine and 10 ratings as fully satisfied. This will also allow easier analysis of what bottom-line effects satisfaction has, since such tools as regressions work better with a more granular score." - Brad Bortner, principal analyst with Forrester Research, "<a target="_blank" href="http://www.forrester.com/Research/Document/0,7211,45043,00.html#reference8">Best Practices: Why Customer Satisfaction Studies Fail</a>"</li><li><b>Eleven-point scale</b>: "The 0-to-10 scale has many significant advantages: Customers find that the scale makes intuitive sense..., most of the world already uses the metric system..., customers may refuse to give anybody a perfect score..., customers will transpose the top and bottom on a 1-to-10 scale..., scales with fewer points seem more susceptible to grade inflation..., the 0-to-10 standard is being adopted by many of the world's leading companies." - Fred Reichheld, Fellow with Bain & Company, The Ultimate Question, p. 98-99</li></ul><p>What's your favorite size for a rating scale, and why do you prefer it?</p> </article> <article> <h1>ACSI (American Customer Satisfaction Index) Model: Strengths and Weaknesses</h1> <p>Jeffrey Henning — Tue, 24 Mar 2009 08:31:00 GMT</p> <p>The <a href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation" class="" mce_href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation">ACSI Score</a> is just one of five <a href="http://blog.vovici.com/Blog/bid/18157/Multi-Item-Scale-Index-Construction-in-Survey-Research" class="" mce_href="http://blog.vovici.com/Blog/bid/18157/Multi-Item-Scale-Index-Construction-in-Survey-Research">multi-item scales</a> that make up the expanded model of the American Customer Satisfaction Index. Each multi-item scale represents a different aspect of customer attitudes: Customer Expectations, Perceived Overall Quality, Perceived Value, Customer Satisfaction and Customer Loyalty (Customer Complaints, shown in the graphic below, is based on a single question).</p> <p align="center"></p> <p>The American Customer Satisfaction Index has greater predictive validity than most other customer-satisfaction measures. </p> <ul> <li>Macroeconomically, ACSI has been shown to correlate to GDP and PCE (Personal Consumption Expenditure) growth ("The Effect of Buyer Satisfaction on Consumer Spending Growth", Fornell & Rust, 2005).</li> <li>Microeconomically, <a href="http://cache.consumerist.com/assets/resources/customersatsifactionequalsprofts.pdf" target="_blank">ACSI predicts stock market performance</a> for indices as well as individual stocks and <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1004117" target="_blank">even correlates to CEO bonuses</a>.</li></ul> <p>Since the ACSI was started in 1994 and is used as a benchmark measuring changes to satisfaction over time, the underlying model has not been changed significantly despite subsequent research into its shortcomings. Most of the following criticisms are from "<a href="http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V8H-430G2XK-6&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=e5ea5116d86e2f770a7f485ae9508a00">The evolution and future of national customer satisfaction index models</a>" (Johnson, Gustafsson, Andreassen, Cha; 2001). </p> <ol> <li><b>No WOM/Recommendation Index</b> - By contemporary standards, the model is remiss in that in leaves out any measurement of positive word of mouth. Early word-of-mouth research focused on complaining behavior (Gronhaug and Kvitastein, 1991; Singh, 1988), as does the ACSI model. Starting in 1991 and popularized in 1995, WOM research has shifted to recommendations and customer advocacy (Brown et al., 2005; Christopher et al., 1991; Jones and Sasser, 1995).</li> <li><b>Complaints are a Driver not Consequence of Satisfaction</b> - When the ACSI model was first constructed, there was little awareness of the effects of complaint resolution on satisfaction. Complaints are in the wrong place in the model (Johnson, et al; 2001).</li> <li><b>Satisfaction as an Intermediary</b> - The effects of Quality, Value and Expectations on Loyalty are all mediated by the cumulative satisfaction index. In reality, Quality and Value most likely directly affect Loyalty without going through Satisfaction, as this explains <a href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model" class="" mce_href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model">how Satisfaction and Loyalty can diverge</a>. Value, in particular, is important when a customer re-evaluates whether to remain loyal (Johnson, et al; 2001).</li> <li><b>Expectations' Effect on Cumulative Satisfaction</b> - The link from the Expectations index to the Perceived Value index is weak, as is the link to the Customer Satisfaction index. Expectations correlate to Satisfaction less closely as time from initial transaction increases. This index can be safely omitted and in fact is omitted from the revised Norwegian Customer Satisfaction Barometer. </li> <li><b>Link from Quality to Value Problematic</b> - This link has no sound theoretical basis (Johnson, et al; 2001).</li> <li><b>Value and Quality Indices Overlap</b> - By creating a separate Price Index, Value can be removed and better correlations obtained (Johnson, et al; 2001).</li> <li><b>Uses Partial Least Squares for SEM</b> - See "<a href="http://www.emeraldinsight.com/Insight/viewContentItem.do?contentType=Article&hdAction=lnkhtml&contentId=842092" target="_blank">Customer Satisfaction in a Reduced Rank Regression Framework</a>", Pietro Giorgio Lovaglio, 2004.</li> <li><b>FID (Fuzzy Influence Diagrams) May Outperform SEM (Structural Equation Modeling)</b> - FID has only recently been applied to the measurement of customer satisfaction and seems to outperform SEM "in solving the problems of nonlinearity, validity and causality" (<a href="http://ieeexplore.ieee.org/iel5/4405868/4406334/04406437.pdf" target="_blank">Na An, Jinlan Liu, Yin Bai, 2007</a>).</li></ol> <p>The expanded ACSI model would be a great foundation for another academic organization to use to build a modern customer satisfaction and loyalty model, unconstrained by the need for complete backward compatibility.</p> </article> <article> <h1>Qualitative and Quantitative Research Fusion, Powered by Online Communities</h1> <p>Jeffrey Henning — Mon, 23 Mar 2009 07:48:00 GMT</p> <p><a href="http://www.forrester.com/rb/analyst/brad_bortner" target="_blank" mce_href="http://www.forrester.com/rb/analyst/brad_bortner">Brad Bortner</a>, a principal analyst with Forrester Research, has a new independent white paper, "<a href="http://www.forrester.com/Research/Document/Excerpt/0,7211,46896,00.html" target="_blank" mce_href="http://www.forrester.com/Research/Document/Excerpt/0,7211,46896,00.html">Fused Research Modes Will Save You Money</a>: How To Master The Faster And Cheaper Imperative In Stark Economic Times". Here's the summary [the links are mine]: </p> <blockquote> <p>The economy is both eroding financial wherewithal to conduct market research and putting a larger burden on market research professionals to identify successful markets. While vast economies have already been realized in <a href="http://blog.vovici.com/Blog/bid/17986/Qualitative-vs-Quantitative-Research" class="" mce_href="http://blog.vovici.com/Blog/bid/17986/Qualitative-vs-Quantitative-Research">quantitative research</a> using online panels, the same benefits are about to be realized in traditional qualitative research. New <a href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR" class="" mce_href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR">fusions of qualitative and quantitative research</a> approaches have finally emerged that are <a href="http://blog.vovici.com/Blog/bid/18013/Research-Communities-are-Affordable" class="" mce_href="http://blog.vovici.com/Blog/bid/18013/Research-Communities-are-Affordable">economically viable alternatives</a> to traditional and expensive approaches. Companies must harness them or resign themselves to doing less with less at a time when sharper market insights are more necessary than ever.</p></blockquote> <p>A graphical representation of one form of this fusion of techniques:</p> <p mce_keep="true" align="center"><a href="http://blog.vovici.com/Blog/bid/17989/Ideas-Prioritized-Faster-than-Ever" class="" target="_new" mce_href="http://blog.vovici.com/Blog/bid/17989/Ideas-Prioritized-Faster-than-Ever"></a></p> <p mce_keep="true">A standing online community takes the place of a series of <a href="http://blog.vovici.com/Blog/bid/17983/Focus-Groups-vs-Online-Communities" class="" mce_href="http://blog.vovici.com/Blog/bid/17983/Focus-Groups-vs-Online-Communities">focus groups</a>. Questions can be posed to the community regularly to develop qualitative insights. These qualitative insights can then be used to shape survey instruments; members of the panel - which includes, as a subset, active community members - are invited to take the survey. Fielding the survey beyond the community provides better representation than using just the community itself.</p> <p mce_keep="true">Why is that important? Too often qualitative insights that illustrate a subset of the target population are taken as representing the overall population. Catharine Taylor recently posted "<a href="http://www.mediapost.com/publications/?fa=Articles.showArticle&art_aid=102374">Listen Up, Marketers: The Focus Group Is Dead</a>", which she illustrates with examples of four brands that mistook qualitative research as being representative of their target audience: Tropicana, Motrin, Facebook and The Sci Fi Channel. Each brand faced a public-relations backlash as a result of the changes they made based on that research.</p> <p mce_keep="true">Clearly, as Brad writes, the fusion of qualitative and quantitative research is more important than ever. If you haven't considered deploying <a href="http://www.vovici.com/survey-solutions/online-communities.aspx">online community software</a> and connecting it to your <a href="http://www.vovici.com/survey-software/online-survey-software.aspx">survey tool</a>, now is a good time to do so.</p> </article> <article> <h1>Multi-Item Scale (Index) Construction in Survey Research</h1> <p>Jeffrey Henning — Fri, 20 Mar 2009 21:20:00 GMT</p> <a class="" target="_new" mce_href="http://www.amazon.com/Handbook-Marketing-Scales-Multi-Item-Association/dp/076191000X/" href="http://www.amazon.com/Handbook-Marketing-Scales-Multi-Item-Association/dp/076191000X/"></a>In the past week, I was asked to give advice to a prospect who wanted to create their own multi-item measure (also known as an index or latent variable). A multi-item or summated scale consists of a number of ratings combined into a single value. Traditionally, multi-item scales are used to represent complex psychological constructs that can't be summarized in a single question: attitudes, stress levels, personality, loyalty and satisfaction, among others.<br><br>Multi-item measures are more reliable and less volatile than single-item questions (see <a target="_blank" href="http://www.amazon.com/Customer-Satisfaction-customer-experience-customers/dp/0955416116/">Customer Satisfaction - The customer experience through the customer's eyes</a>). For instance, if a respondent misunderstands one part of the question, it will affect part of the measure rather than all of the measure. As a result of this increased stability, multi-item scales make excellent benchmarks: <ol><li>The <a class="" mce_href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation" href="http://blog.vovici.com/Blog/bid/18135/ACSI-American-Customer-Satisfaction-Index-Score-Its-Calculation">ACSI Score</a> is a superb example of a multi-item satisfaction scale.</li><li>For those of our customers who use the <a class="" mce_href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model" href="http://blog.vovici.com/Blog/bid/17877/The-Apostle-Model">Apostle Model</a>, we advise them to use two indices: <a class="" mce_href="http://blog.vovici.com/Blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template" href="http://blog.vovici.com/Blog/bid/18079/Apostle-Model-Best-Practices-and-Survey-Template">one for satisfaction and one for loyalty</a>.</li><li>An ISP (Internet Service Provider) developed a Renewal Likelihood Index consisting of three questions designed to measure the probability of an individual customer renewing when their annual service contract was up. As the predictive value of this measure proved itself over time, all surveys to customers included these three questions and a low score would send an email trigger to alert a service representative.</li></ol><p>A more affordable approach than developing your own the way that ISP did is to adopt a proven index used by others. I've had a copy of the first edition of <a target="_blank" href="http://www.amazon.com/Handbook-Marketing-Scales-Multi-Item-Association/dp/076191000X/">Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behavior Research</a> at hand for 15 years now for specifically this reason. While you could define for yourself a multi-item measure (latent variable) simply by summing or averaging the answers to two or more questions (manifest or observable variables) that you selected, it is unlikely that you would have hit upon a very effective measurement. According to Robert Peterson in <a target="_blank" href="http://www.amazon.com/Constructing-Effective-Questionnaires-Robert-Peterson/dp/0761916407/">Constructing Effective Questionnaires</a>, a good index needs to be unidimensional, reliable, valid and generalizable:</p><ul><li><b>Unidimensional</b> - A multi-item scale needs to measure only one attribute or behavior.</li><li><b>Reliable</b> - The scale needs to produce similar results over time, and its internal measures need to be consistent with one another.</li><li><b>Valid</b> - The scale needs to demonstrate that it actually measures what it is meant to measure: if it predicts that customers will recommend your brand, successive research should demonstrate that those customers do in fact recommend your brand.</li><li><b>Generalizable</b> - The scale should work in multiple collect modes (web, paper, telephone, face-to-face) and across the target population.</li></ul><p>I point out these requirements primarily to discourage you from attempting to develop your own multi-scale measures: this is actually a rigorous science known as psychometrics. If your organization can benefit from a proprietary measure of satisfaction or loyalty, then you are best served by hiring a psychometrician to develop the measure for you.</p><p>And that was the advice I gave our prospect, who wanted to develop a metric of satisfaction and loyalty (which would not have been unidimensional, since satisfaction and loyalty are independent).</p> </article> <article> <h1>Satisficing and Survey Respondent Behavior</h1> <p>Jeffrey Henning — Thu, 19 Mar 2009 21:03:00 GMT</p> <P>As survey authors, we ask a lot of our respondents-literally and figuratively. In "<A href="http://doi.wiley.com/10.1002/acp.2350050305">Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys</A>", Jon Krosnick outlines four mental steps that respondents work through to answer a survey question: </P> <OL> <LI>Interpret the meaning of a question</LI> <LI>Recall all relevant facts related to the question</LI> <LI>Internally summarize those facts</LI> <LI>Report summary judgment accurately </LI></OL> <P>Ideally, we want responses to carefully think through every answer they provide, but: <B>the respondent is not a machine, your survey is not a test</B>. Not every respondent is as conscientious as we'd like. They realize they are not being graded on their answers. This will not affect their permanent record.</P> <P>As a result, respondents take shortcuts, which Krosnick describes as satisficing, an economic term that is a blend of satisfy and suffice. In economics, consumers are supposed to select the optimal product or service for their needs, but in reality consumers often choose a satisfactory solution rather than the optimal solution, because it is too time consuming to evaluate all the available alternatives. In surveys, respondents are supposed to select the optimal answer, but in reality respondents often choose a satisfactory answer instead.</P> <P>When a respondent works through each of the four mental steps with a maximum level of effort, they are optimizing their answers. When a respondent works through each step with minimal effort, they are engaging in weak satisficing, according to Krosnick. When they skip the steps of recall and internal summarization, they are strong satisficing. When they are simply jumping to a conclusion or selecting any answer, I consider them to be cheating.</P> <P><A style="DISPLAY: inline" onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/respondent%20coping%20strategies.png"></A> <BR><BR>Some of the behaviors that are weak satisficing include: </P> <UL> <LI>Selecting the first choice that appears reasonable in a list of choices, rather than reading the entire list</LI> <LI>Agreeing with assertions without thinking the contrary position through ("acquiescence response bias")</LI></UL>Strong-satisficing behaviors include:<A onclick="window.open( this.href, '_blank', 'width=640,height=480,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false" href="http://blog.vovici.com/.a/6a00d83548632853ef0112797b06df28a4-popup"></A> <UL> <LI>Endorsing the status quo instead of change</LI> <LI>Failing to differentiate in ratings</LI> <LI>Selecting "Don't know" rather than expressing an opinion</LI> <LI>Randomly choosing an answer (which Krosnick categorizes as strong satisficing but which I consider to be cheating)</LI></UL> <P><A style="DISPLAY: inline" onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/satisficing%20examples.png"></A> <BR><BR>A well-designed survey needs to account for these behaviors. Some tips from a follow-on paper by Krosnick, et al, "<A href="http://doi.wiley.com/10.1002/ev.1033">Satisficing in surveys: Initial evidence</A>": </P> <UL> <LI>In the introduction, let the respondent know the importance of providing accurate answers.</LI> <LI>Inform respondents of the value of their input and how its results will be used.</LI> <LI>After some questions, ask the respondent to justify their answer with an open-ended question.</LI> <LI>Re-factor complex questions into simpler questions (especially re-factor matrix questions, see <A href="http://www.surveysampling.com/files/imce/SSI_Question_Design.pdf" target=_blank>SSI's just published study on Grid Questions</A>).</LI> <LI>Rewrite questions for a lower reading level.</LI></UL> <P>Some tips of my own, some specific to web surveys: </P> <UL> <LI>When a <A href="http://vovici.web6.hubspot.com/Blog/bid/18095/Open-Ended-Questions#bestpractices" mce_href="http://vovici.web6.hubspot.com/Blog/bid/18095/Open-Ended-Questions#bestpractices">choose-one question has no logical order</A>, randomize the list of choices. If you don't do this, a choice near the top of the list might be selected more often than it deserves.</LI> <LI>To avoid acquiescence, recast agreement questions as offering multiple alternatives rather than one implicit alternative (disagreement). Some assessments later in the questionnaire ask for agreement with the inverse of an earlier statement to test respondent reliability; this lengthens the questionnaire and should be done sparingly.</LI> <LI>Avoid <A href="http://blog.vovici.com/Blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires" mce_href="http://blog.vovici.com/Blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires">Yes/No questions</A> that no one wants to say no to.</LI> <LI>On rating scales, omit a "Don't know" option to force respondents to select a rating. (Continue to use "Don't know" options when asking respondents to recall past action and choose from a list of alternatives.)</LI></UL> <P>Respondents aren't machines, executing questionnaires in their heads like source code, and questionnaire designers need to plan accordingly.</P> </article> <article> <h1>Hierarchical Questions Enable Respondents To Choose from Thousands of Choices</h1> <p>Jeffrey Henning — Wed, 18 Mar 2009 20:02:00 GMT</p> <P>When writing a questionnaire, sometimes you have thousands of choices that you want a respondent to choose from: the make and model of a car, the brand and name of a specific product, the state and county a respondent lives in. Showing the respondent a list of all the choices would be impractical. You could provide the respondent an open-ended question, but then you would have hundreds, thousands or even tens of thousands of responses to read through and categorize. (For my first job in market research, I would sometimes get called on to code such lists of responses: as time-consuming and tedious a task as you can imagine.)</P> <P>A better alternative is to present the user a hierarchical question (click image for a working demo):</P> <P align=center mce_keep="true"><A href="http://www.vovicifeedback.com/se.ashx?s=4CA210A5668EEEFC" target=_new mce_href="http://www.vovicifeedback.com/se.ashx?s=4CA210A5668EEEFC"></A></P> <P>The choices are arranged in an outline, but the respondent only sees the first-level choices, and then only the second-level choices of the chosen first level. (This can be repeated for three levels: for instance, make, model and year of car.) </P> <P>While creating and maintaining the outline of possible choices becomes an ongoing chore, it takes less effort than building such an outline from respondents' choices. And it makes for a much better experience for respondents.</P> </article> <article> <h1>Survey Monkey Over a Barrel</h1> <p>Jeffrey Henning — Tue, 17 Mar 2009 14:18:00 GMT</p> <p>Many Vovici customers came to us after <a href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business" class="" mce_href="http://blog.vovici.com/Blog/bid/18131/Survey-Monkey-Business">outgrowing SurveyMonkey</a>, which is an excellent "first barrel of monkeys" for playing around with <a href="http://www.vovici.com/survey-solutions/web-survey.aspx" class="" mce_href="http://www.vovici.com/survey-solutions/web-survey.aspx">web surveys</a>. Some of our customers who moved upmarket: </p> <ul> <li>A professional association wanted a tool as easy to use as SurveyMonkey but with more flexible reporting and analysis.</li> <li>A market-research firm was using SurveyMonkey but ran into difficulties writing screener questions (which often require advanced branching) and needed rich analytics and application integration.</li> <li>A professional services firm needed to field a survey simultaneously in six languages and wanted all the results to collect in one data table rather than six separate tables. </li> <li>A government agency was using SurveyMonkey and Zoomerang, among other survey tools, and wanted to standardize on one system across the agency. </li> <li>A telecommunications equipment manufacturer wanted to move from pockets of <a href="http://blog.vovici.com/Blog/bid/18116/Survey-Software-in-the-Cloud" class="" mce_href="http://blog.vovici.com/Blog/bid/18116/Survey-Software-in-the-Cloud">survey software to enterprise feedback management</a> across the company. </li> <li>A university liked the free-form word processor of <a href="http://www.vovici.com/demo/">Vovici v4</a> and the time-savings of its report generation.</li></ul>As a research analyst at the university said, "Everybody wants to do a survey for everything under the sun. The demand for surveys is already high because there is a culture of data-driven strategic planning. The power coming out of Vovici's app has stoked an increased demand. People get addicted to data through SurveyMonkey but realize how hard it is to create a great and successful survey. They ultimately realize how restricted they are, so they come to us to use Vovici." </article> <article> <h1>Reminder Invitations Double Survey Response Rate</h1> <p>Jeffrey Henning — Mon, 16 Mar 2009 14:11:00 GMT</p> <P>We often argue for the need to respect your potential respondent's time and interrupt them as little as possible. Yet our <A href="http://www.vovici.com/">survey software</A> includes the ability to schedule the <A href="http://blog.vovici.com/vovici_blog/2009/02/writing-survey-invitations-six-points-to-cover.html">survey invitation</A> and as many follow-up email reminders as you would like:</P> <P align=center></P> <P>Reminders can be an annoying nuisance to recipients and should be avoided unless really necessary to your survey. Here are when they are necessary: </P> <UL> <LI>You are surveying a small population (e.g., employees, major accounts) and have not yet reached your <A href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys" mce_href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys">recommended sample size</A> for 95% confidence.</LI> <LI>You are attempting a <A href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey" mce_href="http://blog.vovici.com/Blog/bid/18130/The-Case-For-and-Against-a-Census-instead-of-a-Survey">census instead of a survey</A>. </LI> <LI>You suspect that initial responders may vary in key ways from later responders. For some studies, when we run the numbers we find no measurable difference between early and late respondents; for others, we find important differences in satisfaction and loyalty.</LI> <LI>You are using <A href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" mce_href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">email triggers</A> to alert you to dissatisfied respondents and want to make sure that you have the opportunity to address their issues. </LI></UL> <P>Following up survey invitations with reminders is the most dramatic way to improve your response rate. For one recent study, a series of three reminders doubled the response rate:</P> <P align=center></P> <P>Many of our users do not send out reminders because they are concerned about harassing potential respondents. However, since over 80% of responses to a closed-participation survey come within 24 hours of sending out an email invitation or reminder, think of that first reminder as doing the recipient a favor: often they meant to respond, but went on to other things, and the invitation got pushed down further in their in-box. Your reminder brings it back to the top of their mind - and their in-box.</P> <P>One way to minimize the pain of reminders is to make sure that each recipient is given a unique hyperlink to the survey (this may be inappropriate in certain surveys where anonymity is vital). That way, reminders will only be sent to those recipients who have not yet completed the survey.</P> <P>The only thing more annoying than receiving an email reminder about a survey you haven't taken is to receive a reminder to one that you have taken!</P> </article> <article> <h1>Yes/No Matrix Questions</h1> <p>Jeffrey Henning — Fri, 13 Mar 2009 12:47:00 GMT</p> <p>As I reviewed questionnaires with <a href="http://blog.vovici.com/Blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires" class="" mce_href="http://blog.vovici.com/Blog/bid/18123/Yes-No-Questions-Common-Pitfalls-to-Avoid-When-Writing-Questionnaires">Yes/No questions to identify common mistakes</a>, I encountered quite a few Yes/No matrixes. Here's one of my own:</p> <p align="center"> </p> <p>An interesting decision for the survey author is whether or not to show such questions as choose-many questions (checkbox lists) instead:</p> <p align="center"> </p> <p><a href="http://www.amazon.com/Asking-Questions-Practical-Questionnaire-Behavioral/dp/0875895468">Some researchers</a> have suggested that converting select-all-that-apply questions to Yes/No matrix questions ("forced-choice questions") with each answer required will improve the quality of results. The rationale is that respondents will need to spend more time answering each question.<br><br>Certainly, this question format is more time-consuming, since respondents must click a choice for every single item, whereas with a choose-many question they only need to check the items that apply. Since I am usually in favor of reducing the burden on the respondent as much as possible, I would normally never even consider using Yes/No matrices instead of checkboxes.<br>But before I rule in favor of the choose-many question in this case, let's hear from the defense. In the paper "<a href="http://www.sesrc.wsu.edu/dillman/papers/Check%20all%20Draft%20_final%20revision_.pdf">Multiple Answer Questions in Self-Administered Surveys: The Use of Check-All-That-Apply and Forced-Choice Question Formats</a>" [PDF] - by DA Dillman, JD Smyth, LM Christian and MJ Stern - the authors found that respondents select more choices in the force-choice format (Yes/No or equivalent labels) than in the standard checkbox format. Which format is more accurate is yet to be determined. The authors write that "forced-choice formats cannot simply replace check-all-that-apply formats and decisions about which format to use should be evaluated for each individual survey and question."<br><a onclick="window.open(this.href,'_blank','scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false" href="http://www.thebawdycloister.com/reliquary/2005/07/luke_skywalker_.html"></a> Such a lukewarm recommendation is the burden of a scientific researcher. As a blogger, I bear no such burden and can happily pontificate about when you should use which format. By default, I would use the checkbox format. The only exception I would make is when the list of possible choices is long (more than 7 items) and it is imperative that no choice item that should be selected is skipped. The Yes/No matrix is a speed bump in the questionnaire, and the respondent will pay more attention to each of its items than in a checkbox list. <br>Here are two of the few applications I can think of for the Yes/No matrix: </p> <ol> <li>When, as part of a technical support satisfaction survey, you are asking respondent to report all the steps they took as part of the problem resolution, consider using the Yes/No question matrix to make sure that no step was missed.</li> <li>When asking a consumer immediately after a retail purchase to recall a list of specific actions taken by the cantina staff, since some retailers grade their outlets based on franchise consistency.</li></ol>And for those of you who think I should have included Clone Wars in my list of Star Wars movies, I confess that I gave up on George Lucas while he was still making live-action Star Wars films. I haven't even seen all three prequels. Sad words to write for a kid who had the entire set of Burger Chef Star Wars movie posters in his room. </article> <article> <h1>Survey Portal: Panelist Web Sites Increase Response Rates</h1> <p>Jeffrey Henning — Thu, 12 Mar 2009 12:15:00 GMT</p> <p>A survey portal is a web site which your customers or employees can visit to see a list of surveys they've been invited to, as well as to review or update their profiles. The portal is a convenient destination for customers to use to manage their interactions with your <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a> system. </p> <p><br><a href="http://blog.vovici.com/.a/6a00d83548632853ef01127947d05428a4-pi"> </a></p><div style="text-align: center;"><a href="http://vovici.web6.hubspot.com/Portals/60483/images/6a00d83548632853ef01127947d05428a4-pi.png" style="display: inline;"></a> </div><br>Common components of a survey portal include: <ul> <li><b>A list of profiles</b> that panelists can complete. <a href="http://blog.vovici.com/Blog/bid/17932/Tips-for-Defining-Community-Member-Profiles" class="" mce_href="http://blog.vovici.com/Blog/bid/17932/Tips-for-Defining-Community-Member-Profiles">Well-designed profiles</a> should be <a href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors" class="" mce_href="http://blog.vovici.com/Blog/bid/18021/CRM-Connectors">synchronized with your CRM</a> or HRIS system.</li> <li><b>A list of surveys</b> that panelists have been invited to. Outstanding invitations are listed.</li> <li><b>Completed questionnaires</b>. For later review by the panelist.</li> <li><b>Shared survey results</b>. Aggregate reports from completed surveys, as well as discussions of how the organization is responding to the feedback it gathered.</li></ul>Some of our customers have reported that their survey response rates doubled once they added a portal! One advantage of portals is that even if a member <a href="http://blog.vovici.com/Blog/bid/18071/Unsubscribe-Survey-Templates" class="" mce_href="http://blog.vovici.com/Blog/bid/18071/Unsubscribe-Survey-Templates">unsubscribes</a> from receiving <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">survey invitations</a>, they will still be able to log on to the portal and see the surveys that they've been invited to. While many survey portals are standalone, those survey portals that generate the highest response rates provide respondents multiple reasons for logging in: for instance, to check on the status of open support tickets, see vendor headlines and news, and access other customer-only documents.<br>Survey portals cannot be considered <a href="http://blog.vovici.com/Blog/bid/17991/MROC-Market-Research-Online-Community" class="" mce_href="http://blog.vovici.com/Blog/bid/17991/MROC-Market-Research-Online-Community">MROCs</a> because they do not provide a method for <a href="http://blog.vovici.com/Blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels" class="" mce_href="http://blog.vovici.com/Blog/bid/18066/Social-Networks-vs-Online-Communities-vs-Panels">panelists to communicate with one another</a>. While they start as simply interfaces to panels, survey portals are one step away from reliance on email lists and <a href="http://blog.vovici.com/Blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009" class="" mce_href="http://blog.vovici.com/Blog/bid/18069/The-Top-Ten-Reasons-for-Building-an-Online-Community-in-2009">one giant leap toward online communities</a>. </article> <article> <h1>ACSI (American Customer Satisfaction Index) Score & Its Calculation</h1> <p>Jeffrey Henning — Wed, 11 Mar 2009 11:34:00 GMT</p> <p>The American Customer Satisfaction Index (ACSI) is the most well known national customer satisfaction index model, a type of economic indicator that assesses the overall satisfaction of consumers in a country. The ACSI is compiled by the National Quality Research Center (NQRC) at the University of Michigan. While intended as a macroeconomic measure of U.S. consumers in general, many corporations have used it to measure the satisfaction of their own customers.</p><p>The heart of the American Customer Satisfaction Index is a set of three questions that assess satisfaction, each on a different 10-point scale:</p><p align="center"></p><p>Some organizations normalize and average the three ratings, like this:<br><br></p><blockquote><p><b>(Satisfaction + Expectancy + Performance - 3) / 27 * 100</b></p></blockquote><p><br>This produces an overall score from 0 to 100 that can be used as an approximate benchmark to <a target="_blank" href="http://www.theacsi.org/index.php?option=com_content&task=view&id=148&Itemid=156">industry results published by TheACSI.org</a>. </p><p mce_keep="true" align="center"><a style="display: inline;" href="http://vovici.web6.hubspot.com/Portals/60483/images/ACSI-score.png" onclick="window.open( this.href, '_blank', 'width=800,height=560,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false"></a> </p><p>The true ACSI customer-satisfaction score is a weighted average of the answers to each of these three questions, using a proprietary weight for each of the three questions, with different weighting schemes for different industries. For instance, overall satisfaction is typically given a higher weight than expectancy, which is given a slightly higher weight than performance. The State of Ohio uses the following weights:<br></p><blockquote><p><b>((Satisfaction-1)*.3885 + (Expectancy-1)*.3190 + (Performance-1)*.2925) / 9 * 100</b></p></blockquote><p><br>If you want the precise results for your industry, you will want to commission a custom ACSI research program with ACSI, for $50,000; this will include the precise question wording and survey methodology used for gathering the benchmark data. Otherwise the above question phrasing and calculation can be used for a Do-It-Yourself customer-satisfaction index and benchmark.</p> </article> <article> <h1>Survey Response Rate Directly Proportional to Strength of Relationship</h1> <p>Jeffrey Henning — Tue, 10 Mar 2009 11:24:00 GMT</p> <p><a onclick="window.open( this.href, '_blank', 'width=940,height=645,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0' ); return false" href="http://vovici.web6.hubspot.com/Portals/60483/images/Survey%20Response%20Rate%20Directly%20Proportional%20to%20Strength%20of%20Relationship.png" style="display: inline;"></a>We're often asked by customers what response rate they can expect from a survey that they are about to launch. While many factors affect the response rate - including how long the survey is, what incentives are offered (if any), the subject matter of the study, and so on - we find that the biggest factor is the strength of the organization's relationship with its potential respondents.<br><br>Response rates to web surveys seem to vary in roughly the following ranges depending on who the target audience is: </p> <ul> <li><b>No Relationship</b> - 0%-20% response. The most demanding group to survey is a group that has no relationship with your organization. The 0% response is from a general market research survey that we did to a rented list, asking recipients about "Your last visit to the dentist"! (We repeated the mailing, with an incentive, to get the needed results.)</li> <li><b>Lost Customer</b> - 10%-30% response. Since a lost customer has already made his or her decision and moved on, these recipients are less likely than other groups to respond to your survey.</li> <li><b>Prospective Customer</b> - 15%-35% response. While prospective customers are interested in you, they have about an average chance of responding to your request for a survey.</li> <li><b>Current Customer</b> - 20%-65% response. Current customers have a vested interest in providing feedback to an organization that they do business with. For customer-satisfaction studies, consumers have a lower response rate than businesses, and businesses in general have a lower response rate than major accounts.</li> <li><b>Student</b> - 25%-80% response. Student response rates vary widely, depending on the nature of the survey. Course-evaluation surveys for corporate-sponsored training tend to have lower response rates than surveys for semester-long classes.</li> <li><b>Employee</b> - 60%-90% response. Your organization has no stronger relationships than with its employees. As a result, employee-satisfaction surveys typically have a high response rate. Tactical employee surveys will provide lower response rates. </li></ul><b>Update:<br></b>Interested in tips and tricks for increasing response rates? View a complementary webinar given by Brian Koma and me: <a href="http://www.vovici.com/demo/recorded-boost_Mar122009.aspx" target="_blank" mce_href="http://www.vovici.com/demo/recorded-boost_Mar122009.aspx">Boost Survey Response Rates with Proven Tactics</a>. Also see the post, <a href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate" class="" mce_href="http://blog.vovici.com/Blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Rate">Reminder Invitations Double Survey Response Rates</a>. </article> <article> <h1>Recommended Survey Length</h1> <p>Jeffrey Henning — Mon, 09 Mar 2009 11:17:00 GMT</p> <p>Last week, to my surprise, I was disappointed by a survey I was invited to take, because it was too short! It asked me only three questions: one clarifying my usage, one asking me how likely I am to recommend it, and an open-ended question to explain my rating. That was it. It reminded me of my brother-in-law, who will call me from work occasionally to catch up, then cut it short to say he has to go back to work (as if I had called him!).</p> <p>In the case of this survey, the software company initiated the request for feedback, then "hung up" before I had a chance to really provide much input. They had my attention, and they squandered it, and ticked me off in the process. I had clicked on the link in the invite to take the survey because it was about the personal finance software I've used for years. While I'd admit it is much less fun to use this application than it used to be (as it resolutely tracks my holdings going down!), it is still an important program to me, one I use three to four times a month. I had quite a few points I wanted to make about where I thought they could improve.</p> <p>The software is important to me, but the software company made me feel that my opinion wasn't important to them. </p> <p>As an industry, we have heard the refrain from respondents that surveys are too long, and we've come up with <a href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" class="" mce_href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">ways to shorten questionnaires</a>. I have joined the chorus of complaint, as far more often I see surveys that are much too long than surveys that are too short. But I think I need to adjust my message: surveys should be the right length. That three-question survey would have been the right length it had been about my oil change. And I have had customers happily answer 100-question surveys, which can be the right length as part of an annual account review, when the customer is spending $1 million a year or more with your organization.</p> <p>After a couple quick searches, I couldn't find suggested questionnaire length by type of survey. Here are some ideas to start the discussion. </p> <ul> <li><b>2-4 questions - <i>Transactional Survey</i></b> - A quick follow-up to a standard service interaction, such as a retail purchase, a customer service call or a billing inquiry, designed to provide ongoing measurement of service quality. Reasons to exceed this length: measuring conformance with a Service Level Agreement or analyzing behavior patterns to shape better experiences in the future.</li> <li><b>5-10 questions - <i>Event Evaluation</i></b> - A more detailed follow-up than a transactional survey, acknowledging the respondent's investment of time and providing them an opportunity to rate speakers, venue and logistics. To be used to guide future event planning.</li> <li><b>10-20 questions - <i>Customer Satisfaction</i></b> - An automated follow-up to a purchase, after the recipient has had enough time to use the product for a while. Provides the customer the opportunity to rate the product across the broad areas that comprise the overall product experience.</li> <li><b>20-30 questions - <i>Planning</i></b> - An opportunity to gather detailed feedback of possible future direction for a product or service, to help the sponsoring organization prioritize.</li> <li><b>50-70 questions - <i>Major Account Review</i></b> - An annual assessment of customer satisfaction with an organization's largest customers, often with multiple individuals across multiple departments. For really long surveys, consider using an executive interviewer conducting face-to-face or telephone surveys to gather the results.</li> <li><b>70-90 questions - <i>Employee Satisfaction Review</i></b> - An annual or biennial measure of employee satisfaction, designed to prioritize HR initiatives and measure employee engagement.</li></ul>As a unit of measure, of course, a question is as bad as a <i>league</i> ("the distance a man can walk in an hour"): a question, after all, can be as short as a checkbox or as long as a multi-sided matrix. For practical purposes of following these guidelines, think of "a question" as the average single-select question. Treat that two-sided matrix with ten topics as 4 or 5 questions.<br><br>Now, if someone pressed you for the right length of a particular type of survey, what would you recommend? </article> <article> <h1>Survey Monkey Business</h1> <p>Jeffrey Henning — Fri, 06 Mar 2009 11:08:00 GMT</p> <p>Whenever a prospect tells us that the competition for their business is between SurveyMonkey and Vovici, we know that one of us is in the wrong place.</p> <p>SurveyMonkey is great for a consumer or blogger doing a quick survey. Vovici v4 is designed for business professionals interested in the systematic gathering of feedback, not just one-off surveys. The key features our prospects are looking for:</p> <ol> <li><b>Integration and automation.</b> Our customers are often conducting ongoing transactional surveys to measure satisfaction after a customer-service contact, a product order, a warranty repair or some other standard event. Using SurveyMonkey, this requires regular manual intervention. Vovici v4's rich API enables organizations to integrate <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">feedback management</a> with CRM systems, HRIS applications, customer-support systems, help-desk applications, web portals and more. </li> <li><b>Panel management.</b> As organizations do more and more surveys, they outgrow the simple list management of SurveyMonkey and want to maintain panels of customers, employees and resellers, with each panel storing a rich set of <a href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration" class="" mce_href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration">profile data</a>: demographic, <a href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template" class="" mce_href="http://blog.vovici.com/Blog/bid/18047/Firmographic-Definition-Template">firmographic</a> and attitudinal information that can be used to target <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">survey invitations</a>. They also want to track when individuals were last invited to a survey and how many surveys they've completed. The sophisticated clients want to use <a href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration" class="" mce_href="http://blog.vovici.com/Blog/bid/18044/Panel-Management-Software-and-Data-Integration">panel management</a> to <a href="http://blog.vovici.com/Blog/bid/17913/Treating-Your-Survey-Takers-like-Royalty" class="" mce_href="http://blog.vovici.com/Blog/bid/17913/Treating-Your-Survey-Takers-like-Royalty">treat their respondents like valued partners</a>.</li> <li><b>Advanced reporting.</b> Many organizations have unique, legacy report formats that have evolved over time to meet their needs. The standard SurveyMonkey reports won't do. Generating these custom reports each month can be quite time consuming and labor intensive. Vovici v4 APIs can be used to streamline this process. Other organizations wish to integrate reports with business-process data or <a href="http://blog.vovici.com/Blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day" class="" mce_href="http://blog.vovici.com/Blog/bid/18093/Hierarchical-Reports-So-Survey-Report-Generation-Isn-t-Like-Groundhog-Day">disseminate reports throughout the enterprise</a>, with individuals receiving their own unique views at the data. Vovici <a href="http://www.vovici.com/survey-analytics/survey-data-analysis.aspx">Feedback Intelligence</a> is the solution for these firms.</li></ol>If your organization needs any of the above, by all means call SurveyMonkey. Then call us: +1.703.481.9326 in the United States and +44 (0)208.547.4040 in Europe. When it comes to surveys, we mean business. </article> <article> <h1>The Case For and Against a Census instead of a Survey</h1> <p>Jeffrey Henning — Thu, 05 Mar 2009 11:01:00 GMT</p> <p>Typically, when consulting with large organizations about <a href="http://www.vovici.com/survey-solutions/enterprise-feedback-management.aspx">enterprise feedback management</a>, one of the big issues I encounter is that they have too many surveys done with too many of their customers. For such organizations, using a centralized panel system with <a href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">random sampling</a> dramatically reduces the frequency with which any individual customer is surveyed.<br><br>I want to be clear, though, that when I am criticizing a census I am really criticizing needless attempts at a census. For me, the canonical example will always be <a href="http://blog.vovici.com/Blog/bid/17985/Disconnected-Listening-An-EFM-Case-Study" class="" mce_href="http://blog.vovici.com/Blog/bid/17985/Disconnected-Listening-An-EFM-Case-Study">the automobile manufacturer that started subjecting their 8,000 dealers</a> to web surveys about all manner of issues, from the strategic to the tactical, simply because they could. Just like the tragedy of the commons, though, this wanton surveying ruined a good thing, as dealers stopped participating.<br>Although I describe this approach as a "census", it is really an attempt at a census, selecting everyone in the target population to participate but not reaching many of them. The 2000 United States Census surveyed 99.6% of the population, while census attempts by a corporation typically get much lower participation-as low as 20% for firms that overuse the technique.<br></p> <h2>Advantages of a Census</h2> <ul> <li>Every customer or employee is invited to participate. This is important for annual measures of <a href="http://www.vovici.com/survey-solutions/customer-satisfaction-survey.aspx">customer satisfaction</a> and <a href="http://www.vovici.com/survey-solutions/employee-satisfaction-survey.aspx">employee well-being</a>, which are opportunities for everyone to provide their feedback.</li> <li>Maximizes exposure to negative feedback. By contacting everyone, and by setting up <a href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction" class="" mce_href="http://blog.vovici.com/Blog/bid/18053/Survey-Alerts-Trigger-Emails-Improve-Satisfaction">email triggers</a> to alert you to negative responses, you will have the opportunity to intervene with more individuals to attempt to improve their satisfaction.</li> <li>Greater confidence levels in the results. For target populations with fewer than 1,000 individuals, <a href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys">the recommended sample size</a> is so large as a proportion that you must typically attempt a census to gain enough responses to have a high degree of statistical confidence that your results represent the target population.</li></ul> <h2>Disadvantages of a Census</h2> <ul> <li>Limits opportunities for other surveys. If everyone in the target population has just been invited to participate in a survey, they are unlikely to respond to an immediate subsequent request to take an additional survey. We once had a client that realized it needed to do some additional unrelated research, right after it had emailed its house list of 60,000 for a tactical survey.</li> <li>Leads to declining response rates. A survey important enough to a large organization for it to be sent as a census is typically longer than average. Inviting everyone to take a long survey makes them less likely to participate in the next survey.</li></ul>Sometimes you simply must do a census instead of a survey. Just make sure you are doing so because you have to. And, if you're in the States, remember to be kind to the census worker when he or she comes to your door next year. </article> <article> <h1>Yes/No Questions: Common Pitfalls to Avoid When Writing Questionnaires</h1> <p>Jeffrey Henning — Wed, 04 Mar 2009 08:11:00 GMT</p> <p>Nothing could be simpler than asking a Yes/No question in a survey, right? <br><br>Sorry, that was rhetorical.<br><br>Here are some of the common mistakes I see made when surveys include Yes/No questions. All of the examples are based on actual questionnaires I've reviewed, with the wording paraphrased to protect the guilty.</p> <p><b>Asking for a single Yes/No to multiple items.</b> </p> <blockquote> <p>Example: Was service prompt and courteous? Yes/No.</p></blockquote> <blockquote> <p>Someone who found the service prompt but not courteous (or vice versa) will probably feel conflicted but answer "No". When you write such a question, chances are that you are thinking of the two items as synonymous (as in "Was the policy clear and easy to understand?"); however, you've actually embedded two separate topics. Another example: "Do you think our campaign was effective at registering and mobilizing new voters?" A respondent might think the registration work was effective, but not the mobilization to the polls.</p></blockquote> <blockquote> <p>Solution: Split the problem question into two separate questions. "Was service prompt?" "Was service courteous?"</p></blockquote><b>Letting the user select both Yes and No.</b> <blockquote> <p>Example:</p> <p mce_keep="true"></p> <blockquote> <p>This may seem comical to you, but I've seen it literally dozens of times. Many first-time survey authors don't distinguish between <a href="http://blog.vovici.com/Blog/bid/18043/Checkboxes-vs-Radio-Buttons-in-Web-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18043/Checkboxes-vs-Radio-Buttons-in-Web-Surveys">checkboxes and radio buttons</a>.</p></blockquote> <blockquote> <p>Solution: Make sure to use a "choose one" or "single select" question type for all of your Yes/No questions.</p></blockquote></blockquote> <p mce_keep="true"><b>Force-fitting a question into a yes/no format by overriding what "Yes" or "No" means.</b> </p> <blockquote> <p>Example: Did you watch the documentary "No Direction Home: Bob Dylan"? (Please vote "yes" if you watched at least half of the film).</p></blockquote> <blockquote> <p>Solution: Either reword the question ("Did you watch any of the documentary...?") or instead give the user a list of choices to choose from: "How much of the documentary ‘No Direction Home: Bob Dylan' did you watch? (a) None of it (b) Less than half (c) About half of it (d) More than half (e) All of it."</p></blockquote> <p mce_keep="true"><b>Providing caveats to the Yes/No choices.</b> </p> <blockquote> <p>Example: </p></blockquote> <blockquote> <blockquote> <p>Have you been personally affected by asthma?</p></blockquote></blockquote> <blockquote> <blockquote> <ul> <li>No</li> <li>Yes, someone I care about has been affected by asthma</li> <li>Yes, I am a sufferer</li></ul></blockquote></blockquote> <blockquote> <p>The author started to write a Yes/No question, then decided that there were finer points that were important to capture. The problem is that these addendums to the choices are typically a sign that author did not think the question through thoroughly. For instance, how would someone who once had asthma but no longer does answer this question? (Never mind the poor word choice of a subjective term like "sufferer"; many respondents would see that term as too negative.)</p></blockquote> <blockquote> <p>Solution: Rewrite the question as a close-ended question with a thorough list of choices or as a series of Yes/No questions. "Has someone you care about been affected by asthma? Yes/No. Have you ever been diagnosed with asthma? Yes/No."</p></blockquote> <p mce_keep="true"><b>Asking questions that no one wants to say "no" to.</b> </p> <blockquote> <p>Examples: </p></blockquote> <blockquote> <blockquote> <p>Did you think using Facebook to reach young voters politically was an effective strategy?</p></blockquote> <blockquote> <p>Do you think we should continue to organize creative political initiatives?</p></blockquote></blockquote> <blockquote> <p>Solution: Instead ask the respondent to rate the items on a scale. "Please rate the importance of using Facebook to reach young voters on a scale of 0 to 10, where 0 is not at all important and 10 is very important."</p></blockquote> <p mce_keep="true"><br><b>Asking questions that can't be answered Yes/No.</b> </p> <blockquote> <p>Example: Should customer-service reps place a caller on hold while researching an answer, or is it okay to ask to call back? Yes/No.</p></blockquote> <blockquote> <p>Hard to believe this was even posed as a Yes/No question, but this is a paraphrase of just such a question. A survey author may write something like this after writing a series of Yes/No questions in a row.</p></blockquote> <blockquote> <p>Solution: Rephrase as a question with the appropriate choices. "When faced with a time-consuming issue to research, should the customer-service representative (a) place the caller on hold or (b) ask to call back once the answer is found?" </p></blockquote> <p mce_keep="true"><b>Listing a bunch of similar yes/no questions in a matrix.</b> </p> <blockquote> <p>Example: </p></blockquote> <blockquote> <blockquote> <p>Did the ombudsman greet you promptly?<br>Did the ombudsman explain the appeals process clearly?<br>Did the ombudsman answer all your questions about the appeals process?<br>Did the ombudsman treat you with respect?<br>Did the ombudsman call you after you received the decision?</p></blockquote></blockquote> <blockquote> <p>Solution: Rephrase to use checkboxes instead. This will be much more concise.</p></blockquote> <blockquote> <blockquote> <p>Did the ombudsman...? (Please check all that apply.)</p></blockquote></blockquote> <blockquote> <blockquote> <blockquote> <p>Greet you promptly<br>Explain the appeals process clearly<br>Answer all your questions about the appeals process<br>Treat you with respect<br>Call you after you received the decision<br>None of the above</p></blockquote></blockquote></blockquote> <p mce_keep="true">Too often, Yes/No questions are a cop-out for the busy or overwhelmed survey author. Often, you really need to give the respondent more choices. When that's the case, write a clear list of choices instead; perhaps review the <a href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions" class="" mce_href="http://blog.vovici.com/Blog/bid/18088/Open-Ended-Questions-vs-Closed-Ended-Questions">best practices for writing closed-ended questions</a>. Nothing could be simpler, right?</p> </article> <article> <h1>The Seven Deadly Sins of Online Community Management</h1> <p>Jeffrey Henning — Tue, 03 Mar 2009 07:54:00 GMT</p> <p>Community managers are human and imperfect. Here are the Seven Deadly Sins that community managers are sometimes guilty of:<br></p> <ol> <li><b>Pride: Preventing community members from criticizing you.</b> The fear of negative feedback in communities tends to be overblown; fiercely negative comments are atypical. Communities thrive on member-to-member communication, and the authenticity of the community is at stake if members can't make critical comments or find that those comments are deleted. <a href="http://blog.vovici.com/Blog/bid/17882/Gramma-C-s-Process-for-Handling-Irate-Community-Members" class="" mce_href="http://blog.vovici.com/Blog/bid/17882/Gramma-C-s-Process-for-Handling-Irate-Community-Members">Negative feedback is valuable feedback</a>; better to have it articulated in your own community where you can respond to it then have it only appear elsewhere on the Web.</li> <li><b>Greed: Failing to share with community members the results of feedback and the changes inspired by the community.</b> Where survey respondents typically don't expect to receive detailed summaries of the results, community members are participating in part to learn from their peers. They want to see how their views compare. And for feedback communities, they are participating because they value their relationship with you and they want to see you change to serve them better. It is critical to demonstrate to members how their feedback is translated into ideas.</li> <li><b>Lust: Using the community to try to win sales.</b> Survey researchers have confronted this and banned it: sugging (Selling Under the Guise of research), deemed unethical by standard market research codes of conduct. Don't treat the community as a lead generator for events and programs and sponsorships. Provide a <a href="http://blog.vovici.com/Blog/bid/18056/Writing-a-Community-Code-of-Conduct" class="" mce_href="http://blog.vovici.com/Blog/bid/18056/Writing-a-Community-Code-of-Conduct">community code of conduct</a> that discourages members from commercializing the community to one another. A good community is primarily <a href="http://blog.vovici.com/Blog/bid/17980/Social-Norms-and-Market-Norms-in-Online-Communities" class="" mce_href="http://blog.vovici.com/Blog/bid/17980/Social-Norms-and-Market-Norms-in-Online-Communities">a social space rather than a commercial space</a>.</li> <li><b>Envy: Starting a community because "everyone is" rather than defining clear goals for your own organization.</b> We encounter this among our prospects more than we would like: organizations that built an online community because it seemed like a good idea, but hadn't put in the effort to plan their objectives and how they were going to achieve them. The clearer you can be about what you expect to get out of your community, the easier it will be to measure the return on investment and to justify continued investment.</li> <li><b>Sloth: Failing to devote the time and staff required.</b> Forty-four percent of organizations <a href="http://www.beelinelabs.com/img/tobs/tob9.jpg">surveyed by Beeline Labs</a> reported a significant challenge was not finding enough time for their online community. In the initial launch, community management needs to devote time to starting and participating in discussions, and marketing needs to execute an extensive campaign to get people to log in and then to get them to come back. Over time a successful public community becomes self-sustaining, building a <a href="http://blog.vovici.com/Blog/bid/18052/Virtuous-Circle-of-Online-Community" class="" mce_href="http://blog.vovici.com/Blog/bid/18052/Virtuous-Circle-of-Online-Community">virtuous circle of online community engagement</a>, but the community management must prime the pump.</li> <li><b>Wrath: Punishing community members.</b> Negative behavior in an Internet forum can take many forms, from flamers to troublemakers to pet-project promoters. Prepare some guidelines for how you will <a href="http://blog.vovici.com/Blog/bid/17885/Dealing-with-Detractors-in-Online-Communities" class="" mce_href="http://blog.vovici.com/Blog/bid/17885/Dealing-with-Detractors-in-Online-Communities">deal with detractors in your online community</a> (see the linked post for advice on five types of detractors and what to do about them).</li> <li><b>Gluttony: Over-surveying and over-researching members.</b> Provide a <a href="http://blog.vovici.com/Blog/bid/18057/A-Bill-of-Rights-for-Online-Community-Members" class="" mce_href="http://blog.vovici.com/Blog/bid/18057/A-Bill-of-Rights-for-Online-Community-Members">Bill of Rights for community members</a> discussing the restrictions you'll honor. For instance, limit surveys to twice a month and <a href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length" class="" mce_href="http://blog.vovici.com/Blog/bid/17976/The-Long-and-the-Short-of-Questionnaire-Length">limit the length of surveys</a>. One area where gluttony is acceptable: encourage as many members as possible to join!</li></ol>Guilty of any of these sins? Contact me and I'll sell you some indulgences! </article> <article> <h1>Employee-Satisfaction Survey Gotchas</h1> <p>Jeffrey Henning — Mon, 02 Mar 2009 07:49:00 GMT</p> <p>Here are some common mistakes to avoid when creating employee-satisfaction surveys. Some workplace environments are filled with negativity and paranoia-especially in this economy. Accordingly, you have to be extra sensitive to employee fears. </p> <ul> <li><b>Asking detailed demographic information</b>. Respondents realize that you can triangulate answers to different questions to identify them. How many employees are male, over 40, with 15 years of service, in Houston? Ask only a few key demographic questions.</li> <li><b>Assigning unique URLs to every respondent</b>. Most employee-satisfaction surveys promise confidentiality, yet survey invitations go out with different hyperlinks for different email addresses. In a poor working environment, recipients will compare URLs. While an open-participation link enables ballot-box stuffing, that risk is worth it if employees don't trust the survey system's safeguards to confidentiality.</li> <li><b>Trying to cover all aspects of being an employee in one survey</b>. Be focused: are you exploring benefits or ways to improve how the customer is being served? Narrow and deep is often better than wide and shallow.</li> <li><b>Asking employees about things you cannot fix</b>. Do you really want to ask if they get enough vacation time? Or if they want 401K matching?</li> <li><b>Failing to get enough completed responses to have representative results</b>. Since early respondents in employee-satisfaction surveys tend to be on the extremes (very satisfied or very dissatisfied), broad participation is crucial if your data is to be representative. Make sure you get enough completes for your <a href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys" class="" mce_href="http://blog.vovici.com/Blog/bid/18119/Recommended-Sample-Size-for-Accurate-Surveys">recommended sample size</a>.</li> <li><b>Not sharing the results</b>. The more negative the results, the more important to share them. Of course, when sharing the results, make sure to communicate what your management team has identified as the <a href="http://blog.vovici.com/Blog/bid/18096/Priorities-for-Improving-Employee-Loyalty" class="" mce_href="http://blog.vovici.com/Blog/bid/18096/Priorities-for-Improving-Employee-Loyalty">top priorities for improving employee loyalty</a>.</li></ul>What common mistakes have you seen in employee-satisfaction research? </article> <article> <h1>Recommended Sample Size for Accurate Surveys</h1> <p>Jeffrey Henning — Fri, 27 Feb 2009 07:34:00 GMT</p> <p>I'm often asked, "How many responses do I need to get to my survey for it to be accurate?"</p> <p>First, if you are posting a link to your survey on blogs and Twitter feeds, it will not be representative of any target population, and no number of responses is going to make it so. The information will be interesting from a <a href="http://blog.vovici.com/Blog/bid/18015/When-Survey-Results-are-Qualitative" class="" mce_href="http://blog.vovici.com/Blog/bid/18015/When-Survey-Results-are-Qualitative">qualitative</a> standpoint, but-since it is <a href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win" class="" mce_href="http://blog.vovici.com/Blog/bid/17974/Random-Samples-Win">not a random sample</a>-it is not <a href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR" class="" mce_href="http://blog.vovici.com/Blog/bid/17990/Quantitative-and-Qualitative-Research-The-Yin-and-Yang-of-MR">quantitative</a>. In this case, whether you get one hundred, one thousand or one million responses doesn't matter. The information is interesting to talk about and might be fine for illustrative purposes for a blog post or a webinar, but the findings will not be useful for decision making. (For instance, see the convenience sample I used <a href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes" class="" mce_href="http://blog.vovici.com/Blog/bid/18067/Survey-of-30-Unsubscription-Processes">to survey list unsubscription methods</a>. Good for a blog post, bad for a vendor estimating the market opportunity of a new unsubscription technology.)</p> <p>If you are doing a <a href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained" class="" mce_href="http://blog.vovici.com/Blog/bid/18107/Random-Sampling-Explained">random sample</a> or attempting a census, then the statistical accuracy can be measured. The two key measures are confidence level and margin of error, and they work together. If you have a 95% confidence level and a 5% margin of error and a question choice selected by 50% of the sample, then 19 out of 20 times the true answer-the answer you would find if every single person in the target population answered this question-can be assumed to be in the range of 45% to 55%. </p> <p>The accompanying table summarizes the number of responses you need, depending on the number of people in your target population (e.g., how many employees or customers you have).</p> <p>Because response rates to surveys are not 100%, we advise you to try to conduct a census for target populations of 1,000 or less. Send out a <a href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover" class="" mce_href="http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover">survey invitation</a> to everyone in the target group, followed by periodic reminders.<br><br>For populations greater than 1,000, invite a random selection of 1,000 possible participants to take the survey. This way the next survey that you conduct will not need to go out to all the same people, improving your response rates for future research.</p> <p>Now, you may often wish to crosstabulate survey results across key subsegments. If it is very important that you have a 95% confidence level and 5% margin of error for leading subsegments, then you will want to invite a much larger group to take your survey. The Raosoft <a href="http://raosoft.com/samplesize.html">Sample Size Calculator</a> can help you determine appropriate sizes. Weigh the importance of that increased accuracy versus the need for other surveys of your target group, and live with the larger margin of error for subgroups, if you can.</p> </article> </main></body></html>