Each Respondent is a Riddle, Wrapped in a Mystery, Inside an Enigma
Posted by Jeffrey Henning on Fri, Jan 08, 2010
One of my blogging resolutions was to look back at some of the conferences I attended last year, and now--with the advantage of hindsight--tease out some additional themes or insights. As I've thought back to the
AMA Market Research Conference, one of my new takeaways is that survey researchers often overestimate the reliability of respondents.
As Carl Marci, CEO of Innerscope Research, reminded attendees, "The old view was that people ‘1) Think, 2) Do, 3) Feel' but the new view is that people ‘1) Feel, 2) Do, 3) Think'." What we frequently took for reasons were instead rationalizations.
Jack Wakshlag, chief research officer of Turner Broadcasting, pointed out that consumers are not rational decision makers, and that what consumers tell us often reflects their best guesses or what they feel is socially acceptable. Bad research generates and perpetuates the myth of consumer self-awareness. Wakshlag emphasized: "When good researchers fail to challenge bad research, all researchers - including the good researchers - lose credibility. When we report what people say - while behavioral data tells us what people actually do - we lose."
Andrew Baird, VP of Marketing for Convenience Retail for BP, gave several examples from his convenience-store research where research participants wanted to be socially acceptable: not claiming to let sweepstakes prizes influence purchase decisions when such prizes did influence them; discussing their desire to eat healthy while harboring every intention to purchase unhealthy comfort foods.
If you need to develop the most accurate pricing models, advised Eliot Roth, Senior Manager of Pet Custom Research for Del Monte, don't rely on customers' assessments of their price sensitivity, but look to historical records of actual purchase behavior at different price points.
David Weinberger, VP of Insights, Georgia-Pacific, discussing researcher blindspots, identified the lack of customer intimacy available to most researchers and argued in favor of research approaches that were deeply immersive. To develop accurate understanding of customer behavior, look to ethnography, store and lifestyle shadowing, deep dives and immersion excursions.
Respondents are complex. Humans are complex. To paraphrase Churchill on the Soviet Union, each respondent is a riddle, wrapped in a mystery, inside an enigma.
Given the limits of consumer self-awareness, what other subjects do you think are best handled through other market research methods besides survey research?