The Survey is Dead, Long Live the Survey
Posted by Vovici Blog on Mon, Jan 31, 2011
This is an update to a post originally published on the MRA blog, The Researcher’s Perspective.
It has become fashionable to say that the survey is dead. At last week’s Net Gain 5.0 conference in Toronto, Ray Poynter, founder of The Future Place and as passionate a public speaker as we have in market research, launched into a spirited attack on surveys—their low quality, the myth of random samples, the decline in the number of people on survey panels (as reported by Annie Pettit). Check out an earlier blog post of Ray’s for a fuller treatment of his themes: No Surveys in 20 Years?
Many, including Ray, are talking about social media research as the future. As the web has evolved from a place to shop to a place to socialize, more and more conversations are taking place online. People are talking about products and brands: looking at just Twitter alone, one Penn State study found that 20% of tweets were asking about or discussing products. While today the young are disproportionately likely to use social media, certainly in the future such conversations will be representative of consumers in general.
I for one welcome our new social media overlords. But even in a world of oversharing, people aren’t talking about everything. I now frequently use social media research in the early stages of a study, and I find some brands and products are hardly being discussed at all, especially business brands or consumer brands in uninspiring product categories. And even for those brands that are being discussed, consumers are often talking about only the most prominent products or a few types of services. Very often specific questions that you may have are not being discussed at all.
Social media does not have to be “listen only”: you can certainly ask specific questions to social networks or to proprietary MROCs (market research online communities). Both can produce quite rich qualitative discussions that can provide preliminary answers to research questions. But who are these conversations representative of?
Esteban Kolsky last week makes the case that feedback can’t be “listen only”: until you do a survey you can’t find out how projectable these attitudes and behaviors are to your target population. Ray would argue that because of the decline of RDD (Random Digit Dialing) surveys are no longer projectable: but if we are going to assume that social media will become more representative in the future, then online activity in general will become even more broadly used. And once you are capturing customers’ email addresses for online transactions and service requests, you can initiate surveys of random samples of those lists, and have projectable results to your online population. In fact, from his blog Ray links to his daughter’s e-shop: as she grows her customer base, she will be able to survey random samples of her customers.
Does she want to subject her customers to 30-minute surveys, full of grids, incomplete choice lists and required questions? Of course not. Many organizations can conduct short, well-designed questionnaires of small random samples. Those that can’t can still improve their survey practices, or watch their own response rates decline.
Let’s hope that in the future only bad survey practices are dead. Long live the survey!