Welcome to the Listening Post!

Surveys in 20 Years

Tweet

 

It's very hard to predict the future. You often underestimate the changes coming in the short term and overestimate the changes happening in the long term. Few in 1995 expected the World Wide Web to grow so dramatically in the following five years, and most of us of a certain age expected regular Pan Am flights into space by, say, 2001.

So Ray Poynter takes on a difficult task, when - at Research 2010 - he predicts there will be no commercial market research surveys in 20 years.

Ray underestimates short-term changes.

• "Instead of surveys we are getting to the point where we can ask and process open-ended questions. I expect this to be in place in 20 years." We can do this today with verbatim responses, and it is cost-effective for large volume surveys. Certainly in the next five years it will be cost-effective for even surveys with few responses. (The current high cost is the human time spent automating the coding; preprogrammed coding templates will arise for many common questions.)
• "Given that most survey data comes from panel members, and that all their previous responses are stored in data warehouses, we only need to ask genuinely ‘new' questions - not whole surveys." Access panel providers could do this today, if they trusted their panelists; instead they ask the questions again to verify that the panelists are being consistent. Corporate researchers with proprietary panels of customers do embed previous answers in the data collected. The result is what are perceived to be short surveys, but have much more data underneath: iceberg surveys, with their depth hidden.

Ray overestimates long-term changes, as well. He says, "We have given up on random probability samples." I would argue a second Golden Age of probability sampling will bloom in the next decade (the first Golden Age, now over, was RDD sampling in the 1980s-1990s). As more and more commerce is driven to the Internet, it becomes possible to do random probability samples again--though only of customers, not of the general market. Ray's own daughter runs an online clothing shop; she has the email and physical addresses of all her customers - she can easily do a random probability sample once she has enough accounts. I order takeout pizza several times a month, typically online, and get surveyed about the pizza. As commerce moves online, more surveys can be done online, using robust sampling methodologies.

I think that surveys are the single superpower of the research world [see: The Survey Superpower in a Multi-Polar World]. As Ray writes in the paper he is presenting today, "Some 80% of market research is quantitative, with just 14% being qualitative. So, when we generalise about market research we are normally, implicitly talking about quantitative market research. Indeed, the problems with market research are almost exclusively related to quantitative research."

Surveys in 20 years will be one of many major powers, rather than the ascendant power. And that has created this huge sense of loss and a belief in the decline of surveys. The many qualitative NewMR techniques that Ray rightfully champions will each take share away from survey research. In 20 years, surveys will have changed. Television didn't kill radio, but it forced it to drop its comedy hours and long-running serials in favor of news and music programs that could easily be tuned into or tuned out of. Similarly, qualitative Internet techniques will not replace surveys, but they will transform surveys, in ways we are only starting to imagine.