The Survey Taker Is King
Posted by Jeffrey Henning on Mon, Sep 22, 2008
One of the panels at the American Marketing Association's 2008 Market Research conference in Boston was Twists and Turns on 21st Century Internet Express. Panelists included David Santee, director of market research at H&R Block; David Shanker, COO of OTX Research; and Michael McCrary, a vice president at Greenfield Online, sitting in for Keith Price.
David Santee said:
Our theme is "The survey taker is king!" - on the client side, it's not taking too much of a leap to say that we don't think about this guy too much. We have to balance 15 projects, work with providers, try to be a consultant to senior management; that's a lot of stuff, so the respondent gets the shaft in terms of our mind share.
There are many initiatives about data quality, but we aren't addressing those, because we want to focus on the survey taker. It is our collective agreement that the more we can increase the quality of the survey experience and make the survey fun and compelling the more we can help with data quality.
The panelists conducted an instant poll: "What percent of your research is completed online?" Surprisingly, 32% of audience members said 80-100% and 25% said 60-79%. Only 17% said 0-19% of their research was conducted online.
Michael said:
To do a lot of online surveys, we need a lot of online survey takers. It is all of our responsibilities to make sure that we engage these respondents, so that they are engaged and cooperative and give us better responses. It is our job to engage as many people in the survey research as possible.
We do enough satisfaction surveys after surveys to know what works and doesn't work. As service providers, no matter how many times we talk about guidelines, those are not always followed. Again, doing a survey for more than 20-25 minutes is a bad idea. That 45-minute survey - we know there are times when we need to do that - will get a lot of drop-offs and will take more time to fill.
We need to come to a consensus. If we don't engage survey takers, we are going to lose that growth curve.
We [at Greenfield Online] are not competing with other sample providers but with people's other uses of the Internet. When we started, email was unique and new, and our surveys were cool and hip, as was anything on the Internet. How times have changed! Now it's "text me" or "FaceBook me, don't email me".
I try to draw correlations to taking an exam, like taking a web survey. It gets hard to focus after 40-45 minutes, asking them to do things for a long time. The Internet is an evolving communications platform For the most part, we've just taken forms and put them online; we are not using the innovative creative techniques out there. Respondents are frustrated with answering mind-numbing surveys, with answering grid questions.
The audience was asked to respond to two polling questions:
[Audience members could make only one choice for each of these selections. Approximate sample size was 200. Results are only representative of AMA MR conference attendees.]
David Shanker said:
Surveys should be shorter; no amount of making it more fun is going to make hour-long surveys enjoyable.
Lots of straightlining or inconsistent answers shows where frustration sets in. Some questions are confusing. This ups the importance of writing a questionnaire that is friendly.
David concluded by asking, "Do you take your own surveys?" To the poll, 77% (134) of responding audience members said that they did take their own surveys, 23% (40) said that they didn't. The panelists were surprised at how many people were taking their own surveys.
Panelists recommended the following best practices:
- Respect the survey taker
- Take your own surveys
- Clearly define the value proposition to the survey taker: "You have to compare surveys to other experiences they have online; for instance, young men play games."
- Do not ask the same question over and over: "Minimize the repetitiveness. I know, the first time is to establish interest, the second time for pricing, the third time for purchase likelihood. They get to that third battery, and they do not want to do it anymore."
Michael again:
The demand [for online panel] has increased dramatically, but the supply has not increased; the price has gone down, but the demands for quality have gone up; all those things have happened at once, which is illogical.
What is the value proposition for the survey taker? As you build communities, as you build panels, it is not a financial windfall to join these, so it has to be about the experience. Define the fact that when they take the surveys it is for a good reason: it affects the products they buy, the commercials that they see, those things resonate with people in ways the financial incentive doesn't.
For another attendee's take on the presentation, check out Josh Mendelsohn's recent post about it.
Update: For my own reaction to this presentation, see Treating Your Survey Takers like Royalty.