Survey Software, Web Survey, Online Surveys, and Enterprise Feedback Management solutions from Vovici

Your email:
   

Welcome to the Listening Post!

Your single source for everything Voice of the Customer (VoC) and Customer Experience (CxP). And, don’t forget you can follow us on twitter @vovici, or come check us out on Facebook and join the Vovici Network on LinkedIn.

 

Current Articles | RSS Feed RSS Feed

58% of Respondents Don't Like Surveys

 

Survey NationI admit that the headline for this blog post sounds like it came out of the satirical newspaper The Onion: “58% of respondents don’t like surveys.” But it’s a true statement. We asked 100 respondents to our Survey Nation study, “What, if anything, do you like about taking surveys?”

34% said “nothing” and another 24% said that they “don’t like” surveys. And this was to a question about what people liked!

The survey was designed to be representative of the population of American adults, and I think it was. We succeeded at reaching a population that rarely participates in surveys:  29% said they were not invited to participate in any surveys in a given month, 46% said that they did not start any surveys and 50% said that they did not complete any surveys. As one respondent commented, “I typically don't take surveys. This is the first one I’ve taken in a long time.”

Given the popularity of surveys among business people and the ease with which surveys can be created, it’s helpful to step back and realize that surveys are not ideal from the respondent perspective.

Survey Likes

So what do respondents actually like about taking surveys?

  • 13% want to voice their opinion. Sample comments: “I like to express my opinion”; “I can have my own opinion”; “Voicing my opinion about products and services.”
  • 11% want an incentive. “When you get things for free, or coupons. I like that.” “If I get something out of it.  Like a reward or coupons or money.” “I have no feeling one way or the other. Unless there is an incentive, then I really enjoy it.” Included within this category were three respondents who liked receiving free products to evaluate: “Surveys where they send me stuff to use.”
  • 10% like to be helpful. “To see if I can help with research.” “I just do it to help people out.” “I’m just doing it to be nice.” “I guess it's research. I just do it to be polite. I like to give them more data.”
  • 9% like interesting topics. “I'm interested in what they're surveying about.” “It depends if it is something I'm interested in.” Three of these respondents were interested in learning: “Valuable information on good topics”; “I like to read them.”
  • 7% want their answers to make a difference. “I'm hoping it will change or make things better.” “Hopefully something good is happening because of taking them.” “The only thing I would like is to find out something that really matters. Like if it actually helped a human being and not just a big company or bad politician.”

What are the practical ramifications of these attitudes toward surveys?

  • Survey people more strategically, especially customers. Use panel management to survey small subsets of customers rather than surveying everyone; strive to minimize the number of people you invite to surveys: for a probability sample, a sample size of 400 with a response rate of 20% will provide a high degree of statistical validity.
  • Emphasize what respondents like in your invitations. Compelling survey invitations should discuss the opportunity for respondents to give voice to their opinions. Emphasize how their response helps and let them know how their participation will make a difference.
  • Reward each respondent. Only one respondent said they liked points, and only one respondent said they liked a chance to win money. But most appreciated some form of compensation, even if it was only a coupon. If you are not going to offer any other incentive, give them an executive summary of the results of the survey. Find the right incentive strategy for your respondents.

Only 4% of respondents said they “enjoy” taking surveys. We have a lot of work to do.

Comments

Hi Jeffrey, 
You said that "a sample size of 400 with a response rate of 20% will provide a high degree of statistical validity." The associated margin of error is +/-9.81%, i.e. it's within +/-10%. My own clients are becoming increasingly interested in margin of error. And the guidance I've been offer is that getting inside that +/-10% is very respectable. The conundrum arises when the population is too small to achieve that - even with a decent response rate....
Posted @ Friday, January 07, 2011 9:56 AM by Brendan Cullen
I'm surprised you did not also ask what people do NOT like about surveys. I expect they might say they are too long and take too much time with many repititious questions.  
 
Do you have any insight on this side of the issue?
Posted @ Friday, January 07, 2011 10:18 AM by Robert Burian
Robert, we did ask 100 respondents about their complaints - see Survey Nation: 7 Billion Invites a Year for a summary. Thanks for commenting!
Posted @ Friday, January 07, 2011 10:23 AM by Jeffrey Henning
I think a side point that Robert notes is that far too often we marketers try to do things by the statistical book. We're taught that in order to fully validate our results, we need to ask a battery of questions with subtle repetitions of questions. This way, we can best determine whether a positive statement about something is due to the mere phrasing of a question or truly consistent across within the respondent's opinion.  
 
However, this need for statistical validity must be balanced by a desire to actually get much needed data. We have to factor in what we really want to achieve from our data collection activity and develop the most appropriate method. Unless I absolutely need statistical validity, I prefer to opt for short, progressive profiling of my market segments. I get more useful data and insights.
Posted @ Saturday, January 08, 2011 12:33 PM by Chris Bailey
Hi Brendan, if you have a sample of 400 drawn from a reasonably large population the error term at 95% is +/- just under 5%. However, in Jeffrey's example I think we have to consider two populations 
 
a) the population who would have answered his survey is asked 
 
b) the population who would not have answered his survey if asked. 
 
If the invitation process was fair and representative then and we work on the 20% figure you discuss, we might reasonably assume that if the whole, eligible, US population had been asked there would have been about 20% answering the survey and 80% not. 
 
The error term for the population who might have answered the survey is +/-5% but what about the other 80%? 
 
In my 'rule of thumb' experience, the 20% are often a good proxy for the 80% if there are no strong biases. However, if we take the example fof the study about whether people like surveys, we might very well postulate that the non-responders could be even less well disposed towards surveys that the people who responded. 
 
When we use error terms we often use a form of words that implies validity in terms of the broader population. My recommendation is that we change that wording to reflect reliability, not validity. 
 
If we Jeffrey ran his test again, on the same population, with a new sample, there is a 95% chance that he would have found that between 53% and 63% of people don't like surveys, but the figure for the population could easily be more like 75% if perhaps 80% of the non-responders did not like surveys (and if the response rate was 20%). 
 
 
 
I think you flag an interesting issue Brendan, more and more clients seem to be asking about error terms, but I fear that they won't like what they hear.
Posted @ Saturday, January 08, 2011 5:34 PM by Ray Poynter
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics

Latest Posts

Loading
What's New
Don't Be in the 4%
VoC on Twitter
Verint Blog
Verint Blog: Read the Latest from the Verint Systems Blog