When Instant Feedback is Like Instant Coffee

Email Article |  Facebook |  Twitter |  LinkedIn 

Instant coffee is great when you're in a hurry.  Of course, it has less caffeine and it tastes more bitter than regular coffee, but it will get you going in the morning. 

Likewise, instant feedback is great when you're in a hurry.  Of course, it has less accuracy and its findings are more tentative than surveys done over a few days, but it will get you some talking points in a hurry.

Web surveys have long been sold as offering the benefit of "instant feedback", but it's important to remember the caveat.  And that caveat is stated fresh for us today in two polls:

1. Few in AP poll back Bush market rescue plan.  Caveat:  "Polls conducted on one night can be less reliable than surveys conducted over several nights because they only include the views of people available that particular evening."
2. Americans Favor Congressional Action on Crisis: Four out of 10 say this is biggest financial crisis of their lifetimes. Caveat: "In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls. Polls conducted entirely in one day, such as this one, are subject to additional error or bias not found in polls conducted over several days."

For the purpose of a quick pulse of the American public about their reaction to recent news events, such polls are great.  But in the majority of cases, for businesses and non-profits, one night is too short a time to gather feedback.  I'll close with my retelling of a story from this year's MRA conference:

Besides the Internet placing price pressure on all forms of survey research, it has also created an expectation that research can and should be conducted rapidly.  Robin Pearl of Estée Lauder related a humorous anecdote in her keynote.  A salesperson for a panel company excitedly told her how research would be turned around in hours in the middle of the day.  To which Robin replied, "Well, I guess I won't be using your company then."  The salesperson didn't understand that conducting an entire survey in a few hours would not be representative of the U.S. population, who are not yet chained to their email 24 hours a day but have usage that varies by hour of day and by time zone.

Sometimes good feedback, like good coffee, is worth the wait.