Twitter Research: Supplementing Surveys with Twitter Analysis
Posted by Jeffrey Henning on Thu, Sep 10, 2009
Last night here in the U.S. we had the latest passing of the political football on healthcare, as the President exhorted Congress to pass health-care reform. For CNN,
Opinion Research Corporation surveyed by telephone 427 adult Americans who watched the Presidential speech.
Surveying Presidential speeches is a pretty narrow domain, but given its importance, one that has been studied and shown to be full of issues: the most important issue to keep in mind is that such speeches are watched mainly by fans of the President, not by a representative group of Americans. And, in that tradition, this most recent CNN poll is representative only of that group that watched the speech last night, a group that skewed to the left politically: 45% identified themselves as Democrats, 37% as Independents and 18% as Republicans.
Some highlights from the poll:
- 77% of respondents had a somewhat or very positive overall reaction to the speech.
- 70% thought the President’s policies will move the country in the right direction, up from 60% of the same panel surveyed in the four days before the speech
- 67% favor Obama’s health care plan, up from 53%
- 75% think it somewhat or very likely that Congress will pass most of his proposals
Taken by itself those highlights don’t tell us much. Like most quantitative research, the narrative, the story behind the story, is missing.
Tonight I’ll go from watching political football to watching the start of the NFL season. NFL games are broadcast with a play-by-play announcer, who describes the details of the game as it unfolds, and a color commentator or color analyst, who provides background information and anecdotes. Let’s flesh out the play-by-play of the poll with some color commentary from Twitter this morning:
- 77% of respondents had a somewhat or very positive overall reaction to the speech.
- “President Obama Rocked My World Last Night Greatest Speech Ever by any President ever in History of USA !”
- “hi good evening! Have u watched obama's speech on health care? It was a good one, as expected. Obama is one eloquent man.”
- “I thought President's Obama's speech on healthcare was much needed.”
- “Had to laugh at Obamas speech last night! Bi Partiscian negotiations on the healthcare bill? NOT!!!!!!”
- “Watching Obama give a speech is like watching someone watch a tennis match as his head just bobs back and from teleprompter to teleprompter.”
- “Dennis Miller had it right - if POTUS cannot start a speech on time, do you trust the govt to deliver healthcare on time & under budget?”
- 70% thought the President’s policies will move the country in the right direction, up from 60% of the same panel surveyed in the four days before the speech
- “Busy @work today. Looking forward to catching up on the Obama health speech later, the fate of 46million uninsured Americans is in the balance”
- “scared for our Country. That speech didn't convince me that his plan is best for America...I still don't believe he is best for America”
- “So..big Obama speech. Is there any more agreement? No! I'm really disappointed that America has lost it's MOJO”
- 67% favor Obama’s health care plan, up from 53%
- “finally listening to yesterday's obama health care speech...health care is ONE SIXTH of our economy??!! lord. we've got to get it together..”
- “Good morning! I think his speech accomplished nothing, he didnt speak of specific details, and mandatory healthcare is dumb”
- 75% think it somewhat or very likely that Congress will pass most of his proposals
- “Its not enuf to be excited by Obama's speech - we must act! If you believe, call or email your Congress reps TODAY & tell them enuf is enuf!”
- “Overall I think Obama's speech is not the issue, it's what happens after the speech. Will Congress give us same garbage or real reform?”
A survey supplemented with Twitter commentary provides an interesting mix of quantitative and qualitative information. The Twitter audience is not representative of last night’s speech watchers to any degree; reading the tweets, in fact, you see that some watched last night, some this morning, some plan to watch, some read the transcript and some are simply reacting to news reports with no plans to watch. So it’s a different cross-section of the populace, and one not limited to America.
I downloaded about 400 tweets this morning between 9:17 am and 10:35 am to write this post [BTW, I'm
@jhenning on Twitter]. In the interests of time, I downloaded any tweet that had the word “speech” in it; that seemed more neutral than “President” or “Prez” or “Obama” or “POTUS”. It did mean that some tweets unrelated to the speech crept in.
A quick Wordle word cloud of the results make a few additional points:
Besides the point-of-fact words “Obama” and “last night”, you see many references to Joe Wilson, the U.S. Representative who must have dreamt he was in the House of Commons last night shouting at the Prime Minister. Words related to that include “Rep” and “outburst” and “LIE”. Clearly Joe Wilson is the unintentional story of the day.
But then, if you live in the U.S., you didn’t need Twitter to tell you that.
In the comments section, please let me know how you are using Twitter to supplement your market research.