Sentiment Analysis Symposium 2010
Posted by Jeffrey Henning on Sat, Apr 17, 2010
I missed the Sentiment Analysis Symposium 2010 this week in New York City but fortunately it was well blogged and tweeted (under the hashtag #SAS10).
Nancy Lazarus provides a good overview of the opening Visionaries Panel session:
The value of sentiment analysis is enabling the tracking of a wealth of online conversations. However given its limitations, they [panelists] recommended its use in conjunction with other research techniques.
Bradley Honan of StrategyOne summarized the evolution. "We've gone from traditional market research to media monitoring to mining data that helps formulate business and communications strategy."
Karla Wachter of Waggener Edstrom added, "A while ago the ability to track sentiment was limited and we had to guesstimate, but today mountains of data are available. Sentiment analysis is a key part of the toolkit, and it's a hybrid approach involving both automation and humans."
Brad McCormick provided a case example of how Porter Novelli used sentiment analysis for one of their clients, who had invested heavily as one of Tiger Woods' sponsors... McCormick said, "Many tools showed negative sentiment spiking, but our client was reluctant to let it go at that. Around the same time, sympathy for Tiger's situation started to rise, and this led to the assumption that there could be a comeback. This shift allowed the company to maintain the relationship with Tiger."
Accuracy of Sentiment Analysis
Looking specifically at accuracy, Nancy wrote:
Greg Radner of Thomson Reuters said, "Beyond 80 percent, the law of diminishing returns sets in as it becomes more costly."...
McCormick echoed that opinion, and said, "One of our tools measures to 90 percent, but we need huge amounts of data to get to that threshold, or 150 conversations per day."
Jennifer Zaino provides a nice crosscut of discussions at the symposium about accuracy, including these comments:
Conference organizer Seth Grimes explained that two humans will only agree on expressed sentiment in 82 percent of cases. Expect machines to do better? No, and yes. Estimates of sentiment analysis accuracy can range from 70 to above 90 percent....
Bradley Honan, senior vice president of StrategyOne, isn't bothered by a 70 percent accurate take on sentiment - "if we figure 30 to 40 percent of people are actively engaged putting content online using social media, then we're talking about 70 percent of that 30 or 40 percent," and 70 percent of a small pie of what are probably the most partisan (positive or negative) individuals anyway is pretty good. As a method of capturing the sentiment of those to the right and left on the intensity curve, whose conversations can reverberate and influence, it's a good arrow in a quiver that also has to include surveys, polling and so on to really understand the customer base, he said.
Marshall Sponder of WebMetricsGuru takes a more contrarian view on sentiment accuracy, "If you were scanning a document into Optical Character Recognition and you only got 75% of the characters right (the level of what we can expect with Sentiment Analysis Systems today) would you be able to figure out what the document says or means? Sometimes you could - but would it be worth the effort?"
Other Sessions
Shlomo Argamon, a professor with the Illinois Institute of Technology, presented Sentimental Market Segmentation, which discussed identifying authors of comments by the language they use. Automatic authorship profiling seeks to estimate gender, age, native language, personality type and education level. Once this is done, it is possible to segment sentiment by the different demographic groups.
Saplo demonstrated its application which highlights names that were being talked about more or less often, with the sentiment of the comments using those names.
Sally Church of Icarus Consultants discussed how to use Leximancer for pharmaceutical work for market research and PR monitoring for therapies, diseases, companies and physicians. Stephanie Nobel of Paden Nobel presentenced Pharma Sentiment Surprise! and provided a case study about listening to those who suffer from epilepsy.
And, of course, no summary of a sentiment-analysis symposium would be complete without linking to a sentiment analysis of tweets about it!
