Opinion Mining: Text Analytics of Web Content
Posted by Jeffrey Henning on Sun, Feb 28, 2010
At the 2010 CASRO Panel Conference, Raymond Cyr, co-founder & CEO of Voxco, and Alkis Papadopoullos, co-founder & COO of HotGrinds, discussed their platform for "social discovery" - mining web content for meaning. Some data that Raymond shared:
- 81% of Internet users have done online research on a product at least once
- 20% of Internet users do online product research on a typical day
- 73-87% of readers of online reviews of restaurants, hotels and others services report that such reviews significantly influence their purchase
- 32% have provided an online rating of a product, service or person
- 30% have posted an online comment or review regarding a product or service
- Consumers report being willing to pay from 20% to 99% more for a 5-star-rated item than a 4-star-rated item (with the amount varying by product and service category).
The Voxco/HotGrinds solution mines data from across the web, extracts it, parses it, linguistically analyzes it and provides dashboards for trends and tracking.
In opinion mining, HotGrinds has found interesting differences between positive and negative commenters:
Negative people are much more vocal than positive people. While document lengths vary from a sentence to a blog post, negative comments are longer than positive comments. But negative people tend to be very repetitive. They focus primarily on the issue that went wrong.
Positive feedback, on the other hand, is really interesting because positive people can be very detailed about it. Some don't share detail, but just say "I love it!" Not very useful. When they do share detail, though, they provide more reasons for their happiness.
One attendee asked, "What's the current thinking about the right our industry has to take data from the Web and aggregate it?" Alkis said, "This is a legal question and a marketing question - my lawyer tells me we are fine so long as we don't misrepresent the user. We don't aggregate content by user; I'm not identifying users. We do have a report that shows here are what the top 5 influencers are saying about you. In terms of perception, customers seem fine with it."
Another attendee asked if opinion mining could replace current techniques. Alkis replied:
Text analytics can't replace surveys or panels for two reasons:
- You are dealing with content on the web without understanding credibility, intent or source. We look at sentiment analysis to filter out blatant attempts at misrepresentation.
- People who are unhappy on the web are going to comment in much greater detail than we initially thought.
This complements work being done by panel provides and surveys to identify areas to focus on. Our clients use surveys but maybe they are having trouble focusing on one area or they want to see how what they hear on surveys is being echoed on web sites, especially on popular web sites.
After the session, Reg Baker and I discussed other limitations with "social discovery":
- B2B brands and small consumer brands aren't discussed in significant volume, so there is not much opinion to mine.
- Many web discussions don't address questions that companies need answers to.
Like Alkis and Raymond, we see opinion mining and survey research as highly complementary.