Posted by Jeffrey Henning on Sat, May 31, 2008
Brad Bortner, a principal analyst with Forrester, has written an excellent white paper about the representativeness of online research. Here's an abstract of Does Declining Research Projectability Matter? (emphasis added):
"Conventional wisdom holds that online panels are inherently problematic in supplying projectable findings to the general population - online and offline. Critics assert that online panels aren't representative and that new types of dirt in the data - such as professional survey takers - erode the accuracy of projections. However, it's hard to find any substantive marketing decisions that have gone awry due to using online panel-based research. Why? Because good online panels actually deliver as much accuracy as they give up. Market research professionals should leverage what online panels are good at - speed, cost, and new analytic approaches - and at the same time hedge risk by mixing data collection modes, expanding their analysis framework to create 'what if' models, and understanding that sometimes speed of result trumps accuracy and cost."
Brad writes about third-party panels, which he sees as sufficient for research but suffering from three key problems:
- Wide variances in projectability from different panels
- Underrepresentation of key groups
- Bias introduced by "professional" survey takers
Because proprietary panels are derived from company databases, proprietary panels in contrast are highly representative of the target population of a company's clients and employees. See this blog post last month for a more detailed comparison of the two types of panels: Panelist Quality of External Lists vs. Feedback-Focused Communities.
If you have any concerns about the projectability of online research, I strongly encourage you to buy Brad's report.
Posted by Jeffrey Henning on Thu, May 29, 2008
Recently a former colleague I had lost touch with found me through my Linked In profile, and he asked, "What have you been up to lately?" It was a short question, requiring a short answer, and I cast about for a succinct way to summarize feedback communities.
When he and I started in the market research industry in the 1980s, the primary methodologies we were involved with were mail surveys, telephone surveys, face-to-face surveys and focus groups. It occurred to me that feedback communities are, in a sense, standing focus groups, and so that is how I described them to him.
Unlike surveys, which are typically completed one respondent at a time, focus groups involve multiple participants, who not only answer the moderator's questions but comment on each other's responses as well. Feedback communities do that as well, and they eliminate some of the problems with focus groups:
- Where focus groups have small participant size (an individual group typically has no more than ten participants), feedback communities can have hundreds of participants.
- Where focus groups have limited time (90-120 minutes divided by 10 participants yields only 9-12 of minutes of potential response per participant), feedback communities can be a resource that lasts for years.
- Where a focus group can be dominated by one or two garrulous individuals (my hardest job as moderator was always politely but firmly ending one participant's comments so that others could be heard from), it's hard for a member of an online community to monopolize the discussion.
- Where a focus group is set in a major metropolitan area, a feedback community can have participants from around the world.
(For a good discussion of the strengths and weaknesses of online focus groups, see Debbie McNamara's recent post.)
When I organized studies involving a series of focus groups across the United States, we typically developed a two-page written questionnaire for participants to complete as they waited for the session to begin, giving us quantitative research to supplement the qualitative nature of the focus-group discussion. Similarly, feedback communities by design include web surveys, to provide those quantitative results.
Of course, unlike traditional focus groups, feedback communities don't provide you an opportunity to show participants a product in real life and don't allow you to gauge their facial expressions, body language and vocal cues as they respond.
And neither does LinkedIn. So I can't be sure if my succinct answer about feedback communities as "standing focus groups" wasn't met with a quizzical raised eyebrow. Next time someone asks me that question, I guess I'll give the 21st-century answer: "check out my blog"!
Posted by Jeffrey Henning on Fri, May 23, 2008
This week I tagged along on a sales call to a VP of marketing who was interested in hearing about our new Community Builder Module. Her company is a Fortune 1000 firm, and she is responsible for marketing many different brands. What was fascinating about her firm was that all of the different brands she is responsible for represent a microcosm of the different stages of online community adoption.
No Online Community - Many of her oversea brands still have websites that are "brochureware", offering product and customer service information but little else. These brands want to build online communities, and they are starting from scratch. For them, the Community Builder Module is perfect, as it can be used to rapidly launch general-purpose online communities with integrated feedback.
Existing Online Community Needing a Feedback Annex - Some of her domestic brands already have basic online communities, but these communities don't provide any feedback mechanisms; further, the brands are worried that too much orientation on feedback would distract from the main purposes of these particular communities. For these, we can use the Community Builder Module to create separate community-feedback portals and recruit community members to join this feedback community to act as advisory councils to the community.
Existing Online Community Needing Integrated Feedback - An interesting mix of her mature brands and new brands have recently rolled out or are currently rolling out new online communities to their specific customer segments. Some of these efforts are quite far along and have advanced social media capabilities built in, but again little emphasis on feedback. For these, our Community Builder Module is not appropriate - the firm has already built rich web applications for these online communities. However, EFM Community itself has a robust web-services API that we can use to integrate with these new communities; in fact, we've been doing such integration work with existing online communities with the API since 2005 and in a more custom fashion for years before that. Community members can see personalized survey invitations within their existing online portal, and EFM survey administrators can invite participants using the demographic information from the profiles of community members.
This organization has quite the mix of online communities; the only thing it is lacking today is an online community with robust feedback capabilities! Once that is in place for one brand, the other brands will be able to see directly the value that a community feedback platform offers.
Posted by Jeffrey Henning on Mon, May 19, 2008
George F. Colony, CEO of Forrester Research, has an interesting take on the power of feedback and communities to refine products: "Loic Le Meur ... uses his social networks to hone his ideas -- the smart ones get perfected, the bad ones get shot down. The next day I asked Vinod Khosla, the venture capitalist, what he gets from blogging. Same answer -- it's a way to get instant feedback on his ideas. I call this phenomenon Social Sigma. You've heard of Six Sigma. That's the discipline that companies use to perfect products through process improvement. Social Sigma is using the continual feedback from your customers to perfect your products."
You can certainly use a schedule of frequent, traditional surveys to pursue Social Sigma, but the real potential comes when you create conversations between respondents and your organization. Our new Community Builder lets you do just that: set up forums and wikis where your customers (or employees or resellers or members of any key constituency) can engage with one another and members of your own organization on an ongoing basis.
I believe that Social Sigma represents the next wave of online feedback. In the 1990s, online feedback was just traditional surveys ported to the new medium: same old survey, new venue. By the start of this decade, the big innovation was the transformation of surveys from quarterly or annual events to continuous, ongoing studies. With Social Sigma, online feedback can now really take advantage of the Web, enabling participants around the world to communicate with one another directly and regularly. Imagine it as a focus group convening daily, constantly providing your organization with fresh new insights as participants react and expand on one another's comments. You can then use surveys for quantitative analysis to prioritize the most important emergent ideas.
While Social Sigma itself might not be the next catchphrase, I'm certain that within the next five years every Fortune 500 company will be including Social Sigma initiatives in their research and planning operations.
Posted by Jeffrey Henning on Tue, May 06, 2008
Yesterday we announced the release of our Community Builder module. This is a significant release for Vovici and it is a product that I believe will be provide significant value for our customers - existing customers and new ones looking to leverage communities for research and feedback.
So the obvious question is: "why is this different from your EFM Community offering?". Well, Community Builder is about the respondent. While EFM Community has features to build and publish surveys to a community, the Community Builder module is all about engaging the community member. Community Builder has an interface that is specific to each panel member where the panelist can take surveys, view results of completed surveys, and interact with the community and the organization via collaborative tools such as wikis, forums and blogs. The goal is to create an environment where the community panelist will be engaged through both survey activity and the collaborative tools.
The melding of multiple types of feedback tools provides a compelling balance between quantitative and qualitative research. The survey process takes care of the quantitative side and really finds out what the community wants or is thinking about. Using the collaborative tools to further engage the community provides the how and why of the qualitative side. For example, in our customer advisory community the surveys are used to determine what features customers want, and then our product managers engage the community to see how these features will look and operate.
EFM Community has been modified to allow the survey authors and administrators to interact with the community panels through the EFM administrative interface. Community Builder is a point and shoot, no coding required operation with an impressive list of objects that handle the integration with EFM Community as well as power the collaborative functions. You can brand the interface so it looks just like existing sites. Customer can build their own community or attach community builder to add feedback capabilities to existing community sites.
Go to the web site and watch the flash file (http://www.vovici.com/products/online-customer-community-software.aspx) for a visual overview of the product and the concepts behind it. It will help you get your hands around the offering. Or better yet, request a demo. I have personally shown this to 15 people since last week and within minutes people really get the benefits as quickly as any product I have launched in the past.