When Survey Results are Qualitative
Posted by Jeffrey Henning on Wed, Oct 29, 2008
Last week I talked about the difference between quantitative and qualitative research. I wanted to clarify one point of contrast. A dangerous shorthand people resort to when comparing qualitative research to quantitative research is to say that all research that produces numbers is quantitative. That is not true, as surveys can be qualitative, and typically are qualitative, when used in conjunction with online communities. For instance, in the online communities that Novartis runs, its surveys are only qualitative. Brad Bortner of Forrester points out that most surveys in MROCs (market research online communities) are qualitative.
The word quantitative in "quantitative research" means that you can quantify the wider population based on the results from the survey. The reasons most community surveys for open communities are not representative is because of the self-selection bias for joining the community (similar to the self-selection bias that makes many online polls unrepresentative). Even in closed communities, those who agree to participate in communities may represent positive or negative extremes, since agreeing to participate in a community is even more time consuming than agreeing to take a survey. Further, most community managers do not ensure that the community demographics match the overall audience demographics.
The following diagram illustrates how active community members are a small subset of the target audience.
It is hard to develop representative community surveys. You often must achieve representativeness by reaching as far beyond the active community members as possible. One way to do this is by using online communities in sync with enterprise feedback management, where you empanel as many members of the audience as possible, for instance, by using CRM connectors to have all the customers in the panel. (The empanelled audience is still a subset of the target audience, since audience members can opt out of receiving emails from your organization.)
EFM systems give you the best opportunity to make all your community surveys quantitative rather than qualitative.