Sample Sourcing Lessons from Panel Professionals
Posted by Jeffrey Henning on Fri, Apr 09, 2010
Twenty eight of the 30 panel providers we researched in our Lessons from Professional Panel Providers study use non-probability samples, recruiting panelists in an ad-hoc fashion rather than ensuring with a similar probability that any household or individual is invited to join the panel. Such convenience samples are much more affordable than building probability-based panels.
The affordability comes at a cost in terms of what types of research can be done. The AAPOR Online Panel Task Force concludes that non-probability panels should not be used for estimating population values. AAPOR (the American Association for Public Opinion Research) says such panels can be used to contrast the receptivity of different target markets to new product concepts and to determine how attitudes, behavior and intentions interact with the personal attributes of respondents. Such panels are satisfactory for many businesses to use when studying Internet behavior and potential opportunities for Internet services.
Professional panels have either closed or open membership. Many commercial panels are open for anyone to join, typically from the panel provider's web site, while others have closed membership and recruit participants using mailing lists and web partners.
Only closed panels use probability sampling, where the panel provider has randomly targeted households to join the panel using Random Digit Dialing or Address Based Sampling. Because external selection is a key part of probability sampling, such panels typically have extensive recruitment campaigns, starting with direct mail campaigns and culminating in a series of telephone calls. Because of the expense and level of expertise required, few professional panels build panels in this way; Knowledge Networks provides a probability sample of all U.S. households, and Norstat provides a probability sample of all households that have access to the Internet (i.e., "the online population") in the Nordic countries, the Baltic states and Poland.
What Panel Pros Can Teach Us about Sample Sourcing
- Align your panel design with the types of decisions you will make and the reliability you need. If you need to estimate the percentage of your customers with certain attitudes or propensities to purchase, then build a probability panel by randomly selecting customers to participate (this is much easier for organizations with a known and documented customer base, such as ecommerce businesses and B2B organizations with large accounts). If you are more interested in qualitative insights (directional but not projectable), then take a more ad-hoc approach to recruiting panelists.
- For non-probability panels, be creative about promoting the panel. Use corporate websites for recruitment, running banner ads across your organization's sites marketing the panel. Include promotional materials for the panel with your online and offline communications to customers.
- For B2C firms or other organizations with offline relationships, consider building a panel of loyal customers instead. It is cost prohibitive for traditional retailers, for instance, to build online panels of general customers. Instead of creating a panel to represent the average customer, work with your brand's loyalty program to create a panel of your best customers.
Want to learn more? For the full results of this study, download our white paper, Panel Management Secrets: Lessons from the Professionals, or view our recorded research webinar of the same name.