Welcome to the Listening Post!

Worried about Panel Quality? Build Your Own Panel!

Tweet

 

One aspect of the continuing panel quality battle – paying panelists enough to offer their opinion, but not so much that they are encouraged to cheat – has produced tools designed to rat out panelists who are in it just for the money. With quite different approaches, Peanut Labs has its OptimusID machine fingerprinting technology, and MarketTools has its TrueSample certification program.

The original MarketTools TrueSample announcement back in 2008 quoted Doug Doyle, director of market research at Microsoft, as saying, "Microsoft has found that trust in external sample quality [emphasis added] is the number-one challenge facing online market research."

This is the huge difference between surveying external lists and internal lists of customers, partners, employees and other key constituencies. MarketTools states that “TrueSample is a three-part process aimed at ensuring authenticity in survey respondents. It provides objective assurance that survey respondents are Real, Unique and Engaged™ [a MarketTools trademark].”

Building panels of your constituencies is a great way to ensure that your own respondents meet these criteria.

• Panelist are real: Because panelists are recruited from existing company records, you can be assured that these are real people with real relationships with your organization.
• Panelists are unique: Your organization already expends resources to validate that each customer record in your CRM system is unique, and most likely has similar data-scrubbing procedures already in place for other databases of important constituencies. You can synchronize your panel management system with these databases to leverage your existing investment in ensuring unique responses.
• Panelists are engaged: The fundamental challenge of external sample research is weeding out respondents who are merely answering questions randomly in order to qualify for prizes. Incentives offered to customers and employees are typically tokens of appreciation that are not going to encourage respondents to distort their answers to qualify; these respondents value their relationships with your organization and want to see those relationships improve.  The desire to strengthen that relationship provides the ultimate in engagement!

Such concerns about external list quality provide compelling reasons to develop house panels for many research initiatives.

See also:

• Data Quality Validation Lessons from Panel Professionals
• Commercial Online Panels from a Total Survey Error Perspective
• Sample Quality of Online Panels: Putting Lipstick on the Piggy Bank