House Panels in the Ideal World and in the Real World
Posted by Jeffrey Henning on Tue, Dec 14, 2010
Are house panels representative of an organization’s customer base, or are they qualitative and illustrative only?
It depends on how the panel was built, though as a general rule of thumb most panels are illustrative only.
In the ideal world, for instance:
- Every single customer is in the panel and is marked as a customer
- No one but customers is in the panel
Such a panel provides excellent quantitative research. Conducting a survey of a random sample of the panel provides representative research that can safely be extrapolated to the entire customer base.
In the real world, of course:
- Customers have unsubscribed from the panel
- Customers have never been invited to the panel
- Customers have been invited in clumps (by product area, by channel partner, by region) creating different proportions of different types of customers
- Lost customers are still in the panel
- Customers decide whether or not to take a survey in a nonrandom manner, depending on their general busyness and past experience with the brand and the panel
This panel can be a good source of qualitative insights, but take care when extrapolating to your customer base.
Make sure you understand how your house panel is built and analyze its composition to see if it reflects your customer base. Don’t assume that it is going to provide results that you can extrapolate to all your customers.
Of course, many house panels are somewhere in the middle. But being a little unrepresentative is a bit like being a little pregnant.