Welcome to the Listening Post!

A Tale of Two Panels: Customer & Noncustomer Panels

Tweet

 

Illustrating the difference between house panels in the ideal world and the real world, here’s a common request we receive: “Please build us a panel of our customers and a panel of our competitors’ customers.” The two panels are built quite differently, and behave quite differently.

Customer Panel

When it comes to building a customer panel, we advocate empanelment rather than recruitment. Recruiting panelists undermines the objectives of developing a quantitative understanding of customer attitudes and behaviors; the smaller panel size also limits the ability to survey customer segments.

The recruitment process itself introduces biases, with customers more willing to join the panel the more satisfied they are. Recruitment also significantly limits the size of the potential panel, as only a small percentage of those invited will join. Over time, individuals within a smaller panel will be surveyed more frequently, creating a panel conditioning effect, where their understanding of the brands being researched has been changed by exposure to a series of surveys.

Instead, empanel all customers for whom CRM records exists. This creates the largest possible panel of customers. The larger the panel, the less frequently members of the panel need to be invited to take surveys.

Every customer empaneled will be invited to participate in a profile survey, which will provide a baseline for their attitudes towards the brand and will capture data not present in the CRM system. Key questions will be used to qualify participants for subsequent surveys. Other profile data will be used to inform analysis of those surveys.

Obviously, this approach works best where companies have a rich CRM database: while this is typically B2B organizations, it includes many B2C firms as well, especially those in industries like ecommerce, health care, construction, automotive, services and periodical publishing. Industries without customer lists, such as businesses that sell their products at retail or wholesale, need to follow the next approach.

Competing Brand Panel

We build a house panel of customers of competitors through recruitment offers to rented email lists that reflect the target market. To field at least one survey a month for each competitive brand, we prefer to recruit 2,000 panelists per brand. Of course, the breakdown of product models within a brand may not even roughly correspond to installed base market share, depending on the email sources.

With a traditional panel like a competing brand panel, we will run new recruitment drives on a semiannual basis, to compensate for panelist attrition.

As with a customer panel, panelists in the noncustomer panel will complete a profile survey, which will provide key information about their brand loyalty preferences, product usage and demographics, for use by subsequent studies. The profile survey will be designed to allow for direct comparison of competing-brand panelists to customers.

While a customer panel of a known customer universe will provide for robust, representative findings about customers, such representativeness is too cost prohibitive to achieve for competing brands. For true representativeness, you would need to recruit the panel using Address Based Sampling with a series of telephone and direct mail invitations.

At the end of the day, a noncustomer panel is a convenience sample, as its panelists have themselves been recruited from convenience samples. This is cost effective and will allow qualitative assessments to be made about the attitudes and behavior of customers of competitors.

Conclusion

Both approaches described above are panels, and both let you field regular ad hoc surveys of the target markets. With the customer panel, you will be able to say 20% of the customers surveyed were interested in a product concept, therefore 20% (give or take) of all your customers will be interested; with the brand panel, you will not be able to extrapolate: if 20% of users of competing products were interested in the product concept, that most likely does not reflect the general proportion of your competitors’ installed base that would be interested.

Panels can provide a great source of information, but how far you can take that information depends on how the panel was recruited and managed.