An Unexhaustive Look at Panelist Exhaustion

Email Article |  digg it |  reddit |  delicious |  StumbleUpon |  Facebook |  Twitter |  LinkedIn 

QueryMR asks, "How do you prevent panelist exhaustion?  Many firms are using some form of this online community, and people are getting tired of responding to ongoing questions/inquiries."

Here are some ways to limit panelist exhaustion:

• Do polls of the entire community infrequently.  The ABC Inner Circle only polls members twice a month, precisely to limit exhaustion.
• Develop a large enough community that you can target surveys to a random sample, excluding members who have been invited to a survey in the past 30 days.
• Avoid stupid questionnaires.  Stupidity comes in a number of ways.

• First, don't ask respondents questions your systems should know the answer to.  The example I gave in the webcast was of an airline that sent a survey the day after a flight asking the frequent flier to specify in detail the flight number, origin of the flight, destination of the flight, whether or not it was nonstop, the type of aircraft, etc.  The market-research firm that designed the questionnaire was right that such information would be useful to analyze.  The firm was wrong to assume the respondent was the best source of it; instead, the firm should have done a data merge with the airline's records.  Asking mundane questions, especially those that the firm should know, dramatically reduces response rates.
• Second, don't ask questions that are tedious or uninteresting to answer.  This is tougher to screen for, since someone in your organization thought they were interesting enough questions to include in the questionnaire in the first place.  I recently made a consumer-electronics purchase and the follow-up survey contained a double-sided matrix asking me to evaluate the manufacturer and its competitor on many esoteric attributes which frankly held no interest for me.  The icing on the cake was that every item was required.  I did not finish the survey.
• Third, don't ask too many questions.  Especially in a community, make a promise to panelists to limit questionnaire length.  Prune questionnaires ruthlessly, removing any questions that won't help make the decision at hand.  In cases where you really have a lot of legitimate questions, break the questionnaire into two separate surveys and randomly divide the sample into two. That said, panelist exhaustion is subjective.  I've seen community surveys that were long, complex and tedious, but quadrupled the response rate I expected because the subject matter was compelling to the community.  And on the other hand I've seen surveys that were short and to the point but that had a high abandonment rate on the first page, because the subject matter was not compelling. You need to watch how your community members react to survey invitations and measure response rates over time to start to determine an appropriate frequency and content strategy for your particular community.

That's all I can tell you for now.  Perhaps some day I will write the exhaustive treatment of the prevention of panelist exhaustion.