Tragedy of the Respondent Commons
Posted by Vovici Blog on Mon, Mar 01, 2010
At the CASRO Panel Conference, Paul Johnson and Bob Fawson of Western Wats presented the paper "Factorial Design on Survey Router Configuration Effect by Sample Source." While the technical details were fascinating, I'm going to concentrate on the overarching metaphor the presenters used of the tragedy of the commons, as it has greater ramifications.
Here, for once, Wikipedia has a well written and concise summary:
The tragedy of the commons refers to a dilemma described in an influential article by that name written by Garrett Hardin and first published in the journal Science in 1968. The article describes a situation in which multiple individuals, acting independently, and solely and rationally consulting their own self-interest, will ultimately deplete a shared limited resource even when it is clear that it is not in anyone's long-term interest for this to happen.
Central to Hardin's article is an example of a hypothetical and simplified situation based on medieval land tenure in Europe, of herders sharing a common parcel of land, on which they are each entitled to let their cows graze. In Hardin's example, it is in each herder's interest to put the next (and succeeding) cows he acquires onto the land, even if the carrying capacity of the common is exceeded and it is temporarily or permanently damaged for all as a result. The herder receives all of the benefits from an additional cow, while the damage to the common is shared by the entire group. If all herders make this individually rational economic decision, the common will be depleted or even destroyed to the detriment of all.
The presenters said that three characteristics of a commons are that it has joint ownership, is congestible and exhaustible. Does it make sense to think of a pool of survey respondents as a commons?
- Joint ownership - So many individuals are on multiple panels (owned by different companies) than the industry has rapidly adopted digital fingerprinting to identify them when doing sample blending.
- Congestible - Demand for low-incidence populations is so great that panelists are inundated with survey requests (many of which they do not qualify for).
- Exhaustible - That the access-panel industry has rapidly adopted river samples is a strong indication that the supply of panelists is limited.
Clearly, the pool of respondents is a commons. And the industry is degrading that commons. As the authors write:
Individual researchers subject respondents to low qualification rates, poor survey design, low salience topics without respondent rewards, long surveys, and similar practices. As a result of joint access to a resource of respondents, the tragedy of the commons has already occurred in the phone arena. Telemarketers (including "frugging" and "sugging"), poor survey designs, and interviewing techniques have already caused cooperation rates to decline from 60% to 30%...The online industry could be next.
The presenters then transplanted Hardin's prototypical example from medieval Europe to the American West. Bison populations were a commons that were destroyed by settlers. Cattle grazing, however, was a commons that settlers nurtured and preserved. What lessons can the survey industry learn from cattle grazing?
Two key innovations were:
- Branding - Livestock branding enabled cattle to range over much larger grazing lands, expanding the size of the commons.
- Cattlemen's Associations - These organizations set and enforced the rules the industry abided by and policed the behavior of members.
River samples and survey routers are a key way to expand the borders of the commons, bringing new people into the survey-taking process in addition to panelists. Industry associations, especially ARF, have an important role to play to enforce common quality rules and ensure the long-term sustainability of the respondent commons.
My Conclusions for Corporate Researchers
The tragedy of the commons is a great way for organizations to think about their pool of respondents. Many large organizations rely largely on decentralized, Do-It-Yourself survey research. As a result, hundreds and even thousands of individuals are writing and fielding surveys to the commons: the house email list of customers. It is in the rational self-interest of these organization's employees to conduct as many surveys as they need, whenever they need to, regardless of the long-term degradation they are causing to using online surveys with this audience. But they are degrading the resource, by sending too many surveys, of too great a length and of too low a quality. Response rates do suffer from such practices.
As Hardin once observed, it is not really a tragedy of the commons per se, but a tragedy of the unregulated commons. Companies own their respondent commons, yet fail to manage them as collective resources. Large organizations that would never have considered the need for it must adopt panel management and build proprietary panels. Failure to do so will mean that eventually the organization will be sending out surveys that no customer answers.