Welcome to the Listening Post!

Sampling Solutions

Tweet

 

Steve Winkel of Survey Sampling International presented "An Introduction to Sampling" at the AMA Applied Research Methods course in Philadelphia this week. He presented the continuum of sampling solutions available for U.S. research, ranging from solutions with greater coverage but less efficiency (e.g., higher cost) to those with less coverage but greater efficiency.

The Offline Continuum of Sampling Solutions

1. Random Digit Dial (Landline+Wireless) - Full RDD is the method with the greatest coverage that provides a practical probability sample of households; RDD involves randomly generating telephone numbers, which reaches unlisted numbers as well as listed numbers. Because it is illegal in the U.S. to use automatic dialers to reach cell phones, telephone interviewers must manually dial cell phones, adding significant labor when compared to automated dialers, which only select an interviewer when a phone has been answered. Subtypes of RDD trade-off coverage for price, ranging from methodologies with equal probability of selection to stratified RDD and proprietary RDD sampling techniques. A key complication in full RDD, often overlooked, is the presence of multiple phones per household (main line, rarely a second line used for business purposes, but frequently one or more cell phones).
2. Address-Based Sampling (ABS) - ABS is designed to reach the 95% of U.S. households that can receive mail from the U.S. Postal Service (which misses some rural locations). ABS samples go beyond traditional mail samples (derived from white-page listings, which exclude unlisted phone numbers) for broader coverage, often without an attached telephone number and sometimes without even the name of a resident (15% of the time for SSI's ABS sample). Due to the increased cost of RDD to wireless numbers, and issues with representing households rather than phone numbers, ABS is becoming an important method of conducting household surveys with probability samples. Government agencies, government contractors and syndicated research providers have all adopted ABS.
3. Random Digit Dial (Landline) - Because of the expense of calling wireless phones manually, RDD samples often omit cell-phone blocks. Unfortunately, this means that the RDD (Landline) sampling frame is not representative of the overall U.S. population, missing wireless-only households, which are demographically different, being younger and skewing male.
4. Targeted RDD - Due to the cost and difficulty in reaching high-income households, targeted RDD can be used to make calls into high-income phone blocks. A phone block is a set of hundred numbers with the same first 8 digits (e.g., the 216-872-79XX phone block). Regression analysis against U.S. Census data and household data is used to determine phone blocks with high average income. Targeted RDD can also be used to increase sampling rates of Hispanics and African Americans.
5. Listed Household Directory (LHD) - LHD was a once widespread convenience sample, when direct mail surveys were common. White page listings are used to generate households. Unlisted households, which are not available in this sample, are younger, more mobile, more likely to unmarried and differ in income and ethnicity from listed households.
6. Targeted LHD - Subsets of LHD can be used for specific groups, such as selected age demographics.
7. Low Incidence Targeting - Convenience samples are available for commonly researched groups, including people who suffer specific ailments, own particular electronics products, have specific levels of education, suffer from common ailments, and so forth.

The Online Continuum of Sampling Solutions

All online sample sources are convenience samples, since there is no master list of all email addresses that can be referenced. Online sources vary in coverage and recruitment method.

1. Web Intercept (River Samples) - Random web intercepts invite visitors to surveys through banner ads and pop-up windows published to a range of sites, including search engines and social media sites.
2. Social Media Intercepts - Some river samples are solely of social media sites, perhaps offering incentives tied to participation in the social media site itself. This sample skews even younger than general internet users.
3. Commercial Panels - Where intercept surveys can attract people new to online surveys, panels are built upfront with people who agree to join primarily to participate in surveys across a wide range of subjects. Commercial panels are constantly refreshed and maintained but can suffer from a range of quality issues (c.v., http://blog.vovici.com/blog/bid/26949/Commercial-Online-Panels-from-a-Total-Survey-Error-Perspective Commercial Online Panels from a Total Survey Error Perspective).
4. Custom Panels - Organizations can build panels for their own specific research needs. These provide very targeted sample sources.
Selecting the right sample source for a specific research project typically involves trading off speed, price or accuracy. Each sampling solution has its own role to serve.