Random Sampling Explained
Posted by Jeffrey Henning on Mon, Feb 16, 2009
Using a random sample is the single most-important aspect of survey research.
Back in September, I gave an example of an AOL poll that awarded all 50 states to John McCain in the contest for President (see Random Samples Win!). Even though the poll had over 272,000 results, it was less accurate than telephone polls conducted with only 2,000 respondents. Those telephone polls used random sampling, where the AOL poll used a convenience poll where people self-selected themselves to participate.
Done properly, randomly sampling enables you to successfully generalize your survey results to the target group (e.g., customers, employees). To achieve this, random sampling has two key requirements:
- Randomness: an equal chance of selecting any member of the population ("probability sampling").
- External selection: respondents are chosen to participate rather than deciding to take the survey themselves.
The Randomness in Random Sampling
Imagine a big jar of gold and green marbles. You are interested in what percent are gold. You do not need to count all of the marbles to estimate this. Grabbing a handful and counting just those will give you a reasonable estimate.
Now, just to make it more difficult, these marbles are different sizes, meaning there is not an equal chance of selecting either type. You are more likely going to grab the green marbles, because they are bigger. If that happens, then random sampling isn't at work, and you would overestimate the percent that are green.
When dealing with people, you need to make sure that there is an equal probability of selecting any individual to do a survey. Back in the early 1990s, I had the good fortune to work for Bill Ablondi when he was doing a landmark market segmentation of mobile professionals. For this segmentation to be successful, we needed a random sampling of workers who weren't at their desk. Since we were using telephone surveys that called their office line, this was quite a challenge! If we only called a phone number once, we were more likely to get a worker who was less mobile. As a result, the sampling frame Bill designed required calling the same phone number on a dozen different dates, to minimize the likelihood of missing the most mobile of professionals. (By the way, as a result of that study, Bill coined and popularized the term "corridor cruiser" to describe mobile professionals who are away from their desk but not out of the office.)
Interestingly, the random in random sampling doesn't mean that there can't be a pattern. It is perfectly acceptable, and common, to get a list of email addresses or phone numbers, sort them, and then mark every Nth name (e.g., every other, every third, every tenth) as a candidate to be surveyed.
External Selection in Random Sampling
The second key part of random sampling is that the researcher chooses the potential participants. A quick contrast:
- A formal restaurant places a postcard with every other bill. Diners choose whether or not to fill in the postcard later and mail it.
- That same restaurant instead hires a pollster to stand outside the restaurant for a week and ask the bill-payer from every other exiting party some questions.
The first example suffers from self-selection bias, whereas the second offers true random sampling with external selection. Many of those people interviewed by the pollster would not have completted the postcard survey.
Again, if you are going to be making important decisions based on your survey, you should use random sampling to ensure that the results are representative of your target population.
