Paper Survey to Web Survey
Posted by Jeffrey Henning on Thu, Aug 06, 2009
Reading about the
U.S. Post Office's financial shortfall this week reminded me of the following question, which highlights the small role we've played in decreasing the volume of U.S. mail. Roberta, a recent attendee to one of our
research webinars, posed the following question:
Our customer base strongly skews to an older, retired (and presumably less technically-savvy) demographic. Accordingly, we have continued to use traditional postal surveys to measure satisfaction in this conservative customer base.
We are eager to switch to an online survey methodology, but have two related concerns: (1) comparability of historical data (data collected in postal survey vs. online) and (2) switching methodology may effectively 'disenfranchise' some older customers who may not have email access.
From a technical standpoint, how serious are these concerns and what, if anything, should we do to address these concerns -- either in terms of survey methdology or at the analytical stage?
If all of your customers are older, then you have less to worry about than if your customers span a wide range of ages. If you have a mix of under-30 customers and over-60 customers, for instance, than switching to the web would change the proportions that responded, which would have a dramatic effect on the results; for such a case, I would advise weighting the results by age segment to reflect the age distribution of customers (see my
Age Question blog post for how to ask respondents their age, if you don't already have it).
Our most conservative client migrated data collection methodologies over the course of 12 months, ratcheting up the percent of surveys using the new methodology every month until the end of the first year, at which point all surveys were done using the new method. Each month along the way, they compared the answers of the two groups to see where there were differences and to understand what the causes of the differences were. For analysis, they reported the overall results and the results by each collection mode.
If you did something similar, at the end of the year you could do year-over-year comparisons using only the results collected the year before using the new method.
As to disenfranchising customers, you could position the move to the Web as being done for environmental reasons, to minimize your firm's use of paper and the fuel necessary to ship the mail back and forth. (Hat-tip to Gartner for teaching me how to sell
online surveys as 'green'.) You could say that all people would be moved to the new method unless they specifically returned a postcard opting to still receive the surveys by mail. Such a transition to the web is rarely needed for more than a year.
Moving from paper surveys to web surveys will:
- Lower your costs by eliminating the need for printing and mailing surveys and then doing data entry on the completed surveys
- Dramatically speed up reporting by eliminating the current lag where surveys are in transit to you and then queued up for data entry
- Enable you to set up survey alerts and trigger emails to immediately take action when a customer is unhappy.
Oh, and it is definitely better for the environment! (If not the Post Office!)