Welcome to the Listening Post!

Best Practices in Mobile Research

Chris Ferneyhough and Sonia Bishop of Vision Critical discussed best practices for fielding online surveys to mobile audiences in the ESOMAR Online Research 2009 conference. Chris began by pointing out that mobile Internet adoption outpaces desktop Internet adoption and forecasts that eventually usage of the mobile web will be 10 times usage of the traditional web.

Vision Critical asked respondents to an online survey if they had received their email invitation on their phone: 1.9% had in the US, 1.2% in UK and 3.8% in Canada. Clearly, respondents are already completing online surveys on mobile devices, even though authors have often not taken this into account. Chris mentioned that a panel registration survey for a smart phone vendor didn't actually work for that smart phone, because of the registration form's reliance on JavaScript, which was off by default on the phone: the open-ended "Other (please specify)" box was locked and disabled because it required JavaScript to enable it once the corresponding radio button was clicked. Since client-side scripting is disabled on many phones, the data your survey collected may be wrong.

Researchers need to recognize the fact that online surveys are being completed on mobile devices and need to be optimized for that medium. The wide variety of smartphones at GSMArena.com reveals hundreds of different models with dramatically different market share in different countries. Colors and fonts are implemented differently by different phones and may not be implemented at all on a smart phone: a certain color may render some text unreadable.

Many respondents are unfamiliar with their web browser or alphanumeric entry mode on their phone's keypad. Many respondents are concerned about data costs: some have unlimited data plans, others have pay-as-you-go plans. Many have low data connection speeds.

To do further research on these topics, Vision Critical studied respondents who are smartphone users and are willing to complete questionnaires on their phone. The sample was balanced and weighted on gender and age, with 500 Canadian respondents, 118 US respondents and 107 UK respondents. The survey covered attitudes towards the national economy.  No significant statistical differences were found for closed-ended questions on mobile devices vs. desktop devices. For open-ended questions, of course, desktop users were more verbose.

The likelihood to participate in future surveys on mobile phones was greatest for iPhone users (47% were likely to), compared to only 34% of Blackberry users and 23% of all other smartphone users.

Sonia presented back-end mobile research best practices:

• Maximize use of the available space
• Profile your panel for smartphones, whose email addresses may vary for the phone vs. the desktop
• Identify devices and models supported by your data collection software
• Manage the process of deploying surveys to mobile panelists

And questionnaire design best practices:

• Use simpler question types
• Avoid Flash effects and JavaScript validation
• Write more concise questions and answer lists to minimize scrolling
• Put the Next button "above the fold"
• Limit survey length to 10-15 questions (unless heavily incentivized)
• Keep it simple: avoid color, grids, images, etc.
• Develop for the lowest-common denominator devices

Clearly, survey authors fielding online surveys need to take mobile users into account when developing surveys.