Online Research Methods in Focus
Posted by Vovici Blog on Fri, Jan 28, 2011
Two themes predominated at the Market Research Society’s Online Research Methods conference yesterday in London. First, that researchers should embrace failure, as it is sign of experimentation and learning, laying the foundation for best practices. Second, that when it comes to which methodology is best, researchers need to bury the hatchet and embrace each tool in the toolbox as having its own special utility. Gone are the days of claiming one methodology works for all purposes—instead it is the age of synthesis.
The conference chair, Doron Meyassed of Promise Communities, began the day by discussing the need to “fail fast”, learning from technology giants like Google and Facebook and iterating to improve. We are still on the new frontier of the digital revolution, he said, with much to explore.
Online Communities
Derek Eccleston of eDigitalResearch said that “when it comes to managed panels and online communities, one size does not fit all.” Access panels provide low cost quantitative research; managed panels support quantitative research and recruitment for short-term qualitative exercises; permanent research communities provide customer intimacy for high-engagement brands in high-engagement sectors. When using these techniques, “don’t do things because you can but think about it properly,” designing it to meet the research objectives.
Diane Hessan of Communispace said that in the process of launching 415 insights communities (standing private communities) they had made more mistakes than anyone; for instance, creating communities that had little participation and creating health and finance communities while learning the regulatory requirements around those kinds of personal information. Nick Rich of Intercontinental Hotel Groups discussed his own experiments within his communities over the past five years, saying that communities provided the perfect environment for experimenting, failing, learning, iterating concepts and trying again: “a community allows you to make mistakes, such as asking the wrong questions”. Diane said that communities provide an excellent source of qualitative insights that should be validated by other methods. Nick concurred, saying that IHG had “expanded research budgets, not replaced anything with communities” and had used communities to refine ideas for testing in global quantitative studies.
Online Surveys
Tom de Ruyck of Insites Consulting emphasized that communities don’t have to be permanent; temporary communities work well for gathering insights for specific projects and initiatives.
I was next up, and discussed five mistakes I’ve seen in creating and managing house panels, stressing that studying “worse practices” laid the foundation for deriving best practices. (More on this next week.)
The closest anyone came to saying that all the old methods were rubbish was Phil Barden of Decode Marketing, who presented research from neuropsychology. Many customer decisions that researchers have approached as rational are in fact reached on autopilot and rationalized afterwards. Phil outlined the use of timed surveys that measure the speed of response to determine if the answers are given on autopilot or rationally and gave examples that illustrated the value of the technique for measuring the fitness of ads and products for specific brands. Past brand paradigms have failed, he argued, and neuropsychology held the key to a new understanding of brand.
BBFGs, Mobile & SMR
Jim Longo of Itracks showcased the complementary nature of online and offline techniques, stressing the importance of focus groups to meet customers face to face. Steve Hales of Synovate discussed three projects that embodied “bricolage”, using multiple techniques to create a picture of client issues: for instance, one project involved ethnography, then an online bulletin board where those who participated in the ethnography could talk to one another, concluded by a focus group. “Incorporating online as part of a mixed methodology gives us a broader perspective on both the consumer mindset and the business challenge,” Steve said.
Simon Wainwright of T-Mobile laughingly said that he was initially quite bad at his job; “every month we felt like we had information overload, leaving us with touch decisions to make.” Simon replaced an array of KPIs with two primary metrics (brand consideration and value for money) and focused on measuring, monitoring and improving these. He now uses MESH Planning to run an ongoing tracking study, where participants first complete a brand legacy questionnaire then are invited to send text messages of their reactions to ads from wireless phone suppliers (the study is blind, so participants don’t know it’s sponsored by T-Mobile).
Phil Burgess of Fresh Minds talked about companies approaching his agency asking to do social media research. He emphasizes to clients that social media research is a technique not an end in itself and is appropriate for only some business objectives.
Online research has come a long way since when surveys were the most prominent technique, with different methodologies evolving to meet different needs. The consensus was that a synthesis of methods offers the most robust insights for online research methods in the future.