How Text Mining Changes Questionnaire Design
Posted by Jeffrey Henning on Fri, Jan 29, 2010
Tuesday I wrote about how text mining changes survey analysis, one of the key themes of the panel discussion "Two Worlds Together: Market Research & Text Mining" at the Clarabridge Customer Connections conference. Another key theme was the impact that text analytics has on traditional questionnaire design. Todd While of Clarabridge moderated a panel discussion of John Georgesen, Ph.D., senior director at Convergys; Michael House, division vice president at Maritz; and Scott Evans, Ph.D. of Harris Interactive.
John said that there is "an art of crafting open-ended questions in a survey." For some domains, such as the cable industry, an open-ended question provides really rich sets of data, so that Convergys could categorize the source of issues and identify which issues were in the client's control.
Michael said, "Unstructured formats provide a more conversational approach for respondents. As researchers, we can move on from the more traditional route of survey research to a more customer-friendly approach."
I asked the panel what type of usage of open-ended questions was appropriate: while asking a few open-ended questions at the beginning of a survey is a great technique to maximize the number of open-ended responses, a survey with 20 open-ends is not engaging for respondents. What did the panelists recommend?
Michael said, "Our vision long term is to have a conversation, going back and forth, but the flow is to talk about all the topics in more of a give and a take. We've designed questionnaires that flow through paths. Customers are engaged in such conversations and enjoy the process. So the structure in the future is more tailored to the respondent. We are not going to get totally away from closed-end questions but we will have more open ends."
John said, "In an online methodology, engagement is critical. In online research, there is a lot of focus on engagement in the terms of slider bars and drag-and-drop interfaces. I would argue that the data you get from those isn't nearly as rich as asking respondents to tell a story in unstructured ways. This is a key area of focus in the future."
Michael concurred. "Most customers want to tell you their story and if it is significant to them they will write paragraphs. If they don't, the absence tells you something, too. This revolutionizes surveys. What is the most appropriate way to ask open-end questions to engage respondents? We get essays in response."
Scott concluded, "If you open this door, there is pressure on you to respond, for your company to respond back to customers." If they write you an essay, they want a personalized reply.
How do you think text mining will change questionnaire design? Do you see it as a danger or opportunity?