Welcome to the Listening Post!

Survey Translation from 30,000 Feet

Tweet

 

Nancy Porte, VP of customer experience for Vovici, presented this morning to the Technology Services Europe conference in Barcelona on "Joining Voices:  Understanding and Leveraging Multi-National Customer Feedback". One of her key points was that, too often, novice researchers think of translation as a front-end task in the survey design process, when in fact the needs of translation must be considered at each stage of the survey process: Study Design, Survey Design, Fielding and Analysis.

• Study Design - After you decide which languages you need to conduct research in, hire the appropriate translators and develop a conservative schedule that takes into account the many pitfalls that await the international researcher.
• Too often, translation is "insourced" to employees who do not have the requisite experience. The best translators have an excellent understanding of the originating language, the target culture, the industry and survey research itself, since survey translations are more demanding than traditional translations. Find and contract with native speaking professional translators (preferably certified translators) where possible.
• The biggest mistake in designing studies with translation is creating unrealistic schedules. Budget for lots of extra time: time to translate the final questionnaire into each target language, extended time in the field to account for local holidays, and time for translating and post-coding open-ended comments.
• Survey Design - A common mistake is to finalize the questionnaire in the source language and then send it off to be translated into the target languages. Instead, you need to design a master questionnaire that will be localized, not just translated. Structure it for translation:
• Avoid jargon, slang and technical terms. Rewrite for readability.
• Use as few open-ended questions as possible.
• Use closed-ended choice lists tailored for local markets and brands.
• Use country-specific skip patterns since entire sections of the questionnaire may differ, depending on the structure of the industry being studied in that country.
• Make certain to prepare back-translations from the questionnaires to ensure quality and accuracy. You can also turn to panels of respondents to assist with translation, as InSites Consulting is pioneering.
• Fielding - When inviting people to complete surveys by email, send them an email in their language. If your database list does not specify their language, or you are concerned about the accuracy of your data for that field, send them a generic language picker with a unique link for each translation. While you can set online surveys to automatically route to the appropriate translation based on the primary language set in the browser, respondents' browser settings do not always match their native language.
• Analysis - While fielding a multilingual survey, one of the best feelings in the world is analyzing in real-time the results to closed-end questions. Thanks to your earlier work translating the questionnaire, you can immediately chart the answers as they pour in despite being completed in dozens of different languages (assuming you have the right survey software). Open-ended questions, of course, take more time to analyze. During fielding, you can use machine translation to get the gist of what respondents are saying, but for accurate final analysis rely on human translators. By all means compare and contrast behavioral results across markets and languages, but take care with attitudinal data, which can often be subjectively different-better to use that information for exploring attitudes of different segments within the country or for establishing a baseline for future trending.

Nancy gave the case study of the multilingual surveys of one airline, a Vovici customer with over 3000 flights a year to over 200 international destinations. The airline fields a satisfaction survey in ten different translations: Chinese (two dialects), Dutch, English, French, German, Korean, Japanese, Portuguese and Spanish. Local translation companies in each market were used for the questionnaire, which measures satisfaction with flights and with the frequent flyer program. Over 100 email triggers are used depending on what passengers are dissatisfied with. The survey was fielded in four weeks and initially ran well in every country except Japan, which suffered from suboptimal response rates and high abandonment rates. In the past year, over 2.2 million responses have been collected and analyzed, informing decisions about seat selection options and benefits of the frequent-flyer program. As a result, the program is now being rolled out for general passenger satisfaction research.

When conducting your first multilingual study, make certain to consider the impact of translation on the entire process, not just questionnaire design. Of all the research projects you will do, multilingual studies will require the greatest coordination of resources and general project management skills. With proper planning, though, your multilingual survey will have a smooth flight!