Survey Translations: The Translator may be a Traitor
Posted by Jeffrey Henning on Thu, May 28, 2009
In March, Scott Blacker joined us as our new senior director of product management. In his career, Scott has held multiple product management positions, most recently with Rosetta Stone, the leader in language-learning software. Appropriately, then, Scott's first post is on the perils of translation and surveys.
In the race for global market share, many organizations with an international customer base require surveys to be deployed in multiple languages. Typically, the survey is written in the native language of the survey author, translated into the target language(s), and then deployed. Unfortunately, this process misses a critical, often-overlooked step: back-translating a survey into the native language of the survey author.
The reasons for skipping this step are easy enough to understand. Translation costs are expensive, and paying to both translate and back-translate a survey doubles these costs. Additionally, time demands on survey deployment are often intense, and back-translating can add valuable days to the survey deployment timeline. However, skipping this step can have serious consequences when ultimately analyzing survey response data, effectively killing the survey ROI.
The original survey author is the subject matter expert on the topic at hand. The nuance of how a question is posed - and the specific word choices involved -matter greatly in determining the nature and validity of the final data collected. Translators may have several linguistically correct options to choose from during the translation process, but may choose a nuance that misses the original intention of the survey author. Back-translating through a second translator (who has no affiliation with the original translator) greatly reduces the likelihood of this type of error. Back translating allows the original survey author to:
- Validate the quality of the initial translation
- Ensure that the nuance of the translation matches the original intent
- Open up a dialogue with multiple translators to build a consensus around the best possible translation.
This actually happened to an associate of mine just last year. In deploying a satisfaction survey into the Japanese market, he paid a premium for a top-of-the-line translator. While the survey was linguistically correct and made perfect sense in Japanese, the translator has chosen a word for "satisfaction" that was closely aligned with "happy" in the Japanese language. When the results came, it appeared as if the product was a success - nearly 80% of Japanese indicated that they were "very happy" or "somewhat happy" with the product.
However, when looking over some of the open-ended qualitative feedback, he realized that something was amiss. Japanese respondents had interpreted the word not as "happy", but as "fun". So yes...80% of Japanese respondents had indicated that the product was "fun" (which it was - it was a learning game), but fun in this case bore little correlation to satisfaction. Other elements, such as the ability to achieve a learning objective (after all, it was a learning game), turned out to be much more relevant to the customer's overall satisfaction.
In this case, the problem was caught, but not before the results had been presented to the CEO. The survey had to be re-run, incurring additional costs and delaying decisions on whether a major marketing campaign should be run. It also didn't reflect well on the market research department that was responsible for deploying the survey. More seriously though, had the problem not been caught, the company might have invested millions of marketing dollars into a product that had no chance of succeeding in that market.
As a survey author, you spend hours agonizing over diction when constructing questions in your native language...not investing the same time and resources into ensuring that the same nuance is appropriately reflected in your globally deployed surveys could cause you to fail just inches shy of the finish line.