Welcome to the Listening Post!

Where’s the Devil? Oh, right.

Tweet

 

As it turns out, the Devil is in the details.  As someone who is anxious to get results and start taking action to improve processes, I’m often tempted to take short cuts when it comes to research planning and design.    But those short cuts can result in bad data – or, more specifically, the difference between reproducible results and the accuracy of those results.

A recent study, the 2011 Temkin Loyalty Ratings, concluded there were differences in loyalty ratings among ethnic groups.  Key findings included 1) Hispanics were the least loyal group in 10 out of the 12 industries and 2) African Americans gave companies the highest experience ratings in most categories. 

For those of us building customer loyalty programs, this study underscores the importance of understanding the feedback we receive from all subgroups, including ethnic groups.  Attention to detail and effective research design is required to make sure we are hearing all customer voices clearly.

In an article written by Jim Starks, published in the February edition of Quirk’s, specific recommendations are made for those practitioners of Hispanic research.  The challenge exists for researchers because different cultural groups respond to scalar questions differently.  As Mr. Starks states, in the U.S. particularly, Hispanics tend to use higher scale points than other ethnic groups.  The scale response differences between Hispanics and non-Hispanics have long complicated market researchers’ attempts to draw meaningful conclusions about differences between them. Specifically, these practices are recommended:

1. Sample management – Starks recommends careful sample management for any study where subgroup comparisons are important. Primarily, careful quota construction is preferred, acknowledging that costs and timing constraints are often prohibitive. In this case, weighting can be a useful tool, although not without its own pitfalls. At the very least it is recommended to control for country/region of birth, gender and educational attainment.
2. Normalizing data - Some practitioners use various techniques for “normalizing” data among Hispanic respondents.  Starks does not recommend normalizing Hispanic response data on an individual level.  Instead he recommends normalizing data on a group basis for relevant between-group comparisons or for comparing with norms.
3. Choice of scales – Starks’ study only dealt with one type of scale (1-10, anchored) in one application (attribute ratings).  However, they allow that other scale lengths (five or seven point) may be effective in shifting behavior.
4. Question Types – Since scales can reveal little differentiation among brands, products and concepts in Hispanic research, the practitioner should consider choice-based respondent tasks.  Even choice-based questions with degree of preference follow-ups preserve a significant amount of differentiation.

Comparisons between non-Hispanics and Hispanics are frustrating because of the broadly accepted knowledge that Hispanics tend to utilize extreme scalar points.  But with careful attention to detail in research design, understanding about our diverse customers will deepen.