Trend Reports are So Trendy Right Now
Posted by Vovici Blog on Fri, Jan 14, 2011
This January, like most January’s, I have been pulled into a number of internal and external projects involving creating or updating trend reports for ongoing transactional surveys. Let’s face it – they’re addicting. Everyone wants to see how they did last year. Everyone wants to see their satisfaction levels, their loyalty attributes and their WOM ratings increasing steadily.
Trend reports are addictive, but many times they are examples of chart junk, with all the nutritional value of a bag of potato chips. Here are some of the common mistakes I see:
- Showing one year of data – Many times changes in attitude are seasonal, and this seasonal trend is hidden when only showing year-to-date data. If you regularly release new products at a certain time of year, or if any of your products themselves are tied to holidays, you will often see spikes of activity to call centers, for instance, that lead to longer response times, more harried reps and lower satisfaction. Sometimes there is a regular pattern of seasonality that can’t be explained, but that’s obvious when you look back over a few years of data. You’re not going to see these patterns if you’re not looking past 12 months.
- Grouping data by short time periods – I like to have at least one hundred responses for each time period. If you are showing data weekly, but only have a few dozen data points each week, you are going to be showing a wide range of volatility. Even monthly data often needs to be displayed quarterly instead. Use time periods that are appropriate to your sample size.
- Including recent responses in trend reports – Who in MR hasn’t received an urgent call like this,… “I just saw that our satisfaction has declined 20% this month – what’s going on?!” Well, what’s going on is that only 10 responses have come in so far this month. No need to panic – yet. Just because you can produce an up-to-the-minute report doesn’t mean that you should.
- Using trend reports at all – Another common mistake is taking a trend report and filtering it to show just the results for a product line or geographic region. Sometimes this reduces the sample size so significantly that a trend report is meaningless. Instead, compare the overall results for the entire time frame by product or region – that will produce more representative insights.
If I had a trend report of my clients’ use of trend reports, I’m sure it would be shooting up, up, up! They are a powerful technique, but use them wisely and don’t obsess about them.
What mistakes have you seen? What best practices do you recommend?