Welcome to the Listening Post!

Great Expectations Make for Lousy Survey Results

Tweet

 

When JetBlue first came to Boston years ago, I only booked a flight with them because it was the cheapest flight available. I have no brand loyalty to airlines; I’m a budget traveler and always book one of the least expensive flights I can find. I had heard about their seats with built-in TVs, but I didn’t care; I usually spend flight time working anyway.

So I was completely blown away after my first flight aboard an Embraer 190. It seats only 100 passengers, in 25 rows of 4 seats each. This means no middle seats!  I hate getting stuck in the middle seat. Four seats across also means that E190s load and unload much faster than traditional 7x7 planes with six seats across. I even liked the in-seat entertainment centers, as it led to quieter flights, and I got hooked on XM radio. The E190 has become without a doubt my favorite jet.

Now had JetBlue asked me the traditional expectations question – “On a scale of 1 to 10, where 1 is ‘falls short of expectations’ and 10 is ‘exceeds expectations’, to what extent did your flight meet your expectations?” – I would have rated them a 10 that first flight.

But now, after six years of flying JetBlue, they asked me to rate my expectations after a recent trip (using a different question than the old stand- by). Well, my expectations have been reset to a higher level – JetBlue certainly doesn’t exceed my expectations any more, as I now know what to expect. So they got a lower rating.

Am I any less satisfied than I was after that first flight? Not at all.

Unfortunately, JetBlue has fallen victim to use of the “disconfirmation of expectations” model – the dated but still popular theory that failure to meet expectations will greatly reduce customer satisfaction.  In the real world, expectations do not neatly align with customer satisfaction:

• A customer might buy a lower-quality product or service because it all that is available or affordable. This product may meet that customer’s lower expectations without contributing to their satisfaction.
• A customer might purchase a widely hyped or popular product and find that it does not meet the raised expectations they had for it. However, it might still perform significantly better than competitors’ products, thereby producing great satisfaction and loyalty, despite failing to meet expectations.
• Many product or service attributes might not have been part of customer expectations and yet can contribute greatly to satisfaction or dissatisfaction, despite the fact they were not anticipated in advance.
• Customer expectations may be amorphous and inchoate. Customers may enter a transaction with few if any expectations.
• Expectations are subject to being misremembered and misreported. Even when the expectations disconfirmation question shows a high correlation to satisfaction, it may simply do so because respondents’ reconstruct their expectations based on recent product experience. People can’t always accurately recollect their expectations prior to purchase, especially for purchases made a long time ago (as for durable goods) or for impulse buys.
• Over time, performance against initial expectations contributes a diminishing amount to satisfaction.

Asking customers for an expectations rating doesn’t provide enough context to be useful. I typically only include an expectations rating if I need to estimate an ACSI score. When looking to measure satisfaction, I’m more interested in current assessments of attitudes than retrospective evaluations of expectations. For those times when I am interested in understanding expectations, I prefer to explore what customer expectations were and how they matched up with the actual experience through open-ended questions.

Not convinced to forgo having customers rate expectations? Let me state this as baldly as possible: organizations commissioning surveys like to show constant improvement over time. Unlike other satisfaction and loyalty questions, disconfirmation of expectations ratings have a tendency to regress over time, as experienced customers increase their expectations of service and lower their ratings. So not only is it not a terribly useful measure, it’s a measure you’re going to get worse at over time.

Ah, expectations! I very well expect to be stuck in the middle seat of an airplane on my next trip, but it’s not going to make me more satisfied having that expectation met.

See also:

• Best Customer Satisfaction Loyalty and Experience Measure for You
• On a Scale of 0 to 10, Numeric Ratings are a 6 Pack of Suckitude
• ACSI (American Customer Satisfaction Index Score) & Its Calculation