Welcome to the Listening Post!

What's Your Brontosaurus?

Tweet

 

One of my nieces is researching the Apatosaurus for a school project. To help her, my wife and I took her to the Museum of Science, where in fact we found a model of her dinosaur (see right). Now to me, that’s a Brontosaurus, but schools today do a much better job teaching paleontology than they did when I was a cave-kid.

There’s an urban legend that a Brontosaurus was simply an Apatosaurus with the wrong head. It’s true that the first mounted display of a Brontosaurus, back in 1905, used the head of a Brachiosaurus, but that was because scientists had no idea what the head of a Brontosaurus looked like—so they went with the head of a dinosaur they thought was similar.  Two years earlier, Elmer Riggs had realized that the Brontosaurus was really a species of Apatosaurus, and renamed it Apatosaurus excelsus, but it took time before this change was accepted by the majority of paleontologists. By then the name Brontosaurus had been cemented in the public psyche.

The great thing about science is that eventually mistakes are caught and corrected. Like the fact that the Brontosaurus – er, Apatosaurus – didn’t live in water.

When it comes to market research, though, old practices and disproved methodologies persist.

I asked folks on Twitter, “What's an obsolete research practice you continue to see researchers use, despite it being discredited or suboptimal?” In other words, what’s their Brontosaurus? Here were some of their responses:

• @akeirans and @rreitsma both mentioned grids. Check out these tips on how to refactor grid questions into other question types.
• @realtalktstprp chose 10-point rating scales and the Net Promoter Score. Here are some of the criticisms of NPS and some of the alternative customer-loyalty metrics. And here’s my argument against numeric rating scales: On a Scale of 0 to 10, Numeric Ratings are a 6-Pack of Suckitude.
• @VirtualMR is against using demographic questions that don’t reflect today’s lifestyles, and she backs it up with a blog post: Taking Your Market Research Demographics off Auto-Pilot.
• @lightplay is against very long questionnaires and reporting the margin of sampling error for convenience samples. See these tips on how to shorten questionnaires and an AAPOR quote on not using sampling error.
• @loudhouse dislikes qualitative questions that can’t be answered meaningfully: “What were you thinking 5 years ago, when you...?”
• @GrahamHill takes issue with the “publication of research paid for by someone else in expensive journals”.

What’s a wrongheaded practice that you still see today, when it should be extinct? What’s your Brontosaurus?