Welcome to the Listening Post!

Prediction Markets

Tweet

 

Randy Berkowitz, VP of global market research for Combe Incorporated, presented “Tapping the Wisdom of Crowds to Predict the Future – Prediction Markets for Concept Testing” at the 2010 American Marketing Association’s Marketing Research Conference in Atlanta. Combe brands, including “Just for Men” and “Odor Eaters”, have 70% brand awareness but 7% market penetration among its potential users.

“What if you could predict the future?” Randy asked. Most people answer that it will make them money. “We instinctively know that if we could predict the future, we could make more money.” As market researchers, we get paid to predict the future – to predict the product concepts that will be most successful, to predict the ads that will create the most sales and to determine the packaging designs that would leap off the store shelves.

“The wisdom of the crowds” is that a large group of average members can outperform a small group of experts. You can measure the wisdom of the crowds, as easily as using a survey. The audience of “Who Wants to be a Millionaire?” when polled had the right answer 93% of the time.

Real-world prediction markets include:

• The New York Stock Exchange, “with traders who invest in outcomes, weight their decision with currency allocation and are rewarded for accurate predictions.” Stock prices are very predictive of company’s future performance.
• Another prediction market is Betfair, which predicts sporting events more correctly than sports pundits do.
• The Hollywood Stock Exchange (HSX) predicts the Oscars with 90% accuracy. (It predicts that the new movie Chain Letter will do twice as as well as Hatchet 2 this weekend.)
• Iowa Electronic Markets (IEM) were able to predict the November elections in August with 75% accuracy, outdoing polls.
• Google’s internal prediction market is used for internal assessments: how many Gmail users in 5 years? Will the company make its quarterly number?

Surveys of the experts are likely to be less accurate than market prediction. How do the two methods compare?

“Which idea do you think will be most successful in the marketplace? How strong is that conviction (how many shares of that idea will you purchase)?” Traders are more engaged than survey respondents because they have something at stake; they find the process fun and engaging and are rewarded for performance not just participation.

“Combe is a very conservative company and conducted a side-by-side study.” The Combe screening program was conventionally doing a mall intercept, screening for the target audience, with 250 respondents asked many questions to identify the top three ideas. In predictive markets, on the other hand, traders get virtual dollars; each concept has a starting price. Prices start at 100 divided by the number of concepts; prices fluctuate as trading occurs. Price movements are analyzed.

Combe used Infosurv to test two predictive markets: four concepts in health, five in beauty. Each had a control concept to gauge concept strength. Combe recruited 200 traders per market, from the general population, with self-selection for market participation.

“What are the chances of each concept being most popular among consumers?” The trader allocates a $1000 virtual dollars. Over time, as more traders participated, the stock prices of the concepts separated, with Concept A increasing from $25 to $36, with the other concepts declining to around $20.

“The predictive market gave me the answer in one week compared to four weeks for the 7% incidence survey that cost twice as much,” said Randy.

Quotes from traders who participated:

• “Excellent way to judge a product. Makes one think of the best logic and back it up since money is on the line.”
• “I loved it. You should have more of these to do. They’re fun, and it helps you to learn more about stock market trading.”
• “I loved the idea! It was a super fun way to preview new products and get feedback on what other people think.”

Predictive markets successfully predicted the most preferred concept and the same directional rank order of the concept. The study was completed in less than two business days and utilized a general product sample.  Over 70% of traders described the methodology as “fun” or “interesting”.

Prediction markets are powerful alternatives to surveys for concept screening. “Your customers are not always the best predictor of their own behavior.”

For more on prediction markets, check out Infosurv's white paper, Prediction Markets for Concept Testing.