What kills prediction markets

Recently I have been amazed again at the accuracy of prediction markets – and at their uselessness and irrelevance.

In the last weeks we had a few major decision coming up in Germany. The first one was who will win the soccer Bundesliga; the second who will be Germany’s Next Topmodel. Both competitions were very tight calls and had huge media attention. It was no suprise then that there were numerous polls on who will win these races. In the first case, it has been the closest race in recent Bundesliga history. Two games before the end, there were 3 teams that were in close reach and could all win one of Germany’s most important trophies. In the leading spot was Schalke 04, the team with the biggest budget of the three and close runner-ups in recent years. Second came VFB Stuttgart, the surprise team of the season with young players, relatively small budget and not a cup in 15 years. Third came Werder Bremen, the team that dominated the first half of the season and also the Bundesliga champ in recent years. Close match, open outcome. What would the masses predict?

Germany’s biggest newspaper (Bild Zeitung) opened a poll on who would come in first. As people voted they seemed to favor the underdog: VFB Stuttgart. Two weeks later, Germany had a new national champion – the young, inexperienced players from Stuttgart. Wow! That was a great call – no clarity in that decision and the masses were correct. James Surowiecki must have liked that.

The second show-down came with the casting show of Germany’s Next Topmodel. Heidi Klum took the Tyra-Banks role and eliminated a beautiful ambitious girl each week. The final was made up of three hopefuls: Hana (the dark-haired czech with Angelina-lips and previous model experience), Ani (the blonde who worked in her parents boutique) and Barbara (the redhead studying math in Bavaria). Who would wear the crown at the end of this competition?

Again, Bild set up a poll. To my suprise (and some other bloggers as I have read), the predicted Barbara as the winner. Last weeks show came down then with a big suprise: Barbara won the competition. Another Surowiecki-moment!

Both calls have been sort of odd to me. There was nothing clear in any of those calls. But both times the masses trumped the experts. There were two additional observations though that kind of killed the Surowiecki-glory of those polls:

What lacked in both polls was the number of participants. In the Bundesliga draw there were 2000 people engaged and in the Topmodel vote about 1000. That is almost nothing. The Bundesliga is followed closely by maybe 20% of the populiation (would result in 15 million individuals). .The Bild Zeitung is the biggest daily publication in Germany and is filled with Bundesliga news daily. Of all those people only 2000 voted. I am too lazy to do the math, but it doesn’t strike me as a lot.

The picture fort he Topmodel-competition is similar. In pre-final episodes they had a market share of 25%, which is around 3 million people. Of all those media-savvy young people who blog and youtube about this event, on 1000 cast their vote in prediciting the outcome. These marginal percentages in participation are similar to what I have witnessed with the pilot at our company. At ouf the 125 people signed up for the market, only 4-5 really traded. This is a lowsy participation.

It is all the more striking since the accuracy proves so true. Also at the internal market, the active people were quiet good and the returns were nice. People genuienly like the idea of bottom-up information gathering and no-bullshit predicitons. But then no one participated. If I look on the web at prediction scenarios, they don’t look much better: bizpredict is lame etc. So even though the results are beautiful, they are obviously of no use to the people.

Which brings me to the final killer on the use of prediciton markets: irrelevance. What could Werder Bremen do about being traded as a non-winner? What could Schalke or Hana do about it? In a company we might say that these information can filter in to the correction process or uncover problems early. May be. But so far the prediciton markets don’t have a mechanism to feed up the right ideas to address a loosing trade in the market. And that is a problem. A big one. Transparency only helps if it can trigger some correction actions. And if the masses are not involved in solving the problem then the ball is back in the hand of the few experts.

While Surowiecki seems relevant to our thoughts on how to gather information and our ideology of the positive effect of involving people, the reality shows that it is no easy step to do what matters most in business: being useful and relevant.

The (un)wisdom of prediction markets

We are just on the verge of launching a first prediction market in my area at work. I have been infected with the idea of swarm intelligence by reading Wisdom of the crowds. The premise seems great: many people are smarter than the smartes. The promise is: this tacit knowledge can be made visible through a stock-market mechanism. So I researched a bit and found that companies use this for different purposes: Google predicts the launch date of new products; Microsoft the numer of bugs in a software; HP looks at sales volumes and GE judges innovative idea. These are all nice, but the more I deal with it, the more I hear the question: so what?

My standard reply is something around: gathering people, sharing the process, making intangible knowledge transparent, bringing focus to a topic. What draws me to prediction markets is the potential as an organizational tool. It involves the grassroots people who are doing the work and makes decisions transparent and accountable. If the market works correctly and suppliers are judged by their customers, then it could even involve something like an internal supplier-customer-rating that might work better than silo-focused MBOs. Anyway, that is a bit further away. Still, I wonder myself: so what?

What happens if your market shows that the product won’t launch on time? Who has a benefit from that knowledge? I realize that many people don’t even want to know. Not that they don’t want to face the truth – the know already – but they are not comfortable with having it black on white and for other to see. Also: what happens if the market predicts a no-show for a product? Is there any mechanism for judging the reason or suggestion alternatives? That would certainly be nice: use swarm intelligence to suggest improvements and have the market collectively judge the best ones.

Similiar thing with bugs in a software. Nice to know that this might be not up to standards, but even nicer to have a market that predicts (and pre-selects) the most promising levers for chaning it. Sales volums is similar, just as innovation.

It seems to me that the prediction market suffers from a criteria it has to be matched against. The value of the stocks being traded is tied directly to the criteria being defined. The markets don’t allow collective problem-solving, impact estimation or decision-influencing. It reflects what people think about the future. Currently, I haven’t see a way that it enables an organization to shape the future with collective intelligence. With that it might really answer the question of so what. It would be a tool for improvement and collective participation. Now, that is a stock I would buy.

How to get people to work together and change

I had a big aha-moment reading a recent Harvard Business Review article. In this text, Clayton Christensen and colleagues discuss that there are different tools to get people to work together and change behaviors. Most change books are about the one method you can use to change organizations. But they actually highlight that there are different ways to drive change depending on the context factors. So they propose to look on the agreement along two dimensions: what people want and how the envision it happen.

Depending on where you are along these axes, different tools are important to support collaboration or change. Especially this cause-and-effect axis is very interesting. I have repeatedly observed that in an intense group there is high agreement on what is wanted, but not so much on the means. The tools they propose are helpful to navigate in this system. The message is that there is simply not just one way to bring about collaboration. You need an understanding on where you are and then what tools you use to drive that. Or as they say it so well themselves:

“One of the rarest managerial skills is the ability to understand which tools will work in a given situation—and not to waste energy or risk credibility using tools that won’t.”

Good stuff!

Better than 360 Feedback

Companies invest a lot of money in feedback – 360° feedback. They assume that better feedback will lead to better self-understanding and improved performance. OK, so far. Last night I listened to a presentation by Patrick Lencioni (Confronting Team Dysfunctions) who talked about a cheaper and better way to get feedback. This is how it works:

In a team, everyone writes down one positive thing about each other person on the team. Then you start sharing that, for example starting with the leader. Everyone share what they bring to the team. Not just their function, but what they add in a unique way. Then the next person and so on.

After that first round of positive things, everyone writes down one thing that each other person should improve. Same again: go around and everyone shares their comments.

This seems like an interesting process. Most people will probably feel uncomfortable with the directness of the approach. Someone asked in the show: can this be done anonymous? Sure, it can. But doing it open is even more beneficial, since each person commits to their feedback and is frank about it. It actually builds trust when people start speaking open. It also establishes a culture of feedback when people start opening up on their views.

I would like to try this some time. It seems that the uneasiness with being open is quiet a barrier. It is strange that people are so private about their relationships with the people they work with daily on a team. Why is that so difficult? Why is there such a “don’t talk directly about personal issues” thing? Sure, nobody wants to be constantly evaluated and judged by their peers. On the other hand: wouldn’t it be great to be in a team where you are committed to each other’s success? And where you are aware of the weaknesses and cover each other on them? I’ll let you know if this has actually worked for me…

The frustrated and the world-changers

I’ve often wondered about the attitudes in a company. The atmosphere that people work in can make a huge impact on the productivity and focus an organization has. Through a friend I came across some thoughts on “organizational energy”. It is a research program by Heike Bruch at St.Gallen, Switzerland which tries to answer the questions under what circumstances organizations are hot and under which not. For her, organizational energy “involves the vitality and perseverance of a company and it is expressed in the degree of temperament, intensity, speed, and stamina of its employees in performing their work.”

She proposes 4 energy zones for organizations along the matrix of quality and intensity. She says: “The quality of organizational energy distinguishes between positive energy (e.g., enthusiasm, joy, satisfaction) and negative energy (e.g., fear, frustration, sorrow). Intensity refers to the strength of organizational energy as seen in the level of activity, the amount of interaction, and the extent of alertness and emotional excitement. The intersection of intensity and quality determines an organization’s energy state, which usually falls into one of four categories: the comfort zone, the resignation zone, the corrosion zone, and the productive zone.”

Along with the zones, organizations react in a way and get into “energy traps”. You get stuck in some quadrant, try to correct but drive the thing deeper into trouble.

OK, the interesting part is how to move to the different zones. Here, Bruch describes 2 strategies, which are seen in business quiet a bit: slaying the dragon and winning the princess.
Slaying the dragon is about cranking up the fear factor. Whether unspecific (we face bankruptcy) or specific (kill your direct competitor), it turns up the intensity and gets people going.

Winning the princess is basically about vision crafting. Creating a powerful picture of the future and what that will feel like. It aims at getting people excited and engaged.
The point is that both of these strategies have to be specific, vivid and close enough for the people to get moving.

From their webpage it seems that the publication have been a bit quiet recently, but I hope they develop some more. The two strategies see a bit unspecific and general, but it seems like a good direction for me. Harvard has some more on it as well as others.