Can the wisdom of crowds help improve the hit rate of IT projects? It is well known that many if not most large IT projects fail. It is also well known that they could have been saved if proper intervention had been undertaken at an earlier point. The question is just how to get an early warning? How will management or the project manager know when things are going wrong?
One way could be prediction markets. Prediction markets are based on the basic assumption that crowds are wiser than any individual even an expert. The obvious question then is, why not just send out a questionnaire on a regular basis to ask how people are feeling about the situation. The problem with questionnaires is that they are more easily distorted.
One example is the past few elections for President of the USA, where the outcome has been more precisely predicted by the bookmakers than the polls. The reason for this is that an opinion in a poll is free, but when you bet you have a stake in it. It is not free.
Prediction markets are modelled on this dynamic. Participants in the market are given a virtual currency to invest in the likelihood of future events. This currency then will eventually be turned into some sort of reward in real life.
The British company qmarkets has developed software that allows companies to use prediciton markets to predict the future*. Companies have used it to predict which new product to invest in or which drug to bring through to clinical trials, or which risks are most likely to occur. If applied properly, prediction markets are the most accurate way to predict uncertain events.
The project portfolio managers challenge
One of the most challenging parts of project portfolio management is to manage the uncertainty in the portfolio. It is inherently uncertain whether a project will be finished in time for the deadline, whether resources are available or whether the quality will be good enough.
Standard textbooks prescribe the use of complicated algorithms and reference class data to determine whether the project is under spending or overspending with regard to the burn-down rate of a project. A project that burns less hours than expected may experience resource bottlenecks or lack of commitment. A project that burns more hours than expected may have underestimated the amount of work. Either way, the likelihood that the project will be on time and on budget is diminished.
The challenge is to be able to say exactly when a project is going off track. This can be done if we know when exactly a project is overspending and when it is underspending. The problem, however, is that all projects are different. Some have a much slower start and explosive finish, others start really energetically and then level off.
I Predict
When traditional ways of detecting when a project is going off track don’t work, prediction markets may offer precisely the solution needed. Prediction markets are good at early detection of sentiments and information that is distributed among a group of people. It could therefore serve as an early warning system.
The prediction market could be implemented as a market where the question of whether any project in the portfolio, or at least the largest of them, would meet their deadline. All employees would be given a pool of virtual money to buy stocks in the project. These stocks can be bought and sold at any time. The stock price therefore reflects the probability of a project meeting its deadline.
The portfolio manager and senior management could therefore, just by monitoring the market, much earlier spot projects that had a high risk of failure. That would allow them to do one of two things: discontinue the project or give it more attention to bring it on track. Either of these options would benefit the company. It would either minimize losses or maximize likelihood of delivering quality on time.
These are the hard benefits, but there are also, soft benefits that would not follow from the traditional method. Since this is a game, it is a way for employees to take the mind of their work for a second and still be doing something relevant, instead of posting links to youtube movies on facebook.
It is also a way to create awareness of all the projects your company is running. Suddenly employees will know what other departments are doing and they may even serendipitously discover synergies or redundant projects
Since it is a game it can be entertaining, which may boost morale.
Further it may create positive incentives to make the real projects finish on time. If you invested in it, you want to protect your investment (this is why it is probably a good idea to disable the possibility for shorting project socks).
Future potential
Unfortunately this solution is not in use anywhere yet. It is only a possibility, but with the market maturing and ideas about collective intelligence becoming more widespread, it is probably only a question of time. the current economic climate calls for solutions that would help you minimize losses and maximize success. The problem is just whether the cultural gap of actively letting employees play and even rewarding them for it is too big for most companies.
The typical risk analysis is inherently flawed. It is only based on the project managers subjective opinion and furthermore it is typically made with a 5 point scale idiosyncratically interpreted. This doesn’t guarantee a good estimate of the future
Furthermore we are natural born optimists. We overestimate the likelihood of something good happening and underestimate the probability of negative events occurring.
* Best Buy used decision markets to find out whether a new product idea would succeed (http://online.wsj.com/article/SB122152452811139909.html). They built a market where it was possible for about 2000 employees to trade imaginary stocks that related to questions about the future. So, if a question was very likely to be true the price of the stock would go up and if it was likely to be false, the price would go down. This means that the price of the imaginary stock would match the probability of the answer being true.
References:
Justin Wolfers and Eric Zitzewitz: “Prediction Markets”
James Surowiecki: “The Wisdom of Crowds – Why the Many are Smarter than the Few”
Eduardo Miranda: “Running the Successful Hi-Tech Project Office”
Photo: thetaxhaven @flickr
Danish 
English