Tag: Wisdom of Crowds

Adrian McDermottAdrian McDermott November 8, 2010

Benefitting from the wisdom of the crowd

One reason that innovation software, and the idea of crowd sourcing, has become popular in the last few years has been the influence of a single book, James Surowiecki’s The Wisdom of Crowds. While in the market of business publications there have been plenty of sensations – Malcolm Gladwell’s The Tipping Point would be an obvious example – it is hard to think of any in the last decade whose influence is unchallenged and still growing. I should make it clear, however, that this blog post is not a book review, but rather a discussion of some of the main ideas of “The Wisdom of Crowds”.

The other reason for the attention paid to Surowiecki’s ideas is that they are radically different to received wisdom and could be very important to many companies at a time when the margin for error is becoming very small. The technical and innovation lead that Europe and the US had over China and other Asia Pac economies has been vanishing gradually over decades, as quality of manufacturing and speed of innovation have improved steadily. Companies have to be strong in every area of their operation, from market analysis through to delivering the products that fit and even redefine the market. Intelligence, innovation and decision making have to be done well, and fast!

How businesses usually solve problems
The standard model of business decision making has been that the more important the decision, the more senior and the fewer people are involved in it. Where the expertise at the top is insufficient, outside experts are called in to assist. The rationale of course is that the best brains in the business have got to the top, and there is no point using the company’s second-best. Plenty of tools have emerged to stimulate analytical and idea creation abilities within these groups, but the point remains that the make-up of these groups is very restricted.

The problems with this approach:
1. Life changes: It is a truism that in each new war, the generals at the top, i.e. the commanders who fought most successfully in the previous war, are soon replaced because they are fighting the wrong war. Likewise in companies, a history of promotion shows at best success in dealing with past challenges, at worst a series of lucky breaks or a talent for getting spotted.

2. Experts are people who seem to know something you don’t.  It is very hard to tell whether they really know it. Experts are also famous for disagreeing with each other: the top expert is often simply someone who scored the most recent victory in an continuous argument.

3. Similar to point 1, experts have a history of solving past problems, but this guarantees nothing in respect of new ones, especially where these need a new way of thinking. For experts, too, awareness of past success can create a block to envisaging new solutions.

4. Experts often share a common academic background, which can bias their take on a problem, and often respond to each other either for validation or differentiation, both of which remove the critical element of independence.

5. The smartness of an organization has a relationship to the total smartness of all the people in it – the sheer weight of numbers of people thinking about a problem makes a real difference to the likelihood of successful analysis.

6. Restricting analysis of a problem to a small number of people leads to the danger of consensus for political reasons – either fair of disagreeing with the boss, or trying to pre-agree, i.e. to predict what the boss will want to hear, or maintaining consensus in the group. These are powerful factors and make it a dangerous business to try to be heterodox or introduce information that does not fit.

That all seems pretty damning, but to date, getting wider feedback in a practical way has been very difficult both technically and politically, and this is perhaps the main reason that we have stuck with the status quo for so long. However, collaboration and innovation technologies make solving these problems a practical proposition, and one that it makes no sense to ignore. But before looking at how such solutions can work, we should briefly look at why they work.

The organization is smarter than any of its subsets

That is the main single point that Surowiecki has to make, and he has compiled a lot of convincing research in social psychology to make the case. In experiment after experiment, a diverse group of people who are not expert has more predictive ability than any single expert or of a smaller group with more expertise. The phenomenon has long been known. Over a century ago, Francis Galton compared ordinary people’s guesses of the weight of an ox (in a competition in a country fair) to those of experts (e.g. cattle breeders). Galton was a snob, to put it bluntly, and wanted to prove how defective the judgment of the ordinary person was. The averaged guess of the crowd was astonishingly close, however, to the real weight, and much better than that of the experts. This phenomenon has been seen repeatedly since in experiments pitching large diverse groups of people against small groups of experts to predict election results, share prices and first-week box office takings.

Target practice: amateurs can miss by more, but their combined score hits the spot

In many cases, the accuracy of predictions made by a large, assorted group of people is almost eery, as though there is some paranormal power at work. More prosaically, though, it is easier to think of it as like throwing a large number darts at a target. An unskilled group will generally miss by quite a bit, but not all in the same way, whereas star players may well share strategies and training, hence make similar kinds of errors. Applying the idea to more information-related themes, one could say that a large, diverse group of people have all kinds of different ignorance, and make all kinds of errors, compared with a more homogenous group.

What is essential for groups to make accurate judgments:
1. Diversity. Groups work well when the types of people and the information sources they use are diverse. Larger numbers of participants offer more diversity and less risk of “groupthink”. Small groups, especially where members share common interests, are likely to polarize towards a consensus, rather than risk a split.
2. Independence. Crowd sourcing works best when participants are not responding to each others’ opinions – or trying to predict them. This is perhaps the biggest single source of problems in decision making. One effect is to predict what the most important or prestigious group member might think, meaning that the focus is on second-guessing, rather than on the real problem. A second effect is cascading, when one person’s judgment sways that of others, independent of their own ideas. Nearly all forms of open discussion cause positive feedback effects or distortion
3. Decentralization. When questions of loyalty, prestige or career advancement come in, the effects of distortion and feedback are serious. These distortions are particularly strong when information flow is strongly influenced by, and reinforces, hierarchies.
3. Selection. There has to be a way to actually get the information from the people who have it, to the people who want it, that does not create bias or feedback. It is easy enough to talk about open businesses, but it is a model that is extremely rare in practice, because it is part of a manager’s job to stay in control, and without the sense of control it is hard to manage! This is perhaps the place where collaboration and innovation software play their greatest part, enabling decision makers to retain control, but gain a wide enough variety of input.

The power of social technologies to improve analysis and idea selection

One thing that sets The Wisdom of Crowds apart from titles that have caused a storm and then disappeared, is that the thesis does not depend on the internet for its validity. The research cited spans many decades. However, social technologies are the great enabler. Companies need a strong sense of leadership, and people who have demonstrated good judgment, effort and responsibility are rightly chosen to lead. However, that does not mean they have to have all the ideas. What they need is a way of selecting and aggregating the best of a diverse input of ideas from all the brains of the company – and of course that includes their own ideas, too. What employees need is a simple, enjoyable way to contribute those ideas, and one that establishes their worth as idea generators for the company. This is not a utopian vision, but a simply reality, and one that is increasingly being espoused by companies that dare to think positively.

Ralf HallerRalf Haller October 14, 2010

Why small decision-making groups in business fail so often

In today’s business world small groups of decision makers typically decide on what will be done. This is far away from any broadly organized decision making process and noone seems to have a problem with this tyipcally as long as the group making the decisions does so in a highly competent way using expert inputs when needed. And of course as long as things work out well.
Unfortunately as numerous scientific researchers have shown, small groups are by nature not making good decisions at all and that has a multitude of reasons. Question becomes then of course how can small groups maybe still made to work?
Some of the reasons why small groups might not work:
  • they have an identity of its own even if they are formed ad hoc for a single project
  • the influence of the people in the group on each other’s judgement is inescapable
  • “confirmation bias”: this s what psychologists mean when decision makers seek those bits of information that confirm their underlying intuitions
  • small teams also tend to believe much more than they really do and reinforce this mis-judgement even further making more extreme decisions e.g. more risky or very risk-averse
  • verdict-based juries (vs. evidence-based) see their mission as reaching a decision as quickly and decisively as possible, votes are taken before discussions even
  • small groups tend to start the discussion with its quick conclusion already. as a result every new information that is coming in was reinterpreted to fit that conclusion. this is a very common problem for small groups who have as a consequence a hard time incorporating new ideas and information
  • social psychologist Garold Strasser has also shown that in unstructured, free-flowing discussions, the information that tends to be talked about the most is, paradoxically, the information that everyone already knows
  • the order of when someone presents is very critical as well since it has a clear impact on how the group members frame and judge subsequent talks
There are even more shortcomings in small group decision making but I think the above list (taken from the book The Wisdom of Crowds, by James Surowiecki) is convincing enough.
So what can be done to still make small groups work and with that improve the decision making of so many companies and organizations?
  • have a “devils advocate” role assigned whose job it is to find critique for anything that is being discussed and suggested by others making sure that both the presenters and the others critically question things.
  • one of the consistent findings from decades of small-group research is that group deliberations are more successful when they have a clear agenda and when leaders take an active role in making sure that everyone gets a chance to speak
  • use software tools that allow to vote – anonymously – subjects under discussion, that also help both prepare a meeting as well as discuss further later on when new ideas and information comes to mind, in addition it archives the discussion so one can revisit it at a later time without having to start totally from scratch. also important is the fact that time pressure to make a final go/no-go decision in a meeting is reduced. and lastly others outside the group could be invited to comment, vote and also provide own ideas and opinions.
If groups are made to work they are genuinely smarter though than the smartest people within them so it is the right thing to aim for. (other research work: Cass Sunstein in “Why Societies Need Dissent”, Alan S. Blinder and John Morgan did studies on small-group performances which was modeled later on by a Bank of England study)
One additional comment from James Surowiecki’s book:
if an organization sets up teams and then uses them purely for advisory purposes, it loses the true advantage that a team has: namely, collective wisdom.