On October 15th, Mike Perry of Inkling Books in a comment recommended a book to Groklaw readers called "The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations". The book is by James Surowiecki, who writes The Financial Page for The New Yorker magazine.
Perry asked if any Groklaw readers would write a review, and one reader, Nick, accepted the challenge. Here, then, is a his review of "The Wisdom of Crowds":
We all know that crowds are dumb, right? That's where riots come from. Lowest common denominator, and all that. So why did Mr. Surowiecki call his book "The Wisdom of Crowds" (and yes, it is a direct allusion to Charles Mackay's "Extraordinary Popular Delusions & The Madness of Crowds")? Simply put, while not denying the madness of crowd model in the right circumstances, it turns out that under other circumstances the crowd is wise. Very wise. In fact, wiser, more insightful, more accurate, more intelligent than any individual genius can hope to be. That's a strong statement, but Surowiecki backs it up with example after example to show what he means and why.
For example, note this quote:
"A classic demonstration of group intelligence is the jelly-beans-in-the-jar experiment, in which invariably the group's estimate is superior to the vast majority of the individual guesses. When finance professor Jack Treynor ran the experiment in his class with a jar that held 850 beans, the group estimate was 871. Only one of the fifty-six people in the class made a better guess."
This is only one example of many in the book. No matter how knowledgeable the individual observer, a group estimate, even a group composed on non-experts, routinely trumps the individual in insight. Whether it involves counting jelly beans, estimating the weight of an ox, or assigning blame in the stock market to the correct company responsible for the Challenger accident in 1986, the crowd gets it right faster and more accurately than the individual expert. Note this other fascinating example:
"In May 1968, the U.S. submarine Scorpion disappeared on its way back to Newport News after a tour of duty in the North Atlantic. Although the navy knew the sub's last reported location, it had no idea what had happened to the Scorpion, and only the vaguest sense of how far it might have traveled after it had last made radio contact. As a result, the areas where the navy began searching for the Scorpion was a circle twenty miles wide and many thousands of feet deep. You could not imagine a more hopeless task. The only possible solution, one might have thought, was to track down three or four top experts on submarines and ocean currents, ask them where they thought the Scorpion was, and search there. But...a naval officer named John Craven had a different plan.
"First, Craven concocted a series of scenarios -- alternative explanations for what might have happened to the Scorpion. Then he assembled a team of men with a wide range of knowledge, including mathematicians, submarine specialists, and salvage men. Instead of asking them to consult with each other to come up with an answer, he asked each of them to offer his best guess about how likely each of the scenarios was..Craven believed that if he put all the answers together, building a composite picture of how the Scorpion died, he'd end up with a pretty good idea of where it was...He took all the guesses, and used a formula called Bayes's theorem to estimate the Scorpion's final location..When he was done, Craven had what was, roughly speaking, the group's collective estimate of where the submarine was.
"The location that Craven came up with was not a spot that any individual member of the group had picked. In other words, not one of the members of the group had a picture in his head that matched the one Craven had constructed using the information gathered from all of them. The final estimate was a genuinely collective judgment that the group as a whole had made, as opposed to representing the individual judgment of the smartest people in it. It was also a genuinely brilliant judgment. Five months after the Scorpion disappeared, a navy ship found it. It was 220 yards from where Craven's group said it would be."
Remarkable, right? Is it just in humans that we see this sort of behavior? No. Consider how bees find good sources of nectar:
"They don't sit around and have a collective discussion about where foragers should go. Instead, the hives sends out a host of scout bees to search the surrounding area. When a scout bee has found a nectar source that seems strong, he comes back and does a waggle dance, the intensity of which is shaped, in some way, by the excellence of the nectar supply at the site. The waggle dance attracts other forager bees, which follow the first forager, while foragers who have found less-good sites attract fewer followers and, in some cases, eventually abandon their sites entirely. The result is that bee foragers end up distributing themselves across different nectar sources in an almost perfect fashion, meaning that they get as much food as possible relative to the time and energy they put into searching. It is a collectively brilliant solution to the colony's food problem.
"What's important, though, is the way the colony gets to that collectively intelligent solution. It does not get there by first rationally considering all the alternatives, and then determining an ideal foraging pattern. It can't do this, because it doesn't have any idea what the possible alternatives -- that is, where the different flower patches -- are. So instead, it sends out scouts in many different directions and trusts that at least one of them will find the best patch, return, and do a good dance so that the hive will know where the food source is."
Now we begin to see the secret to this group wisdom effect. The more people involved (or the more bees), the greater the input from the group as a whole and the more likely it is that the correct solution is reached. That makes intuitive sense, for we all know that "two heads are better than one." So that means instead of relying on one expert, get a group of experts together, right? Wrong:
". . . [A] group made up of some smart agents and some not-so-smart agents almost always did better than a group made up just of smart agents. Diversity is, on its own, valuable, so that the simple fact of making a group diverse make it better at problem solving. That doesn't mean intelligence is irrelevant.. but it does mean that, on the group level, intelligence alone is not enough, because intelligence alone cannot guarantee you different perspectives on a problem.. Adding in a few people who know less, but have different skills, actually improves the group's performance."
OK, now we're getting radical. A group of experts and non-experts is better than just a group of experts, even if the group size is the same? Surowiecki knows what you are thinking at this point and addresses it:
"Again, this doesn't mean that well-informed, sophisticated analysts are of no use in making good decisions. (And it certainly doesn't mean you want crowds of amateurs trying to collectively perform surgery or fly planes.) It does mean that however well-informed and sophisticated an expert is, his advice and predictions should be pooled with those of others to get the most out of him. (The larger the group, the more reliable its judgment will be.) And it means that attempting to 'chase the expert,' looking for the one man who will have the answers to an organization's problem, is a waste of time."
So don't worry, he's not deprecating intelligence or expertise, and he acknowledges there are obvious times when you do want the lone expert working on your problem, especially if "your problem" is you need brain surgery. And Surowiecki absolutely acknowledges the problems that can come from relying on the crowd to achieve wisdom. But the principle upon which this book rests is expressed simply thus:
"The idea of the wisdom of crowds is not that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide."
It's that group experience that makes the difference. Expertise is needed, but relying on expertise alone will leave you worse off than if you couple expertise with diversity. Does that sound familiar? It should. It's the Linux model for developing software, and it's the Groklaw model for gathering legal news and insight. The more diverse the crowd, the greater the chance that one of those waggling bees will stumble upon the right answer, or the best answer. It works when looking for nectar, and it works when submitting bug fixes and new features for Linux. Notice what Surowiecki says about Linux:
"In the way it operates, in fact, Linux is not all that different from a market.. Like a bee colony, it sends out lots of foragers and assumes that one of them will find the best route to the flower fields. This is, without a doubt, less efficient than simply trying to define the best route to the field or even picking the smartest forager and letting him go. After all, if hundreds or thousands of programmers are spending their time trying to come up with a solution that only a few of them are going to find, that's many hours wasted that could be spent doing something else. And yet, just as the free market's ability to generate lots of alternatives and then winnow them down is central to its continued growth, Linux's seeming wastefulness is a kind of strength (a kind of strength that for-profit companies cannot, fortunately or unfortunately, rely on). You can let a thousand flowers bloom and then pick the one that smells the sweetest.
"So who picks the sweetest-smelling one? Ideally the crowd would. But here's where striking a balance between the local and the global is essential: a decentralized system can only produce genuinely intelligent results if there's a means of aggregating the information of everyone in the system. Without such a means, there's no reason to think that decentralization will produce a smart result. In the case of Linux, it is the small number of coders, including Torvalds himself, who vet every potential change to the operating-system source code. There are would-be Linux programmers all over the world, but eventually all roads lead to Linus."
So we see that wisdom from crowds comes as a result of certain conditions. There are principles by which wisdom can come from the crowd (as opposed to madness):
". . . the four conditions that characterize wise crowds: diversity of opinion (each person should have some private information, even if it's just an eccentric interpretation of the known fact), independence (people's opinions are not determined by the opinions of those around them), decentralization (people are able to specialize and draw on local knowledge), and aggregation (some mechanism exists for turning private judgments into a collective decision). If a group satisfies those conditions, its judgment is likely to be accurate."
What about the opposite result, the one where crowds are not wise and even dumb? Under what circumstances do crowds go wrong and start to riot (or in the case of online communities, start to turn on the community)? Surowiecki discusses this as well, offering this analysis of what causes a crowd to riot:
"The process by which a violent mob actually comes together seems curiously similar to the way a stock-market bubble works. A mob in the middle of a riot appears to be a single organism, acting with one mind. And obviously the mob's behavior has a collective dimension that a group of random people just milling about does not have. But sociologist Mark Granovetter argued that the collective nature of a mob was the product of a complicated process, rather than a sudden descent into madness. In any crowd of people, Granovetter showed, there are some people who will never riot, and some people who are ready to riot at almost any time -- these are the 'instigators.' But most people are somewhere in the middle. Their willingness to riot depends on what other people in the crowd are doing. Specifically, it depends on how many other people in the crowd are rioting. As more people riot, more people decide that they are willing to riot, too...
"This makes it sound as if once one person starts a ruckus, a riot will inevitably result. But according to Granovetter, that's not the case. What determines the outcome is the mix of people in the crowd. If there are a few instigators and lots of people who will act only if a sizable percentage of the crowd acts, then it's likely nothing will happen. For a crowd to explode, you need instigators, 'radicals' -- people with low thresholds for violence -- and a mass of people who can be swayed. The result is that although it's not necessarily easy to start a riot, once a crowd crosses the threshold into violence, its behavior is shaped by its most violent members. If the image of collective wisdom that informs much of this book is the average judgment of the group as a whole, a mob is not wise. Its judgment is extreme...
"There is, in Granobetter's work, a hint as to what markets need to avoid endless bouts with irrational exuberance or irrational despair. In Granovetter's world, if there are enough people in the crowd who will not riot under any conditions -- that is, whose actions are independent of the crowd's behavior as a whole -- then a riot will be far less likely, because the more people who do not riot, the more people there will be who don't want to riot."
So there we have it, both the pattern for useful behavior from crowds, as well as warnings on how to avoid being sucked into negative mob behavior. When utilized properly, the wisdom of crowds is real. Surowiecki writes interestingly too about the financial markets and political thinking. But for Groklaw readers the application is direct and obvious. We've seen the wisdom of the crowds in the growth of Linux, and also in the growth of Groklaw. Both models follow principles that allow wisdom to bubble up from the group, with, as Surowiecki puts it, someone to pick the "sweetest flower". When you let a thousand flowers bloom, it's easy to pick the one that smells the sweetest. "The Wisdom of Crowds" is very interesting, and I highly recommend it.