Book review: When the many are wiser than the few

"No one in this world, so far as I know, has ever lost money by underestimating the intelligence of the great masses of the plain people," the American newspaperman H.L. Mencken once infamously remarked.

Mencken was wrong, concludes James Surowiecki in The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations. In fact, under most circumstances, crowds make better decisions than the individuals comprising them.

This isn't exactly a revolutionary concept. It's the theory behind the free market forces that Adam Smith wrote about over two centuries ago. It's also the thinking that undergirds American democracy. "State a moral case to a ploughman and a professor," wrote Thomas Jefferson in 1787. "The former will decide it as well and often better than the latter because he has not been led astray by artificial rules."

Yet it's a concept that still could benefit from a robust defense, and Surowiecki delivers one.

"Ask a hundred people to answer a question or solve a problem," he writes, "and the average answer will often be at least as good as the answer of the smartest member. With most things, the average is mediocrity. With decision-making, it's often excellence. You could say it's as if we've been programmed to be collectively smart."

The first story that Surowiecki relates aptly illustrates this concept. In 1906, English scientist Francis Galton performed an experiment that he hoped would illustrate the stupidity of crowds (and by extension, the folly of democratic government). At a rural livestock fair, those in attendance were given the opportunity to guess the weight of an ox that was about to be slaughtered. Almost 800 villagers gave an individual estimate, but none guessed the exact weight of 1,198 pounds.

Collectively, however, they nearly did - the group average was 1,197.

Surowiecki sees a modern day equivalent of this phenomenon in the very popular (and very profitable) Zagat restaurant guide. Back in 1980, Nina and Tim Zagat were dismayed by the consistently poor culinary advice they felt they were getting from many professional critics. They asked some of their friends to fill out a two-page survey rating the quality of food, service, price, and atmosphere of restaurants, and to add any general comments.

Fast forward 25 years. Zagat now has 250,000 voluntary surveyors, evaluating everything from restaurants to movies all over the world. And consumers apparently find this group wisdom valuable. The New York Restaurant Guide, a Zagat publication, sells over a half-million copies alone each year.

The same phenomenon is at work in a wide array of other situations. It's why the crowd on the game show Who Wants to Be a Millionaire is right over 90 percent of the time when polled by a contestant, and it explains Google's uncanny accuracy (Google's search results are determined by how often a site is linked by others).

Without proper guidelines, though, the wisdom of a crowd can't be tapped. An intriguing example of this lies in professional football. Statisticians have continually proved that in terms of probability, teams would usually be better served by going for it on fourth down rather than punting. The close-knit culture of the football world, however, inspires a much more conservative approach.

Though Surowiecki masterfully frames his ideas in the first half of the book, he stumbles a bit in the second part in applying them to practice. By this time, the avalanche of anecdotes becomes a bit overwhelming, and ultimately distracts from his case. The idea for this book grew out of an article in the The New Yorker (where he is a columnist), and one gets the feeling in converting a column into a book the content got spread a bit thin.

Most of this section is devoted to the market, which Surowiecki clearly sees as a powerful force. This isn't a problem, but his belief that market forces are nearly infallible is. It leads him toward blithely writing off stock market bubbles (like the dot com bust) as unusual examples of "irrational exuberance," rather than more systemic flaws.

Nevertheless, Surowiecki's book is a well-thought out and approachable discussion of a vast and amorphous concept. The reader can easily glean lessons that can be applied to all sorts of situations, from the business world to the football field.

Surowiecki should be thanked for new defense of an old idea. In the mold of Malcolm Gladwell, he shows that confidence in crowds need not be misplaced. If many buy his book, it would seem an appropriate validation of his thesis.