Friday 6 July 2012

The Wisdom of Crowds:


Collective Intelligence - Sharing Imagination & Knowledge
A reference resource for the many and varied methods of tapping and using the intellectual wealth and wisdom of many individuals, a work in progress. Suggestions from the crowd welcomed 29.08.12


 This term  was brought into popular usage by staff writer on the New Yorker, James Surowiecki, in his book The Wisdom of Crowds  "Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations", published by Doubleday 2004.
The book relates to diverse collections of independently deciding individuals who, under the right conditions, offer solutions more accurate and ingenious than those of experts or hierarchies.

 This approach captures diverse individual sparks of brilliance and nurtures an organised collective intelligence as opposed to crowd psychology as traditionally understood in riots and stock exchange bubbles. [1]

The much quoted experiment in collective intelligence by Francis Galton, on a crowd accurately judging the weight of an ox, has reportedly been replicated in the 20th Century with different tasks and in various ways, with good success. [2] 

Surowiecki quotes many other ways the wisdom of crowds has been used and makes the point that there needs to be both a clear direction of the process and a mechanism for interpreting collective judgments and individual solutions. He makes controversial statements and has been challenged on questions of mean or median but the potential for collective intelligence is more that judging the weight of an ox or the number of jelly beans in a jar. [3]

The book refreshes old ideas in modern settings and opens up the possibilities for an entirely different  approach to exploring issues of fundamental importance to any notion of "accountability" for our public broadcasting service.

There are many facets to the "wisdom of crowds", from the original Athenian Greek city democracy, where only a minority had voting rights, which were participatory responsibilities, through to the Linux computer operating system, "Who Wants to be a Millionaire?" the modern airline protocol of "Cockpit Resource Management", "brainstorming" exercises and "crowd sourcing" models. [4]

The British jury system is one example of the trust we put in random selection from a defined group - jury service by being a registered elector.

Google functions so well because, at present, the system relies on the wisdom of millions of web site owners who link to pages of their choice. Their favourites allow Google to target the most relevant response to searches.

Linux, the open source operating system, is constantly updated by thousands of individual supporters. Their individual efforts are scrutinised by a core team that includes Linus Torvalds, who created Linux. A selection of the aggregated wisdom of autonomous programers constantly updates and de-bugs the system that challenges Microsoft, the world leader. "Given enough eyeballs, all bugs are shallow" says open source guru Eric Raymond.

In May 1968, the U.S. submarine Scorpion disappeared on its way back to Virginia after a tour of duty in the North Atlantic. It was lost in an area so big it was considered impossible to locate the vessel.

However, as Sherry Sontag and Chris Drew recount in their book "Blind Man"s Bluff", a naval officer named John Craven devised an form of collective intelligence to create a composite picture and used this together with an intriguing formula called Bayes’s theorem to estimate the Scorpion’s final location. Five months after the Scorpion disappeared, a navy ship found the Scorpion. It was 220 yards from where Craven’s group had said it would be. [5]

Will Hutton has argued that Surowiecki's analysis applies to value judgments as well as factual issues, with crowd decisions that "emerge of our own aggregated free will [being] astonishingly... decent".

He concludes that "There's no better case for pluralism, diversity and democracy, along with a genuinely independent press." [He could have added TV]
Guidelines

Surowiecki considers four criteria essential for maximising the potential of collective intelligence:

Diversity of opinion
Each person will have private information even if it's just an eccentric interpretation of the known facts.

Independence
People's opinions aren't determined by the opinions of those around them.

Decentralization
People are autonomous of central control and are able to specialize and draw on local knowledge.

Aggregation
Some mechanism exists for turning private judgments into a collective decision.

BBC license holders belong to our multi-class multi-cultural multi-ethnic society.

 In the quest for a genuinely representative accountability and control of the BBC, ingenuity and innovation is required in how groups of people can form networks of trust without a central system controlling their behaviour or directly enforcing their compliance.

 This kind of autonomous cooperation, is one of the important elements in Surowieki's thesis. In pursuit of this end the ancient Athenians employed the practice of casting lots and rotation to get the best from the "crowd" whilst seeking to ensure no sectional interest dominated. 

The BBC itself acknowledges the centrality of the relationship between the Corporation and the public and runs audience engagement programmes. These are promised to be routes to "make sure [audiences] are at the heart of decision-making in the future". [6]

However, those who set the questions or select comments and ideas can do so to further their own agenda, even if done unconsciously. Nevertheless responses to one survey, asking open ended opinion questions, returned the aggregated comments: "be clear and transparent" and "not just one way" as principles for audience engagement.

One of the fundamentals of meaningfully tapping the wealth of insight of the many is that the process must not be just one way  regarding the power of setting the agenda, framing the questions, selecting and measuring the results. Consultation without those consulted having power is virtually meaningless.

The original Greek City states, where democracy evolved, had every citizen earn the right both to debate and vote. With that right  the duty of service in the defence of the city state, and as a representative and administrator for the city government, was accepted. See C.L.R James "Every Cook Can Govern" [7]

Major decisions were taken by direct democracy, participatory and deliberative. The public will was administered by citizens called to duty by complex systems of drawing lots. Rotation of powerful positions was strictly regulated with transparency, accountability and recall central to the systems.  "The essence of the Greek method, here as elsewhere, was the refusal to hand over these things to experts, but to trust to the intelligence and sense of justice of the population at large, which meant of course a majority of the common people." C.L.R. James. 

This is the form of government under which, many argue, flourished the greatest civilization the world has ever known. 

It was a brief experiment with genuine democracy developed between 600 and 322 BC. The are many interpretations of the period but the outline is generally agreed. [8]

References:

[1]
Terms such as collective intelligence and wisdom of crowds have diverse meanings. The Massachusetts Institute of Technology hosts a information resource, the Center for Collective Intelligence which offers a starting point to the subject.
Negative aspects of crowds:

[2]
Francis Galton, ox experiment: 
Summary of Dutch attempt to replicate result:

[3]
Excerpts from Wisdom of Crowds, Chapter One.  Jelly bean experiment and more.

[8]

[9] 
The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations, published in 2004, is a book written by James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group. The book presents numerous case studies and anecdotes to illustrate its argument, and touches on several fields, primarily economics and psychology.
The opening anecdote relates Francis Galton's surprise that the crowd at a county fair accurately guessed the weight of an ox when their individual guesses were averaged (the average was closer to the ox's true butchered weight than the estimates of most crowd members, and also closer than any of the separate estimates made by cattle experts).[1]
The book relates to diverse collections of independently-deciding individuals, rather than crowd psychology as traditionally understood. Its central thesis, that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts, draws many parallels with statistical sampling, but there is little overt discussion of statistics in the book.
Its title is an allusion to Charles Mackay's Extraordinary Popular Delusions and the Madness of Crowds, published in 1841




Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)