Literature review on collective intelligence: a crowd science perspective

Chao Yu (National Engineering Laboratory for E-commerce Technologies, Tsinghua University, Beijing, China)
Yueting Chai (National Engineering Laboratory for E-commerce Technologies, Tsinghua University, Beijing, China)
Yi Liu (National Engineering Laboratory for E-commerce Technologies, Tsinghua University, Beijing, China)

International Journal of Crowd Science

ISSN: 2398-7294

Article publication date: 11 April 2018

Issue publication date: 10 July 2018




Collective intelligence has drawn many scientists’ attention in many centuries. This paper shows the collective intelligence study process in a perspective of crowd science.


After summarizing the time-order process of related researches, different points of views on collective intelligence’s measurement and their modeling methods were outlined.


The authors show the recent research focusing on collective intelligence optimization. The studies on application of collective intelligence and its future potential are also discussed.


This paper will help researchers in crowd science have a better picture of this highly related frontier interdiscipline.



Yu, C., Chai, Y. and Liu, Y. (2018), "Literature review on collective intelligence: a crowd science perspective", International Journal of Crowd Science, Vol. 2 No. 1, pp. 64-73.



Emerald Publishing Limited

Copyright © 2018, Chao Yu, Yueting Chai and Yi Liu.


Published in the International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at

1. Introduction

A group can be of more power and better wisdom than the sum of the individuals. Foreign scholars have noticed that for a long time and called it collective intelligence. It has emerged from the communication, collaboration, competition and brain storming, etc. Collective intelligence appears in many fields such as public decisions, voting activities, social networks and crowdsourcing.

Crowd science mainly focuses on the basic principles and laws of the intelligent activities of groups under the new interconnection model. It explores how to give full play to the intelligence agents and groups, dig their potential to solve the problems that are difficult for a single agent.

In this paper, we present a literature review on collective intelligence in a crowd science perspective. We focus on researchers’ related work, especially that under which circumstance can group show their wisdom, how to measure it, how to optimize it and its modern or future applications in the digital world. That is exactly what the crowd science pays close attention to.

2. Research process

In the eighteenth century Enlightenment, French politician Condorcet put forward his famous jury theorem stating that if each individual of the decision group will make correct choice than not based on the private information, the whole group’s public voting shows a better result in jury judgement. Although his result is reasonable, the sociological academia showed a conservative point of view on the intelligence of groups. They believed that the crowds know much less than the elites and experts, so it would be better if you had never let them make decisions.

Due to the working class movement and intense social conflict in the nineteenth century, more scholars tended to hold the view that the group’s judgments are destined to be extreme, more than a single individual judgment error (Mackay, 1841). Gustave Le Bon used the French Revolution as a background to think about the relationship between individuals and groups (Bon, 2011). Through the analysis of the various behaviors in the revolution, he found that even if individuals have their own independent views, once they join some confused groups, they become a member of the mob, they are parts of a chemical reaction and lead themselves into a group of crazy.

As time goes by, the study of group wisdom has attracted more and more interest. Galton (1907) published an article in Nature proving that groups could, in some cases, surpass the wisdom of the experts. This is the first positive biological research result of group wisdom, which marks the time that group wisdom is widely scientifically recognized. As an experiment, he collected 787 effective data of the group predicting the weight of cattle and found that the median population performed much better than the experts. Entomologist Wheeler (1924) published his observation that a group of ants act so well in their routine that they cooperate with each other in a strict order just like there were only one organism. After almost one century, some researchers from Harvard found that in a group of hives, the average performance of group in decision-making, when the group comes together, scores higher than those of single ones (Seely, 1995).

In the middle of the twentieth century, Muller (1970) proved that the wisdom of the group was more stable than the individual in terms of stability and ability in the choice of education for their next generation. Almost at the same time, a German scientist pointed out the definition of collective intelligence as a group of individuals’ ability of having reasonable thinking under a specific aim or purpose and efficient handling of their surroundings (Wechsler, 1964).

Heylighen (1999) extended group wisdom to modern management, demonstrating the rationality of group decision-making with many cases and mathematical statistics.

Conradt and Roper (2003) published a paper in Nature, which provided a mathematical model for biota decision-making, and analyzed the group decision-making model of biological groups with incomplete information and the presence of synchronization information between members. The synchronic cost model compares the cost of information synchronization in a completely democratic and fully authoritarian case, proving that the complete autocracy costs more than the complete democratic information synchronization while information synchronizing.

Surowiecki’s (2005) book The Wisdom of Crowds describes the phenomenon that a group of ordinary people can lead to the more precise answer for many questions. Furthermore, he concluded our collective intelligence into three types: cognition, cooperation and coordination, which is widely accepted by the academia. Next year Malone et al. (2009) summarized the collective intelligence in a similar way.

In 2007, Couzin from Oxford stated that by tapping into social cues, individuals in a group may gain access to higher-order computational capacities that mirror the group’s responses to its environment (Couzin, 2009).

In 2010, Woolley et al. (2010) published an article in Science demonstrating that collective intelligence does not show a significant correlation with the intellectual mean value of the group and the maximum individual value of intelligence, but with the equality of group communication and the group’s social perception.

Next year Lorenz et al. (2011) proved that the diversity of views of the group would decline when the group was fully exchanging information. Which means the influence of group social impact on individual decision-making is related to the problem object of decision-making, while the more difficult the problem is, the greater the impact of group on individual will be. Individuals with stronger self-confidence are often not affected by group when other conditions are consistent.

Smith and Arrow’s article in Science shows that simulation based on internet, group intelligence can produce better results than traditional approaches in terms of economic development forecasts and election results (Smith et al., 2008). And Koriat (2012) published an article in Science by experimenting pairs with five groups of different simple decisions, proving that in the case of keeping the collective members get the same information, the choices with more confidence, rather than the best individual’s choices, are closer to the correct solution.

Prelec’s (2017) article in Nature proves that the unexpected algorithms in group wisdom can obtain the optimal solution among the limited solutions in public decision-making. In the experiment, the researchers asked the people who were interviewed for their views on a question, and what they thought would be the other person’s view of the problem. Then, the algorithm finds the “unexpected pop” answer, that is, the answer more popular than most people predict. In most cases, these choices that exceed the expectations of most people are the right answers.

3. Measure and modeling methods

3.1 Qualititative measure

If we want to know how collective intelligence and crowd science work, we must set up a system to measure this effect. Krause et al. (2010) divides the value of collective intelligence into explicit and implicit values. It points out that group cognition, group collaboration and group cooperation belongs to the explicit value of group wisdom, and there is still a lot of hidden value that is ignored; therefore, the value of group wisdom is difficult to accurately estimate.

Some scientists believe that the value of group wisdom is closely related to the surrounding environment (Katsikopoulos and King, 2010). In a strange environment, the individual tends to be more agile, and the wisdom of the group will arise too. Others like to describe the group as a human, or an agent, and use the way similar to how we measure IQ to measure group wisdom (Szuba, 2001; Fadul, 2009).

3.2 Modeling method

Almost every kind of research focusing on the crowd science or collective intelligence will build its own model. Strogatz’s (2001) article exploring complex networks reviews the dynamics network. He divides the network into two kinds, regular networks and complex networks. Furthermore, complex networks are divided into random networks, small-world networks and self-similar networks. Small world networks and self-similar networks are between rules and random networks.

The regular networks are networks with topological symmetry, and the number of neighbors of any node is the same.

Small-world networks are a kind of special complex network structure in which most of the nodes are not connected to each other, but most of the nodes are connected by a few connections. In the social network, sometimes you think you are far away from the people, in fact, close to you. The small-world network is a mathematical description of this phenomenon.

The self-similarity of a system is that the characteristics of a structure or process are similar from different spatial scales or time scales, or that the local structure of a system or structure is similar to the whole.

The connections of nodes in random networks are randomly set, but the number of connections of most nodes are roughly the same; that is, the distribution of nodes follows the Poisson distribution and has a characteristic average number. Nodes with connections that are much higher or lower than the average are very few. With the increase in number of connections, the probability exponentially declines. So the random network is also known as the exponential network.

The model network is established from the macroscopic perspective, and the nodes and connection rules are defined in the micro perspective. There are two popular methods, the voter method and majority rule method. The voter model assumes that each point on the connection graph has an elector, where the connection indicates that there is some form of interaction between the voters (nodes) (Holley and Liggett, 1975). Any voters on some issues are influenced by the views of neighboring nodes. The voter opinion at any given time can be one of two values marked as 0 or 1. Each step randomly selects some voter nodes and updates the voter’s opinion according to the rule. If J is in the neighbor node set of I, the formula is as in equation (1):

(1) σi(t+1)=rand(σj(t))

The majority rule model characterizes individual decisions and makes full use of the information of the surrounding neighbors (Agur, 1991). Mathematically, it can be shown that when there are odd individuals in the system, the individual’s point of view is finally updated to be held by more individuals. The improved majority rule model uses a random selection of nodes, the probability for the node to change its opinions into else’s is proportional to the total number of this view held by the surroundings, which is closer to reality (Makowiec, 2004).

4. Optimization

There are wisdom and intelligence in a group, but a group would not always act correct and choose the better; the key is to see how to deal with information and make decisions. Thus, many researchers are determined to find how to optimize the collective intelligence.

People have the intuitive feeling that small-scale groups will find it difficult to achieve scientific group decision-making; the ability of larger groups to solve problems will be more stable and reliable. This is proved by scientific research; Gallupe et al. (1992) has designed two concurrent experiments conducted with groups of varying size. There were less person groups in one and more person groups in the other. His team compared the number and quality of unique ideas generated by groups of each size using electronic and non-electronic, verbal brainstorming. Groups used both techniques in a counterbalanced within-group design. The larger groups in both experiments generated more unique ideas and more high-quality ideas, and members were more satisfied when they used electronic brainstorming than when they used verbal brainstorming. There were fewer differences between the two techniques for the smaller groups in each experiment. He interpreted these results as showing that electronic brainstorming reduces the effects of production blocking and evaluation apprehension on group performance, particularly for large groups. Krause et al. (2010, 2011) identified some of the possibilities and limitations of collective intelligence using the response of the public to two types of cognitive problems. His team proposed a simple measure for the quantification of collective information that could form the basis for collective intelligence in study populations for specific tasks. They concluded three main results: that the potential benefits of collective intelligence depend on the type of problem, that individual performance and collective performance can be uncorrelated and that a group of individually high performers can be outcompeted by a same-size group of individually low performers, and that adding diversity to a group can be more beneficial than adding expertise. Their results question the emphasis that societies and organizations can put on individual performance to the detriment of diversity as far as teams are concerned. Nevertheless, it is important to point out that while diversity is a necessary condition for effective collective intelligence, diversity alone is clearly not sufficient. Also, the potential implications of findings for the evolution of group composition and the maintenance of personality diversity in animals were discussed in this article. But the number of members in group does not convey the more the better (Steiner, 1972). Gregg (2010) proposed that the larger groups do not have enough opportunity for changing opinions of members and to strengthen the quality control of group wisdom, we must rationally organize and constrain the size of the group.

Researchers have found the quality of the group’s collective intelligence is related to the motivations of individuals in the group (Thomas and Fink, 1963). Smith (1994) argued that group behavior, group awareness and control are important factors in achieving group wisdom. Internet can be of great benefit to the collective intelligence because of its improvement on the efficiency of information dissemination. Gregg (2010) provides a framework for designing specialized collective intelligence applications and demonstrates the applicability of this framework through the development of a special education collective intelligence prototype. The prototype development and subsequent field trial makes primary contributions to the understanding of the collective intelligence applications.

But collective intelligence could be led the wrong way. After WWII, researchers have become interested in the effect of group pressure upon the modification and distortion of individual judgments (Asch, 1951), that is when individuals found themselves consistently contradicted by the other members of the group in simple perceptual judgments. This investigation of the minority of one in the midst of a unanimous majority indicated a marked movement toward the majority, and extreme individual differences ranging from high independence to herd mentality behavior. Although varying the size of the majority demonstrated that a minimal majority of three was required for the effect, and larger majorities did not increase the effect. Furthermore, Lorenz concluded the negative effect in three different ways. The “social influence effect” diminishes the diversity of the crowd without improvements in its collective error (Lorenz et al., 2011). The “range reduction effect” moves the position of the truth to peripheral regions of the range of estimates so that the crowd becomes less reliable in providing expertise for external observers. The “confidence effect” boosts individuals’ confidence after convergence of their estimates despite lack of improved accuracy. Based on Stile’s research, he proved that most participants within the open source community contribute to a single project, and virtually all developers do not have knowledge of the entire project (Stiles and Cui, 2010). The developers make contributions based on their personal needs, while their contributions collect and emerge as a related collection of useful functionality and it is clear that decentralization has a positive effect on it. So the mechanism of information processing among individuals in the group does matter. Safferstone (1998) proposed the efficient sharing of information and distributed computing as the premise for a highly efficient collective intelligence.

Sunstein (2002), in his paper, found a striking empirical regularity that under certain circumstances the deliberation tends to move groups, and the individuals who compose them, toward a more extreme point in the direction indicated by their own judgments. This general phenomenon, also called group polarization, has many implications for economic, political and legal institutions.

On the optimization of group decision-making, the team of Xi'an Jiaotong University simulated it in 2016 (Xi'an Jiaotong, 2016). This agent-based simulation based on the NK model proved the effects of three different group structures on group performance when the decision problems are not independent: a centralized group in which group members make decision together; a decentralized group in which group is divided into several subgroups; and a temporarily decentralized group which starts out with a decentralized structure and later reintegrates. The results of simulation suggest that reintegrated group is always the best of these three decision models no matter how complex the problem is, how many the groups are or how fast the individual learning rate is. Besides, there is an inverted-U relationship between centralized timing and group performance, which is unaffected by the complexity of the decision problem, the number of subgroups and individual learning rate.

5. Applications

5.1 Cognition

5.1.1 Market prediction.

As the internet can quickly convey a lot of information around the world, the use of collective wisdom to predict the feasibility of stock prices (Kaplan, 2001). The opinion of all investors can be weighed equally, so that the effective application of collective intelligence can be applied: the masses, including a broad spectrum of stock market expertise or many not, can be utilized to more accurately predict the behavior of modern financial markets.

5.1.2 Political prediction.

Arrow et al. (2008) points out that collective intelligence can make better results than traditional ones in terms of economic development forecasts and election results predictions.

5.1.3 Digital crowdsourcing.

An example is Google’s Project Aristotle in 2012, where the effect of collective intelligence on team makeup was examined in hundreds of the company’s R&D teams (Collective intelligence, 2018). Another example is the digital rating on a large scale for a product sold online, whether a real good or just a digital one; there are so many companies that use this method such as Zhihu, Douban and Netflix. Or you can get ideas from the crowd.

5.1.4 Decision support.

Traditional decision support first identifies problems and forms decision objectives, including the establishment of decision-making model, determines the effectiveness of the measurement, then has experts look at the outcome of a variety of quantitative evaluations to have a comprehensive analysis of all aspects of information to make a decision. Now with the collective intelligence, a selected group with variety can execute the experts’ task and give a better advice (Hosio et al., 2016).

5.1.5 Bionic calculation.

The swarm intelligence algorithm is related to the computer algorithm (Bonabeau et al., 1999). Researchers are inspired by the natural biological groups to design a randomized optimization algorithm based on the imitation of animal behavior. This algorithm is mainly based on the group behavior on the basis of the target for the optimal search for an algorithm to carry out certain complex functions. The core component is that a large number of individuals together constitute a group based on mutual cooperation. Therefore, the group intelligence is not under centralized control. So it does a great job when there were lack of overall data or in the case of complex unsupervised. Typical algorithms include ant colony algorithm, artificial fish swarm algorithm, bee colony algorithm and particle swarm algorithm, etc.

5.2 Cooperation

5.2.1 Human-computer interaction.

The interaction between agents is the key to the realization of intelligence. In this context, the internet is a very good platform for information exchange. In 1995, John Smith introduced Computer Supported Cooperative Work to achieve a collaborative approach for collaboration, and to stimulate collective intelligence (Smith, 1994). Groupware refers to the collaborative environment of the computer that helps the group to work together. It mainly involves the information transmission between individuals or groups, the information sharing in the group, the automation and coordination of the business process and the interaction between the people and the process. At present, the research related to human-computer interaction technology mainly includes: the architecture of groupware system, the way of computer support communication and sharing information, the decision-support tools in communication, the application sharing and the synchronization realization method, etc.

5.2.2 Community management.

Coe et al. (2000) first proposed the concept of intelligent community in 2001, advocated the community’s virtual platform and called for people to join in and manage the community autonomously.

5.2.3 UN millennium project.

The project, through the establishment of a network of academics in developing and developed countries, opens up new research and new ideas through collaboration with experts within the United Nations system aiming at a better future for the entire human race (Sachs and Mcarthur, 2005).

5.3 Coordination

5.3.1 Wikipedia.

Wikipedia is an online encyclopedia run by a non-profit organization; it uses an open architecture, with the help of a community co-authoring technique wiki, to attract the user community to massively produce and update knowledge voluntarily. The sharing of information is the driving force behind the evolution of Wikipedia. Wikipedia attracts many users because of the ease of editing, the objective and impartiality of the position and the open architecture. It has over 20 billion pages and over 500 million active visitors monthly now (Aaltonen and Seiler, 2016). Wikipedia’s information is organized. Different from the traditional model which achieves the order of information through the external organization, Wikipedia organizes information through the user’s collaborative sharing, self-organization. Thus, the organized state of Wikipedia is a dynamic equilibrium state. Wikipedia’s content is a mixture of all thought and is the common result of many people actively involved. Any person in any place as long as the internet is connected can unconditionally get the knowledge. Wikipedia has adopted an open collaboration model, so that individuals can share their own knowledge, and get access to all the intelligence made by the group.

5.3.2 Open innovation.

One example of open innovation is in software innovation (West and Gallagher, 2006). Open source software is software with the source code open to the public. Open source software and its code can be freely downloaded in its corresponding open source community, and welcome the free participation of the community to the development. Some of it will allow commercial organizations to re-develop and in accordance with the corresponding open source agreement to publish. Dr Jan Marco Leimeister quoted Chesbrough’s definition on open innovation in his paper that open innovation refers to the opening of companies’ innovation processed by actively integrating the environment into these activities, and thus, extending its innovation capabilities for developing new products and services for wider application (Leimeister, 2010).

6. Future potential

In public administration area, Brabham (2009) recommends that the relevant proposals for urban planning be entrusted to the public in the form of crowdsourcing, which are involved in the development and supervision of public projects through some government websites.

The internet innovation has promoted the social networking website and e-commerce. Through researching in a wide range of social networking services, from small social media networks to more complex blog communities and fully integrated social business platforms, it can concluded that successful social applications must focus on improving the user’s experience, which can be linked to e-commerce. For example, social shopping will greatly enhance customers’ satisfaction. In the future, considering the huge demand for personalizing, the e-commerce retail and social networks will be linked by collective intelligence tagging algorithm (Chiu et al., 2014; Huang and Shiu, 2012).

As for the study on the mechanism of collective intelligence, the existing research results affirm the positive role of group wisdom in guiding the practice of human society and have given the technical support and platform for the realization and optimization of group wisdom. But there is still a lot of room to explore. For example, what mechanisms are used to ensure that the members of the group are diverse and independent, and how decentralized an efficient organization should be need us continue to focus on.

Through the new technology such as data mining, bionic calculating, cloud computing, all of us are now in brand new networks. With further research on crowd science and collective intelligence, we can obtain a better view of how to give full play to the human or agent individual/group intelligence, dig their potential, to solve the problems which are difficult for computer or human alone now.


Aaltonen, A. and Seiler, S. (2016), “Cumulative growth in user-generated content production:: Evidence from Wikipedia”, Management Science, Vol. 62 No. 7, pp. 2054-2069.

Agur, Z. (1991), “Fixed points of majority rule cellular automata with application to plasticity and precision of the immune system”, Complex Systems, Vol. 5 No. 3.

Arrow, K.J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., Ledyard, J.O., Levmore, S., Litan, R., Milgrom, P., Nelson, F.D., Neumann, G.R., Ottaviani, M., Schelling, T.C., Shiller, R.J., Smith, V.L., Snowberg, E., Sunstein, C.R., Tetlock, P.C., Tetlock, P.E., Varian, H.R., Wolfers, J. and Zitzewitz, E. (2008), “The promise of prediction markets”, Science, Vol. 320 No. 5878, pp. 877-878.

Asch, S.E. (1951), “Effects of group pressure upon the modification and distortion of judgments”, Groups, Leadership, and Men, pp. 222-236.

Bon, G.L. (2011), The Crowd: A Study of the Popular Mind, Dover Publications, New York, NY.

Bonabeau, E., Dorigo, M. and Theraulaz, G. (1999), Swarm Intelligence: From Natural to Artificial Systems, Oxford University Press, Oxford.

Brabham, D.C. (2009), “Crowdsourcing the public participation process for planning projects”, Planning Theory, Vol. 8 No. 3, pp. 242-262.

Chiu, C.M., Liang, T.P. and Turban, E. (2014), “What can crowdsourcing do for decision support?”, Decision Support Systems, Vol. 65 No. 3, pp. 40-49.

Coe, A., Paquet, G. and Roy, J. (2000), “E-governance and smart communities: a social learning challenge”, Social Science Computer Review, Vol. 19 No. 1, pp. 80-93.

Collective intelligence (2018), available at:

Conradt, L. and Roper, T.J. (2003), “Group decision-making in animals”, Nature, Vol. 421 No. 6919, pp. 155-158.

Couzin, I.D. (2009), “Collective cognition in animal groups”, Trends in Cognitive Sciences, Vol. 13 No. 1, p. 36

Fadul, J.A. (2009), “Collective learning: applying distributed cognition for collective intelligence”, International Journal of Learning, Vol. 16 No. 4, pp. 211-220.

Gallupe, R.B., Dennis, A.R., Cooper, W.H., Valacich, J.S., Bastianutti, L.M. and Nunamaker, J.F. (1992), “Electronic brainstorming and group size”, Academy of Management Journal, Vol. 35 No. 2, pp. 350-369.

Galton, F. (1907), “Vox Populi”, Nature, Vol. 75 No. 1949, pp. 450-451.

Gregg, D.G. (2010), “Designing for collective intelligence”, Communications of the ACM, Vol. 53 No. 4, pp. 134-138.

Heylighen, F. (1999), “Collective intelligence and its implementation on the web: algorithms to develop a collective mental map”, Computational & Mathematical Organization Theory, Vol. 5 No. 3, pp. 253-280.

Holley, R.A. and Liggett, T.M. (1975), “Ergodic theorems for weakly interacting infinite systems and the voter model”, The Annals of Probability, Vol. 3 No. 4, pp. 643-663.

Hosio, S., Goncalves, J., Anagnostopoulos, T., et al. (2016), “Leveraging wisdom of the crowd for decision support”, British HCI 2016 Conference, Poole.

Huang, S.L. and Shiu, J.H. (2012), “A user-centric adaptive learning system for E-learning 2.0”, Journal of Educational Technology & Society, Vol. 15 No. 3, pp. 214-225.

Kaplan, C.A. (2001), “Collective intelligence: a new approach to stock price forecasting”, IEEE International Conference on Systems, Man, and Cybernetics, IEEE, Tucson, AZ, Vol. 5, 2893-2898.

Katsikopoulos, K.V. and King, A.J. (2010), “Swarm intelligence in animal groups: when can a collective out-perform an expert?”, Plos One, Vol. 5 No. 11, pp. e15505-e15505.

Koriat, A. (2012), “When are two heads better than one and why?”, Science, Vol. 336 No. 6079, pp. 360-362.

Krause, J., Ruxton, G.D. and Krause, S. (2010), “Swarm intelligence in animals and humans”, Trends in Ecology & Evolution, Vol. 25 No. 1, pp. 28-34.

Krause, S., James, R., Faria, J.J., Ruxton, G.D. and Krause, J. (2011), “Swarm intelligence in humans: diversity can trump ability”, Animal Behaviour, Vol. 81 No. 5, pp. 941-948.

Leimeister, J.M. (2010), “Collective intelligence”, Business & Information Systems Engineering, Vol. 2 No. 4, pp. 245-248.

Lorenz, J., Rauhut, H., Schweitzer, F. and Helbing, D. (2011), “How social influence can undermine the wisdom of crowd effect”, Proceedings of the National Academy of Sciences of the United States of America, Vol. 108 No. 22, pp. 9020-9025.

Lorenz, J., Rauhut, H., Schweitzer, F. and Helbing, D. (2011), “How social influence can undermine the wisdom of crowd effect”, Proceedings of the National Academy of Sciences, Vol. 108 No. 22, pp. 9020-9025.

Mackay, C. (1841), Extraordinary Popular Delusions and the Madness of Crowds, Farrar, Straus, New York, NY.

Makowiec, D. (2004), Cellular Automata with Majority Rule on Evolving Network, Springer Berlin Heidelberg, Heidelberg, pp. 141-150.

Malone, T.W., Laubacher, R. and Dellarocas, C. (2009), “Harnessing crowds: mapping the genome of collective intelligence”, Social Science Electronic Publishing.

Muller, P. (1970), “Collective intelligence tests and educational selection Neuchatel experiences 12 years later”, Psychologie, Vol. 29 Nos 1/2, pp. 194-202.

Prelec, D. (2017), “A solution to the single-question crowd wisdom problem”, Nature, Vol. 541 No. 7638, pp. 532-535.

Sachs, J.D. and Mcarthur, J.W. (2005), “The millennium project: a plan for meeting the millennium development goals”, Lancet, Vol. 365 No. 9456, pp. 347-353.

Safferstone, M.J. (1998), “The distributed mind: achieving high performance through the collective intelligence of knowledge work teams”, Academy of Management Executive, Vol. 12 No. 2, pp.107-108.

Seely, T.D. (1995), The Wisdom of the Hive, Harvard University Press, MA, pp. 20-35

Smith, J.B. (1994), Collective Intelligence in Computer-Based Collaboration, CRC Press, Boca Raton.

Smith, V.L., Arrow, K., Forsythe, R., et al. (2008), “The promise of prediction markets”, Science, Vol. 320 No. 5878, pp. 877-878.

Steiner, I.D. (1972), “Group process and productivity”, Computing Science & Statistics.

Stiles, E. and Cui, X. (2010), “Workings of collective intelligence within open source communities”, International Conference on Social Computing, Behavioral Modeling, and Prediction, Springer Berlin Heidelberg, Heidelberg, pp. 282-289.

Strogatz, S.H. (2001), “Exploring complex networks”, Nature, Vol. 410 No. 6825, pp. 268-276.

Sunstein, C.R. (2002), “The law of group polarization”, Journal of Political Philosophy, Vol. 10 No. 2, pp. 175-195.

Surowiecki, J. (2005), The Wisdom of Crowds, Anchor.

Szuba, T. (2001), “A formal definition of the phenomenon of collective intelligence and its IQ measure”, Future Generation Computer Systems, Vol. 17 No. 4, pp. 489-500.

Thomas, E.J. and Fink, C.F. (1963), “Effects of group size”, Psychological Bulletin, Vol. 60 No. 4, pp. 371-384.

Wechsler, D. (1964), Die Messungder Intelligenz Erwachsener, Bern-Stuttgart, Huber, pp. 20-50.

West, J. and Gallagher, S. (2006), “Challenges of open innovation: the paradox of firm investment in open-source software”, R&D Management, Vol. 36 No. 3, pp. 319-331.

Wheeler, W.M. (1924), “Social life among the insects”, Isis, 15, Vol. 6, No. 4, pp. 119-131.

Woolley, A.W., Chabris, C.F., Pentland, A., Hashmi, N. and Malone, T.W. (2010), “Evidence for a collective intelligence factor in the performance of human groups”, Science, Vol. 330 No. 6004, pp. 686-688.

Xi'an Jiaotong (2016), “Effects of centralized, decentralized and reintegration group structures on group performance”, Operation Research and Management, Vol. 25 No. 5, pp. 6-14.

Corresponding author

Yueting Chai can be contacted at:

About the authors

Chao Yu, the first author, received his Bachelor of Engineering in Tsinghua University in 2016 and is now a PhD Candidate in National Engineering Laboratory for E-Commerce Technologies, Tsinghua University. His research interests are crowd science theory and internet economy.

Prof. Yueting Chai is the Laboratory Chief of the National Engineering Laboratory for E-Commerce Technologies, Tsinghua University. His research interests are the crowd science and e-commerce industry.

Associate Professor Yi Liu, the third author, is now an Associate Research Fellow in both National CIMS Engineering Research Center and National Engineering Laboratory for E-Commerce Technologies, Tsinghua University. Her research interests are information system and e-commerce.

Related articles