The purpose of this paper is to study the participation behaviors in the context of crowdsourcing projects from the perspective of gamification.
This paper first proposed a model to depict the effect of four categories of game elements on three types of motivation based upon several motivation theories, which may, in turn, influence user participation. Then, 5 × 2 between-subject Web experiments were designed for collecting data and validating this model.
Game elements which provide participants with rewards and recognitions or remind participants of the completion progress of their tasks may positively influence the extrinsic motivation, whereas game elements which can help create a fantasy scene may strengthen intrinsic motivation. Besides, recognition-kind and progress-kind game elements may trigger the internalization of extrinsic motivation. In addition, when a task is of high complexity, the effects from game elements on extrinsic motivation and intrinsic motivation will be less prominent, whereas the internalization of extrinsic motivation may benefit from the increase of task complexity.
This study may uncover the motivation mechanism of several different kinds of game elements, which may help to find which game elements are more effective in enhancing engagement and participation in crowdsourcing projects. Besides, as task complexity is used as a moderator, one may be able to identify whether task complexity is able to influence the effects from game elements on motivations. Last, but not the least, this study will indicate the interrelationship between game elements, individual motivation and user participation, which can be adapted by other scholars.
Zeng, Z., Tang, J. and Wang, T. (2017), "Motivation mechanism of gamification in crowdsourcing projects", International Journal of Crowd Science, Vol. 1 No. 1, pp. 71-82. https://doi.org/10.1108/IJCS-12-2016-0001
Emerald Publishing Limited
Copyright © 2017, Zhiyuan Zeng, Jian Tang and Tianmei Wang
Published in the International Journal of Crowd Science. Published by Emerald Publishing. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Gamification refers to the use of gameplay mechanisms in non-gaming contexts to encourage desired behaviors (Werbach and Hunter, 2012). Many scholars and practitioners have applied this approach in several contexts such as education, health, project management, environment protection, outsourcing and software development. (Deterding, 2012). Gamification can benefit corporations and companies by extending the market share, changing the attitude of users and enhancing the motivation effect (Hamari and Koivisto, 2013). The application of game elements helps managers or organizers dismantle the obstacles like the disability to attract users, inadequate participation and low engagement (McGonigal, 2011), and finally, guarantee that projects or activities hosted by those corporations and companies could move forward as expected.
Crowdsourcing is one of the contexts where gamification could be applied, referring to a new outsourcing approach that takes tasks asx∈[a,b] and y∈[c,d], where a < b and c < d.
Suppose that we use m layers, where m∈∥+ an open call to an undefined, large group of people (Howe, 2006). The cooperation and competition traits of crowdsourcing projects allow companies to make full use of knowledge, skills and resources beyond those of their own staff (Neyer et al., 2009; Whitla, 2009), so that they can reduce costs and increase benefits (Howe, 2006).
Crowdsourcing relies on the number of participants who are willing to devote and contribute their time and efforts. Whether crowdsourcing projects could be operated smoothly is highly relevant with the ability of those initiators to attract and obtain adequate participants. Many crowdsourcing projects have failed because they were unable to gain enough attention from the public (McGonigal, 2011). Gamification is considered as an approach to attract participants and improve the engagement. The application of gamification may avoid the situation where a crowdsourcing project may fail owing to the lack of participation. Hence, how to apply gamification appropriately to increase participation and engagement has become a major topic in our research.
Some prior research discussed the application of gamification in the realm of crowdsourcing (Zheng et al., 2011; Mekler et al., 2015; Kavaliova et al., 2016); yet, the extant literature lacks a systematic understanding of motivation mechanism of gamification in crowdsourcing projects. Some scholars who are interested in the application of gamification intended to abstract game elements to provide a more complete structure of gamification (Hunicke et al., 2004; McGonigal, 2011; Robson et al., 2015), but the process or mechanism via which these game elements can have an impact on user participation is barely mentioned in these frameworks; others intended to discuss the motivational influence of some specific game elements in different contexts (Sigala, 2015), but the categories of these game elements having been discussed in these studies are limited. As a result, which kinds of game elements are more effective in increasing participation and engagement in crowdsourcing projects, and whether motivation mechanism of these game elements vary from one kind to another remains uncovered.
To fill these gaps, we intend to investigate the motivation mechanism of gamification in crowdsourcing projects. First, we categorized game elements that have been used widely in crowdsourcing projects. Then, we proposed a research model that depicts the effect of these game elements on human motivation, which may, in turn, influence user participation. In addition, we are interested in identifying the moderating effects of task complexity on user motivation. We designed a series of laboratory experiments to validate our research model. Finally, we discussed expected contributions and implications of our research.
2. Literature review
Game has been considered as the inspiration for designing captivating user interface since 1980 (Deterding, 2012; Malone, 1981). Until now, many scholars and practitioners have applied game dynamics and elements to different contexts to enhance user engagement. The most frequently mentioned and used game elements within these contexts are narratives, fantasy world, point, level, status, badge, leaderboard, challenge, countdown, deadline, etc.
A common definition of gamification embraces the concept of using game elements and design techniques in a non-game setting, often with the end-goal of shaping user behaviors (Kavaliova et al., 2016). Thanks to the high adaptability of gamification, many non-gaming contexts, such as education, health, project management, environment protection, crowdsourcing, software development, have used the idea of gamification (Deterding, 2012). Many corporations and companies have used the concept of gamification and game elements to help them extend the market share, alter the attitude of users and enhance the motivation effect (Hamari and Koivisto, 2013) to maintain high user engagement and obtain user loyalty. However, most initiators cannot meet their expectations because game elements were not implemented appropriately. Hence, how to select proper game elements under varied contexts has gradually gained attention among scholars and practitioners, so is the motivation mechanism of gamification.
Despite the wide application of gamification, a discussion of the inner mechanism of gamification is still open. One of the most frequently used framework of gamification is Hunicke et al.’s (2004) MDA (mechanic–dynamic–esthetics) framework where game elements have been abstracted into three parts: game mechanics, game dynamics and game esthetics. Mechanics refer to the fundamental settings and rules including point, level, trophy, badge, achievement, leaderboard and virtual gift; dynamics refer to the motives which can lead to some particular emotions, such as reward, status, achievement, self-expression, competition and altruism (Simoes et al., 2013); and esthetic, which is substituted sometimes as emotion (Robson et al., 2015), refers to the feelings and emotions evoked by game dynamics. The MDA framework has brought out a hierarchical picture of game elements. However, the MDA framework emphasized on the abstract of the components of gamification while they left out the part on the motivation process of these components. Besides, while this framework takes gamification as a whole to analyze, the distinct ways in which those game elements may function are not discussed.
Some scholars put their emphasis on the effect of some specific game elements. Hanus and Fox (2015) applied badge and leaderboard to the context of education and discovered that these two kinds of game element may decrease the motives and final scores of students. Mekler. et al. (2015) used point, level and leaderboard in a tagging experiment and found that these three game elements have no influence on individuals’ intrinsic motivation, but will change users’ ultimate behaviors by affecting extrinsic motivation. These studies show that the impacts of game elements may vary under different contexts. Hence, we should seek for a more detailed explanation. In addition, game element categories that have been mentioned in these studies are limited. The effect and mechanism of game elements such as narratives and progress bar are not well studied.
To address these issues, we discuss the motivation mechanism and process of different categories of game elements in crowdsourcing projects, and posit a theoretical model that can uncover the relationships among varied game elements, user motivations and user participation.
Crowdsourcing is one of the contexts where the concept of gamification has been frequently used. It refers to a new outsourcing approach that takes tasks as an open call to an undefined, large group of people (Howe, 2006). Knowledge seekers can post their task requirements in some kind of a platform and then problem solvers can participate in these tasks to gain rewards provided by knowledge seekers after they manage to meet all the requirements requested by knowledge seekers (Liu et al., 2014). Through crowdsourcing projects, managers and organizers can make better use of knowledge, skills and resources (Neyer et al., 2009; Whitla, 2009) and reduce the cost (Howe, 2006).
However, as the cost for operating a crowdsourcing project is relatively low, and there is an increasing number of platforms supporting the crowdsourcing projects, the competition among those crowdsourcing project initiators to call up participants becomes fierce. Under these circumstances, many crowdsourcing projects failed as an end owing to the initiators’ disability to absorb enough attention from the public (McGonigal, 2011). As user engagement is of great necessity for accomplishing a crowdsourcing project, the motives and incentives for those users to participate in crowdsourcing projects are becoming a topic for relevant scholars.
2.3 Motivation theory
Porter and Lawler’s (1968) intrinsic and extrinsic motivation and Ryan et al.’s (1985) self-determination theory (SDT) are the most applied motivation theories among all the motivation research. Zheng et al. (2011) developed a research model to explain participation in crowdsourcing contexts based on the theory of extrinsic and intrinsic motivation. And by analyzing the case of Threadless, Kavaliova et al. (2016) used the SDT to explore how companies can motivate contributions to a crowdsourcing project.
Intrinsic motivation involves people doing an activity because they find it interesting and derive spontaneous satisfaction from the activity itself (Gagne and Deci, 2005). In contrast, extrinsically motivated individuals perform an activity as a means to achieve some separable objective or personal benefits, so satisfaction comes not from the activity itself but rather from the extrinsic consequences to which the activity leads (Wong-On-Wing et al., 2010). In Ryan et al.’s (1985) SDT, individual motivation is further categorized as an intrinsic motivation, internalized extrinsic motivation and extrinsic motivation. Internalization is defined as people taking in values, attitudes or regulatory structures, such that the external regulation of a behavior is transformed into an internal regulation and thus no longer requires the presence of an external contingency. Besides, Zheng et al. (2011) put forward an idea in their research on the motives of a crowdsourcing project that job characteristics are of great importance in influencing participants’ behaviors, which may act as regulatory variables to enhance or weaken the effect of intrinsic motivation. This idea is also the kernel of Hackman and Oldman’s (1980) job design theory.
2.4 Motives in crowdsourcing
Under the drive of these theories, there are a number of motives on crowdsourcing projects discovered through various studies (see Table I). Among all these motives, “reward” is one of the most important one, which could be found in all motive lists of distinct crowdsourcing projects (Brabham, 2008, 2010; Zheng et al., 2011; Fuller, 2010; Kavaliova et al., 2016; Sun et al., 2015), “Recognition” appeared just as often as “reward”. However, internalized extrinsic motivation and intrinsic motivation is relatively less found in crowdsourcing projects (Fuller, 2010; Zheng et al., 2011; Kavaliova et al., 2016).
Besides, among all these motives listed above, “skills development”, “professional development” and “social reason” (community building, social contribution, etc.) are determined by the context and purpose of the project itself, meaning that these motives are difficult to be enhanced by game elements. “Relationship building” is related with the function of the crowdsourcing platform. If the platform is very powerful in supporting social communication like providing functions such as liking others, commenting, sharing to other SNS, creating a forum/group, such kinds of motives will be enhanced. However, this can also be difficult for any game element to achieve. Hence, we exclude motives like “skills development”, “professional development”, “social reason” and “relationship building” when we discuss which motives can be enhanced by those game elements (see Table II).
3. Research model and hypothesis development
3.1 Game element categorization
Game elements which have been frequently used in crowdsourcing are point/score, leaderboard/ranking, badge/achievement, level, reward, progress bar, mission, feedback, storytelling and virtual territories (Morschheuser et al., 2016). Based on the characteristics of these game elements and the similarities among them, we further categorized these game elements into four categories: reward, recognition, progress and fantasy. Reward refers to game elements that provide rewards for some specific behaviors or achievements of participants, including tangible money and gifts, as well as virtual goods and game properties. Recognition refers to game elements that support mechanism where actions of participants to accomplish the task are converted to some kind of countable values to be compared with, including point (/score), leaderboard (/ranking), badge (/achievement) and level. Progress refers to game elements that provide information on the progression of an ongoing task on the premise that the accomplishment of a specific project is divided into several submissions or stages. Fantasy refers to game elements that create virtual circumstances, such as storytelling or virtual territory.
The research model based on the categorization mentioned above is shown as Figure 1.
3.2 Motivation mechanism
Reward refers not only to some tangible money or gifts, but also to some virtual goods or properties. This kind of game element is unable to improve the enjoyment itself, which means it cannot enhance intrinsic motivation that participants might feel. Once this kind of reward is canceled, individuals will not continue their behaviors any longer just out of their habits (Gagne and Deci, 2005).
However, no matter it is tangible money or gifts, or virtual goods or properties, all these reward-kind game elements can act as external purposes for individuals to take part in a project. Therefore, we hypothesize:
Reward-kind game elements are positively associated with extrinsic motivations.
Humans make ability judgments about self and others via comparisons (Hoorens and van Damme, 2012). Festinger’s (1954) social comparison theory predicts that individuals compare themselves to others to validate opinions, make judgments and reduce uncertainty. As an intuitive comparison result, leaderboard/ranking could help individuals ascertain the level of their competence. The rise of ranking is no doubt a symbol to recognize the improvement of someone’s ability. Compared with ranking, game elements like points/score, level are not able to present individuals’ relative positions in the whole picture. However, as the basis of ranking and leaderboard, the value of points and levels can be a representative of someone’s ability. Badge or achievement is involved when individuals’ score or level reached some extent, which can be used as a certificate for some kind of a “quality change” and recognition for someone. Hence, recognition-kind game elements like ranking, scores, levels, badges and achievements can recognize the behaviors or improvement of individuals from different perspectives, meeting the psychological need of recognition. Topper ranking, higher scores and levels, more badges or achievements could be considered as purpose or incentives for participants to take the activity enthusiastically. In other words, these kinds of game elements can enhance participants’ extrinsic motivation. In addition, the improvement of one’s ranking, the increase of scores and levels, the acquisition of new badges, the fulfillment of a new achievement could provide participants with a sense of achievement, make them realize their accumulation and progress, which can help turn such a behavior stimulated by outer purpose into an internal regulation. Therefore, by enhancing the sense of achievement, game elements that are supposed to provide recognition could internalize the extrinsic motivation. Based on the reasons above, we hypothesize:
Recognition-kind game elements are positively associated with extrinsic motivation.
Recognition-kind game elements are positively associated with internalized extrinsic motivation.
Another kind of game elements that could provide participants with a sense of achievement is the kind offering the participants information on his/her progress. The common game elements of this kind are progress bar and sub-missions. Progress bar can take every effort participants put into account and turn it into a visible bar, so that participants are able to get aware of how much has been done, as well as how much has to be done in real time, while the design of sub-missions or challenges enables participants to acquire a sense of satisfaction from every tiny fulfillment or accomplishment until they finally meet all the acquirements and finish their job.
Self-efficacy is defined as one’s belief in one’s ability to perform a task (Gist, 1987). Vroom’s (1964) expectancy theory has indicated the positive relationship between self-efficacy and the effort they are likely to put. In another word, higher self-efficacy participants feel leads to more efforts that individuals may expend. Despite the fact that most crowdsourcing projects include lack of challenges and difficulties, there is still a possibility that participants feel impatient and unmotivated. Many participants choose to give up owing to the lack of information on how much time and efforts they must put in to reach the finishing line. However, game elements designed to provide such kind of information can assist participants in ascertaining where they are, how much effort and time is still needed and give them the confidence of their ability to accomplish the job, which in other words, can enhance their self-efficacy.
On the one hand, progress-kind game elements are capable of providing goals for participants to achieve and giving constant recognition of their effort, by which it can enhance extrinsic motivation of participants. On the other hand, these game elements can provide information on one’s progress that can lead to a sense of achievement, as well as enhance participants’ self-efficacy and eventually increase internalized extrinsic motivation. Hence, we hypothesize:
Progress-kind game elements are positively associated with extrinsic motivation.
Progress-kind game elements are positively associated with internalized extrinsic motivation.
Csikszentmihalyi (1990) has brought up the theory of flow to explain the acceptance intention and behavior of individuals toward some specific technology. Flow is defined as a state in which people are so involved in something that nothing else seems to matter. What’s more important is that when individuals are immersed in the flow provided by some object, they will more easily feel a sense of enjoyment and satisfaction. In other words, intrinsic motivation will be enhanced in this state.
Fantasy-kind game elements such as storytelling and virtual territory can inundate participants with the information flow. Such game elements could meet all four dimensions brought up by Trevino and Webster (1992) which are used to describe the characteristics when individuals are in the state of flow. First, participants could get a new clue or feedback when they have conducted any specific activities, making the participants feel like they are the ones who are controlling the process of the whole project; the constant feedback is more likely to grab participants’ attention; participants’ curiosity may be aroused considering they might want to know what will happen next; and ultimately, the constant information flow of the virtual story or territory could provide participants with a sense of enjoyment and satisfaction, while users’ intrinsic motivation gets enhanced. Hence, we hypothesize:
Fantasy-kind game elements are positively associated with the intrinsic motivation.
3.3 Intrinsic motivation, extrinsic motivation and internalized extrinsic motivation
Intrinsic motivation involves people doing an activity because they find it interesting and derive spontaneous satisfaction from the activity itself (Wu and Lu, 2013), which has an influence on whether the application of gamification would meet the expectation. Those companies that figure out how to effectively use gamification to amplify the intrinsic motivations of their employees, fans and customers will have a lasting competitive edge in their markets (Deterding, 2012). Intrinsic motivation is one of the cardinal incentives for participants to take part in crowdsourcing projects, as participants might choose these tasks out of the enjoyment they might gain in the process of overcoming the challenge and solve the problem (Zheng et al., 2011). Thus, the effect of intrinsic motivation has obtained wide acceptance from scholars and practitioners. Therefore, we hypothesize:
Intrinsic motivation is positively associated with user participation.
Extrinsic motivation is the motivation to work for something apart from and external to the work itself, such as reward or recognition from other people (Zheng et al., 2011). Extrinsic motivation requires an instrumentality between the activity and some separable consequences, so satisfaction comes not from the activity itself but rather from the extrinsic consequences to which the activity leads (Gagne and Deci, 2005). Reward and recognition are most common extrinsic motivations in the context of crowdsourcing (Brabham, 2008; Fuller, 2010; Kavaliova et al., 2016; Sun et al., 2015; Zheng et al., 2011). They are both capable of becoming a purpose for individuals to motivate themselves to take some specific actions. Hence, we hypothesize:
Extrinsic motivation is positively associated with user participation.
The SDT and situational relevance theory advocate that the motivational affordances of the game mechanics are most effective when they work toward personally meaningful goals and when users are empowered to design or set their own goals (Wu, 2011). Deterding et al. (2011) even have a viewpoint that a mechanic can motivate only when there is a match between this mechanic and the user’s profile (needs, values and interests). According to the definition of internalization (Ryan et al., 1985), internalized extrinsic motivations could make people “taking in values, attitudes or regulatory structures”, thus could align the personal goals with the context of the project. Hence, internalized extrinsic motivation could motivate the engagement behaviors of participants, even better than extrinsic motivation. Thus, we hypothesize:
Internalized extrinsic motivation is positively associated with user participation.
3.4. Task complexity as a moderator
According to Wood (1986), task complexity is a function of the number of distinct acts that must be completed and the number of distinct information cues about the attributes of the task-related stimulus object an individual has to process when performing a task. Jiang and Benbasat (2007) hold the opinion that high task complexity can increase information processing requirements and demand more resources from task executors. An increase in task complexity equals the increase in efforts and time that participants must spend on the tasks; thus, they may find that the awards or recognition from others are not worthy of efforts of participating in the project. As a result, they are less likely to be lured by extrinsic motivation. Thus, we think:
The effect of game elements that could enhance extrinsic motivation will be less prominent when tasks become highly complex.
Besides, as tasks in crowdsourcing projects are relatively simple, repetitive and unchallenging, individuals’ attention to tasks may get diluted when task complexity reaches a certain level (Todd and Benbasat, 1999; Kahneman, 1973). As participants’ attention is no longer paid to the task itself, they may not be attracted by the information flow those tasks provide and consequently get out of the state of flow and obtain no fun from the process of accomplishing the task. Therefore, we hypothesize:
The effect of game elements that could enhance intrinsic motivation will be less prominent when the tasks become highly complex.
Things may become different when referring to the game elements that could enhance internalized extrinsic motivation. The internalized extrinsic motives we have discussed so far are self-efficacy and achievement. As simple tasks that could be done in no time, the progress bar may not play a significant role as a reminder. However, with an increase in task complexity, these kinds of game elements are beginning to make a difference. These game elements can remind participants of how much they have done and how much remains ahead of them, and thus can enhance the self-efficacy and weaken the anxiety led by task complexity. Besides, simple tasks may not be able to provide participants with enough sense of achievement, but complex tasks can make it happen. Thus, we hypothesize:
The effect of game elements that could enhance internalized extrinsic motivation will be more prominent when tasks become highly complex.
4. Research methodology
To test the hypotheses, a 5 (no game element vs reward vs progress bar vs virtual territory vs point and leaderboard) by 2 (different choice sizes of the kinds of clothes) between-subjects’ laboratory experiments will be conducted. We plan to design a crowdsourcing task where we will act as a clothing company that needs to collect opinions and preferences of customers based on their choices of clothes.
In the control group, none of game elements will be applied and a list of clothing will be shown to the participants who are supposed to choose the clothes they prefer from each kind. In each of our experimental groups, a specific game element will be embedded, so that the motivation effect of game elements from different categories will be examined separately. The setting of our control group and experimental groups is shown in Table III. While in the second dimension, the influence of task complexity that acted as a moderator will be tested by using two different choice sizes of the kinds of clothes which participants are requested to choose from.
Data will be collected from two sources, the performance data from system logs and the subjective data from post-survey. The time those participants have spent on choosing clothes and the number of clothes they have chosen will be retrieved from the system and used as the measure for user participation. Measurement items for several major constructs – intrinsic motivation, two kinds of extrinsic motivation and two kinds of internalized intrinsic motivation – will be adapted from the Work Preference Inventory (Amabile et al., 1994), which is used to assess intrinsic and extrinsic motivation. Two different choice sizes of the kinds of clothes will be taken as the measure of task complexity.
5. Expected contribution
Gamification has attracted increasing attention from researchers and practitioners in varied contexts. Our research can make contributions in several ways.
First, this research uncovers the motivation mechanism of these game elements, enabling us to clarify the latent relationship between game elements and user behavior. Second, we are more likely to find out which game elements are more effective in crowdsourcing projects through a series of experimental groups. Third, by categorizing game elements and selecting representative game elements in each category as our research subjects, we are able to discover motivation processes of game elements. Forth, as we use task complexity as a moderator, we may be able to identify whether task complexity influences the application of game elements, as well as which game elements should be selected in varied extents of task complexity. Last but not the least, as our model is proposed within the context of crowdsourcing, the framework that indicates the interrelationship between game elements, human motivation and user participation could also be referred and adapted by other research investigations that are not limited to crowdsourcing projects.
Motives in crowdsourcing projects
|Extrinsic motivation||Reward||Brabham (2008), Zheng et al. (2011), Fuller (2010), Kavaliova et al. (2016), Sun et al. (2015), Brabham (2010)|
|Recognition||Fuller (2010), Zheng et al. (2011), Kavaliova et al. (2016)|
|Internalized extrinsic motivation||New skills’ acquirement||Brabham (2010), Fuller (2010)|
|New job opportunity||Brabham (2010)|
|Relationship||Fuller (2010), Kavaliova et al. (2016)|
|Self-efficacy||Fuller (2010), Sun et al. (2015), Kavaliova et al. (2016)|
|Achievement||Fuller (2010), Kavaliova et al. (2016)|
|Social consideration||Brabham (2010), Fuller (2010)|
|Intrinsic motivation||Fun||Fuller (2010), Kavaliova et al. (2016)|
Motives related to game elements in crowdsourcing projects
|Extrinsic motivation||Reward||Brabham (2008), Zheng et al. (2011), Fuller (2010), Kavaliova et al. (2016), Sun et al. (2015), Brabham (2010)|
|Recognition||Fuller (2010, Zheng et al. (2011), Kavaliova et al. (2016)|
|Internalized extrinsic motivation||Self-efficacy||Fuller (2010), Kavaliova et al. (2016), Sun et al. (2015)|
|Achievement||Fuller (2010), Kavaliova et al. (2016)|
|Intrinsic motivation||Fun||Fuller (2010), Kavaliova et al. (2016)|
Control group and experimental groups settings
|Group||Game elements applied||Settings|
|Control group||None||Clothes are categorized according to their kinds. A list of clothing will be shown and participants are supposed to choose their favorite clothes from each kind|
|Experimental Group A||Reward||Clothes are categorized according to their kinds. A list of clothing will be shown and participants are supposed to choose their favorite clothes from each kind. Participants will be informed that they maybe gain a gift for themselves if they did a good job, which will be assessed by the time they spent, and the number of clothes they choose|
|Experimental Group B||Progress bar||Clothes are categorized according to their kinds. A list of clothing will be shown and participants are supposed to choose their favorite clothes from each kind. A progress bar that indicates the progress of the task is shown to remind participants of how much work they have done and still left|
|Experimental Group C||Virtual territory||Clothes are categorized according to their kinds. Participants will first choose a model as their virtual “boy-/girl-friend”. Then they are requested to dress their virtual boy-/girl-friend in a given territory by choosing their favorite clothes from each kind|
|Experimental Group D||Point and leaderboard||Clothes are categorized according to their kinds. A list of clothing will be shown and participants are supposed to choose their favorite clothes from each kind. Once participants choose one clothes from its kind, they will obtain a certain amount of score that will be summed up to decide their rankings. Both points of participants and a leaderboard will be shown to the participants in real-time|
Amabile, T.M., Hill, K.G., Hennessey, B.A. and Tighe, E.M. (1994), “The work preference inventory: assessing intrinsic and extrinsic motivational orientations”, Journal of Personality and Social Psychology, Vol. 66, pp. 950-967.
Brabham, D.C. (2008), “Crowdsourcing as a model for problem solving: an introduction and cases”, The International Journal of Research into New Media Technologies, Vol. 14, pp. 75-90.
Brabham, D.C. (2010), “Moving the crowd at threadless: motivations for participation in a crowdsourcing application”, Information, Communication and Society, Vol. 13, pp. 1122-1145.
Csikszentmihalyi, M. (1990), Flow: The Psychology of Optimal Experience, Harper and Row, New York, NY.
Deterding, S. (2012), “Gamification: designing for motivation”, Magazine Interactions, Vol. 19 No. 4, pp. 14-17.
Deterding, S., Dixon, D., Khaled, R. and Nacke, L. (2011), “From game design elements to gamefulness: defining gamification”, Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, ACM, New York, NY, pp. 9-15.
Festinger, L. (1954), “A theory of social comparison processes”, Human Relations, Vol. 7, pp. 117-140.
Fuller, J. (2010), “Refining virtual co-creation from a consumer perspective”, California Management Review, Vol. 52 No. 2.
Gagne, M. and Deci, E.L. (2005), “Self-determination theory and work motivation”, Journal of Organizational Behavior, Vol. 26 No. 4, pp. 331-362.
Gist, M.E. (1987), “Self-efficacy: a theoretical analysis of its determinants and malleability”, Academy of Management Review, Vol. 17 No. 2, pp. 183-211.
Hackman, J.R. and Oldman, G.R. (1980), Work Redesign, Addison-Wesley, Reading, MA.
Hamari, J. and Koivisto, J. (2013), “Social motivations to use gamification: an empirical study of gamifying exercise”, The 21st European Conference on Information Systems, Utrecht.
Hanus, M.D. and Fox, J. (2015), “Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance”, Computers & Education, Vol. 80, pp. 152-161.
Hoorens, V. and van Damme, C. (2012), “What do people infer from social comparisons? Bridges between social comparison and person perception”, Social & Personality Psychology Compass, Vol. 6 No. 8, pp. 607-618.
Howe, J. (2006), “The rise of crowdsourcing”, available at: www.wired.com/wired/archive/14.06/crowds.html
Hunicke, R., Leblanc, M. and Zubek, R. (2004), “MDA: a formal approach to game design and game research”, AAAI Workshop on Challenges in GAME AL, San Jose, CA.
Jiang, Z. and Benbasat, I. (2007), “The effects of presentation formats and task complexity on online consumers' product understanding”, MIS Quarterly, Vol. 31 No. 3, pp. 485-500.
Kahneman, D. (Ed.) (1973), Attention and Effort, Prentice-Hall, Englewood Cliffs, NJ.
Kavaliova, M., Virjee, F., Maehel, N. and Kleppe, I.A. (2016), “Crowdsourcing innovation and product development: gamification as a motivational driver”, Cogent Business & Management, Vol. 3 No. 1.
Liu, T.X., Yang, J., Adamic, L.A. and Chen, Y. (2014), “Crowd sourcing with all-pay auctions: a field experiment on Taskcn”, Management Science, Vol. 60 No. 8, pp. 2020-2037.
McGonigal, J. (2011), Reality is Broken: Why Games Make Us Better and How They Can Change the World, Penguin Press, New York, NY.
Malone, T.W. (1981), “Heuristics for designing enjoyable user interfaces: lessons from computer games”, CHI '82 Proceedings of the 1982 Conference on Human Factors in Computing Systems, ACM, New York, NY, pp. 63-68.
Mekler, E.D., Bruhlmann, F., Tuch, A.N. and Opwis, K. (2015), “Towards understanding the effects of individual gamification elements on intrinsic motivation and performance”, Computers in Human Behavior, pp. 1-10.
Morschheuser, B., Hamari, J. and Koivisto, J. (2016), “Gamification in crowdsourcing: a review”, 49th Annual Hawaii International Conference on System Sciences (HICSS), Kauai, Hawaii.
Neyer, A.K., Bullinger, A.C. and Moeslein, K.M. (2009), “Integrating inside and outside innovators: a socio-technical systems perspective”, R&D Management Journal, Vol. 39, pp. 410-419.
Porter, L.W. and Lawler, E.E.I. (1968), Managerial Attitudes and Performance, Irwin-Dorsey, Homewood, IL.
Robson, K., Plangger, K., Keitzmann, J.H., Mccarthy, A. and Pitt, L. (2015), “Is it all a game? Understanding the principles of gamification”, Business Horizons, Vol. 58, pp. 411-420.
Ryan, R.M., Connell, J.P. and Deci, E.L. (1985), “A motivational analysis of self-determination and self-regulation in education”, in Ames, R. and Ames, C. (Eds), Research on Motivation in Education: The Classroom Milieu, Academic Press, New York, NY.
Sigala, M. (2015), “The application and impact of gamification funware on trip planning and experiences: the case of TripAdvisor’s funware”, Electron Markets, Vol. 25 No. 3, pp. 189-209.
Simoes, J., Redondo, R.D. and Vilas, A.F. (2013), “A social gamification framework for a K-6 learning platform”, Computers in Human Behavior, Vol. 29 No. 2, pp. 345-353.
Sun, Y., Wang, N., Yin, C. and Zhang, J.X. (2015), “Understanding the relationships between motivators and effort in crowdsourcing marketplaces: a nonlinear analysis”, International Journal of Information Management, Vol. 35 No. 3, pp. 267-276.
Todd, P. and Benbasat, I. (1999), “Evaluation the impact of DSS, cognitive effort, and incentives on strategy selection”, Information Systems Research, Vol. 10 No. 4, pp. 356-375.
Trevino, L.K. and Webster, J. (1992), “Flow in computer-mediated communication: electronic mail and voice mail evaluation and impacts”, Communication Research, Vol. 19 No. 5, pp. 539-573.
Vroom, V.H. (1964), Work and Motivation, Wiley, New York, NY.
Werbach, K. and Hunter, D. (2012), For the Win, Wharton Digital Press, Philadelphia, PA.
Whitla, P. (2009), “Crowdsourcing and its application in marketing activities”, Journal of Contemporary Management Research, Vol. 5 No. 1, pp. 15-28.
Wong-On-Wing, B., Guo, L. and Lui, G. (2010), “Intrinsic and extrinsic motivation and participation in budgeting: antecedents and consequences”, Behavioral Research in Accounting, Vol. 22 No. 3, pp. 133-153.
Wood, R.E. (1986), “Task complexity: definition of a construct”, Organization Behavior and Human Decision Processes, Vol. 37 No. 1, pp. 60-82.
Wu, J. and Lu, X. (2013), “Effects of extrinsic and intrinsic motivators on using utilitarian, hedonic, and dual-purposed information systems: a meta-analysis”, Journal of the Association for Information Systems, Vol. 14 No. 3, pp. 153-191.
Wu, M. 2011. “Sustainable gamification: playing the game for the long haul”, available at: http://lithosphere.lithium.com/t5/Building-Community-the-Platform/Sustainable-Gamification-Playing-the-Game-for-the-Long-Haul/ba-p/33601
Zheng, H., Li, D. and Hou, W. (2011), “Task design, motivation, and participation in crowdsourcing contests”, International Journal of Electronic Commerce, Vol. 15 No. 4, pp. 57-88.
This research is supported by the National Social Science Fund of China (Grant No.14AZD045).