Search results

The purpose of this paper is to present a new concept – task affordance in crowdsourcing context, and build it as a theoretical lens to help the authors reconfigure the artifacts and process in task-oriented crowdsourcing projects. The paper differs from previous studies by focusing on the relationships between the task artifacts, systems and goal-directed actors in crowdsourcing process rather than on the pure examination of task properties.

An operational definition of task affordance was proposed and a pseudo-entity-relationship model based approach was employed to portrait the task affordance in online crowdsourcing context. Furthermore, the authors developed a typology of task affordance and decomposed the concept into five dimensions, namely, design affordance, presentation affordance, assignment affordance, task-platform fit affordance, and task-worker fit affordance. A preliminary analysis of task affordances across various crowdsourcing categories was also conducted to validate the proposed typological framework.

The findings show that the task affordances have varying degree and extend among the diverse crowdsourcing categories. For instance, task design affordances seem to be low in the crowd processing and crowd rating cases compared with that in the crowd solving and crowd creation cases. For another example, in terms of the task presentation affordance, crowd rating cases need the lowest affordance while the crowd creation cases need the highest affordance. Therefore, the authors would like to emphasize that the successful adoption, implementation, and design of the task-oriented crowdsourcing owes to the careful examination of the relationships among the actors, artifacts, and environment of the crowdsourcing projects.

To the authors’ best knowledge, this paper is the first study on conceptualizing the task affordance in online crowdsourcing context. The study contributes to the academic literature on a comprehensive overview of task-related studies in crowdsourcing, which are scattered in several information related fields. Furthermore, this research contributes directly to the area of information science and technology due to a common interest in studying the environments and contexts in which people, information and technology interact and interplay. Practically, this study may yield some implications for the requester and platform operator when designing the relevant tasks or developing the specific crowdsourcing platform.

Allocation of complex crowdsourcing tasks, which typically include heterogeneous attributes such as value, difficulty, skill required, effort required and deadline, is still a challenging open problem. In recent years, agent-based crowdsourcing approaches focusing on recommendations or incentives have emerged to dynamically match workers with diverse characteristics to tasks to achieve high collective productivity. However, existing approaches are mostly designed based on expert knowledge grounded in well-established theoretical frameworks. They often fail to leverage on user-generated data to capture the complex interaction of crowdsourcing participants’ behaviours. This paper aims to address this challenge.

The paper proposes a policy network plus reputation network (PNRN) approach which combines supervised learning and reinforcement learning to imitate human task allocation strategies which beat artificial intelligence strategies in this large-scale empirical study. The proposed approach incorporates a policy network for the selection of task allocation strategies and a reputation network for calculating the trends of worker reputation fluctuations. Then, by iteratively applying the policy network and reputation network, a multi-round allocation strategy is proposed.

PNRN has been trained and evaluated using a large-scale real human task allocation strategy data set derived from the Agile Manager game with close to 500,000 decision records from 1,144 players in over 9,000 game sessions. Extensive experiments demonstrate the validity and efficiency of computational complex crowdsourcing task allocation strategy learned from human participants.

The paper can give a better task allocation strategy in the crowdsourcing systems.

The purpose of this paper is to study the participation behaviors in the context of crowdsourcing projects from the perspective of gamification.

This paper first proposed a model to depict the effect of four categories of game elements on three types of motivation based upon several motivation theories, which may, in turn, influence user participation. Then, 5 × 2 between-subject Web experiments were designed for collecting data and validating this model.

Game elements which provide participants with rewards and recognitions or remind participants of the completion progress of their tasks may positively influence the extrinsic motivation, whereas game elements which can help create a fantasy scene may strengthen intrinsic motivation. Besides, recognition-kind and progress-kind game elements may trigger the internalization of extrinsic motivation. In addition, when a task is of high complexity, the effects from game elements on extrinsic motivation and intrinsic motivation will be less prominent, whereas the internalization of extrinsic motivation may benefit from the increase of task complexity.

This study may uncover the motivation mechanism of several different kinds of game elements, which may help to find which game elements are more effective in enhancing engagement and participation in crowdsourcing projects. Besides, as task complexity is used as a moderator, one may be able to identify whether task complexity is able to influence the effects from game elements on motivations. Last, but not the least, this study will indicate the interrelationship between game elements, individual motivation and user participation, which can be adapted by other scholars.

Ensuring quality is one of the most significant challenges in microtask crowdsourcing tasks. Aggregation of the collected data from the crowd is one of the important steps to infer the correct answer, but the existing study seems to be limited to the single-step task. This study aims to look at multiple-step classification tasks and understand aggregation in such cases; hence, it is useful for assessing the classification quality.

The authors present a model to capture the information of the workflow, questions and answers for both single- and multiple-question classification tasks. They propose an adapted approach on top of the classic approach so that the model can handle tasks with several multiple-choice questions in general instead of a specific domain or any specific hierarchical classifications. They evaluate their approach with three representative tasks from existing citizen science projects in which they have the gold standard created by experts.

The results show that the approach can provide significant improvements to the overall classification accuracy. The authors’ analysis also demonstrates that all algorithms can achieve higher accuracy for the volunteer- versus paid-generated data sets for the same task. Furthermore, the authors observed interesting patterns in the relationship between the performance of different algorithms and workflow-specific factors including the number of steps and the number of available options in each step.

Due to the nature of crowdsourcing, aggregating the collected data is an important process to understand the quality of crowdsourcing results. Different inference algorithms have been studied for simple microtasks consisting of single questions with two or more answers. However, as classification tasks typically contain many questions, the proposed method can be applied to a wide range of tasks including both single- and multiple-question classification tasks.

The rapid development of Web 2.0 and social media enables the rise of crowdsourcing. Crowdsourcing contest is a typical case of crowdsourcing and has been adopted by many organisations for business solution and decision making. From a participant's perspective, it is interesting to explore what motivates people to participate in crowdsourcing contest. The purpose of this paper is to investigate the category of motivation based on self-determination theory and synthesises various motivation factors in crowdsourcing contest. Meanwhile, perceived motivational affordances and task granularity are also examined as the moderate constructs.

The paper builds a conceptual model to illustrate the relationships between various motivations (extrinsic and intrinsic) and participation effort under the moderating of perceived motivational affordances and task granularity. An empirical study is conducted to test the research model by surveying the Chinese participants of crowdsourcing contest.

The results show that various motivations might play different roles in relating to participation effort expended in the crowdsourcing contest. Moreover, task granularity may positively moderate the relationship between external motivation and participation effort. The results also show that supporting of a participant's perceived motivational affordances might strengthen the relationship between the individual's motivation with an internal focus (intrinsic, integrated, identified and introjected motivation) and participation effort.

Overall, the research has some conceptual and theoretical implications to the literature. This study synthesises various motivation factors identified by previous studies in crowdsourcing projects or communities as a form of motivation spectrum, namely external, introjected, identified, integrated and intrinsic motivation, which contributes to the motivation literatures. Meanwhile, the findings indicate that various motivations might play different roles in relating to participation effort expended in the crowdsourcing contest. Also, the study theoretically extends the crowdsourcing participation research to incorporate the effects of perceived motivational affordances in crowdsourcing contest. In addition, the study may yield some practical implications for sponsors, managers and designers in crowdsourcing contest.

Today’s software engineers often work in teams to develop complex software systems. Therefore, successful software engineering in practice require team members to possess not only sound programming skills such as analysis, design, coding and testing but also soft skills such as communication, collaboration and self-management. However, existing examination-based assessments are often inadequate for quantifying students’ soft skill development. The purpose of this paper is to explore alternative ways for assessing software engineering students’ skills through a data-driven approach.

In this paper, the exploratory data analysis approach is adopted. Leveraging the proposed online agile project management tool – Human-centred Agile Software Engineering (HASE), a study was conducted involving 21 Scrum teams consisting of over 100 undergraduate software engineering students in multi-week coursework projects in 2014.

During this study, students performed close to 170,000 software engineering activities logged by HASE. By analysing the collected activity trajectory data set, the authors demonstrate the potential for this new research direction to enable software engineering educators to have a quantifiable way of understanding their students’ skill development, and take a proactive approach in helping them improve their programming and soft skills.

To the best of the authors’ knowledge, there has yet to be published previous studies using software engineering activity data to assess software engineers’ skills.

Task recommendation is an important way for workers and requesters to get better outcomes in shorter time in crowdsourcing. This paper aims to propose an approach based on 2-tuple fuzzy linguistic method to recommend tasks to the workers who would be capable of completing and accept them.

In this paper, worker’s capability-to-complete (CTC) and possibility-to-accept (PTA) for a task needs to be recommended are proposed, measured and aggregated to determine worker’s priority for task recommendation. Therein, the similarity between the recommended task and its similar tasks and worker’s performance on these similar tasks are computed and aggregated to determine worker’s CTC quantitatively. In addition, two factors of worker’s active degree and worker’s preferences to a task category are presented to reflect and determine worker’s PTA. In the process of measuring them, 2-tuple fuzzy linguistic method is used to represent, process and aggregate vague and imprecise information.

To demonstrate the implementation process and performance of the proposed approach, an illustrative example is conducted on Taskcn, a widely used Chinese online crowdsourcing market. The experimental results show that the proposed approach outperformed the self-selection approach, especially for complex or creative tasks. Moreover, comparing with task recommendation considering worker’s CTC solely, the proposed approach would be better in terms of workers’ response rate. Additionally, the use of linguistic terms and fuzzy linguistic method facilitates the expression of vague and subjective information and makes recommendation process more practical.

In the study, the authors capture alternative workers, collect workers’ behaviors and compute workers’ CTC and PTA manually. However, as the number of tasks and alternative workers grow, the issue, i.e. how to conveniently collect workers’ behaviors and determine their CTC and PTA, becomes conspicuous and needs to be studied further.

The proposed approach provides an alternative way to perform tasks posted in crowdsourcing platforms. It can assist workers to contribute to right tasks, and requesters to get outcomes with high quality more efficiently.

This study proposes an approach to task recommendation in crowdsourcing that integrates workers’ CTC and PTA for the recommended tasks and can deal with vague and imprecise information.

As the number of joined service providers (SPs) in knowledge-intensive crowdsourcing (KI-C) continues to rise, there is an information overload problem for KI-C platforms and consumers to identify qualified SPs to complete tasks. To this end, this paper aims to propose a quality of service (QoS) evaluation framework for SPs in KI-C to effectively and comprehensively characterize the QoS of SPs, which can aid the efficient selection of qualified SPs.

By literature summary and discussion with the expert team, a QoS evaluation indicator system for SPs in KI-C based on the SERVQUAL model is constructed. In addition, the Decision Making Trial and Evaluation Laboratory (DEMATEL) method is used to obtain evaluation indicators' weights. The SPs are evaluated and graded by the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) and rank–sum ratio (RSR), respectively.

A QoS evaluation indicator system for SPs in KI-C incorporating 13 indicators based on SERVQUAL has been constructed, and a hybrid methodology combining DEMATEL, TOPSIS and RSR is applied to quantify and visualize the QoS of SPs.

The QoS evaluation framework for SPs in KI-C proposed in this paper can quantify and visualize the QoS of SPs, which can help the crowdsourcing platform to realize differentiated management for SPs and assist SPs to improve their shortcomings in a targeted manner. And this is the first paper to evaluate SPs in KI-C from the prospect of QoS.

The purpose of this paper is to investigate the roles of online leadership in open collaborative innovation success by extending functional leadership theory in the context of open source projects.

This study uses negative binomial regression models to empirically test the proposed hypotheses with samples of blockchain open source projects on GitHub.

The results indicate that task-oriented leadership behaviors in forms of technical contributions have little influence on open collaborative innovation success; relation-oriented leadership behaviors embedded in internal social capital and external social capital contribute to open collaborative innovation success prominently. Furthermore, the joint effects of technical contributions, internal social capital and community commitment with openness orientation are positively significant on open collaborative innovation success, respectively.

For leaders and participants of open collaborative innovation projects, they should attach importance to both leadership behaviors and the joint effects with openness orientation so as to make informed decisions.

This study offers a new fine-grained framework of open collaborative innovation success by investigating specific dimensions of task-oriented and relation-orientated leadership behaviors, as well as their joint effects with openness orientation.

The purpose of this empirical study is to investigate extrinsic motivations that may affect adding or acknowledging user-generated content (UGC) on business websites, which are based on voluntary crowdsourcing.

A conceptual model of extrinsic motivations for knowledge sharing in UGC-based websites was developed, suggesting reciprocity, awareness of rewards and prestige as main extrinsic motivations for adding content. The model was examined via an online survey of users of three websites that varied in the attributes of knowledge shared and reward type: The Traveler (tangible rewards), Stack Overflow (virtual rewards) and Waze (virtual rewards).

Importance of extrinsic motivations varied among websites, as it may be affected by attributes of the knowledge shared. Reciprocity positively affected recommending the website, and adding content affected acknowledging content.

Investigating extrinsic motivations is important because websites may take actions that affect them. Further research is required to reveal the potential of voluntary crowdsourcing in business contexts addressing both intrinsic and extrinsic motivations, prosumption and open innovation.

When reciprocity is a major aspect of a UGC website, badges and similar mechanisms may serve as a main extrinsic motivation to share knowledge.

The novel empirically validated model provides theoretical and practical insights for designing mechanisms for increasing extrinsic motivation for knowledge sharing according to specific characteristics of UGC websites.