Search results

Allocation of complex crowdsourcing tasks, which typically include heterogeneous attributes such as value, difficulty, skill required, effort required and deadline, is still a challenging open problem. In recent years, agent-based crowdsourcing approaches focusing on recommendations or incentives have emerged to dynamically match workers with diverse characteristics to tasks to achieve high collective productivity. However, existing approaches are mostly designed based on expert knowledge grounded in well-established theoretical frameworks. They often fail to leverage on user-generated data to capture the complex interaction of crowdsourcing participants’ behaviours. This paper aims to address this challenge.

The paper proposes a policy network plus reputation network (PNRN) approach which combines supervised learning and reinforcement learning to imitate human task allocation strategies which beat artificial intelligence strategies in this large-scale empirical study. The proposed approach incorporates a policy network for the selection of task allocation strategies and a reputation network for calculating the trends of worker reputation fluctuations. Then, by iteratively applying the policy network and reputation network, a multi-round allocation strategy is proposed.

PNRN has been trained and evaluated using a large-scale real human task allocation strategy data set derived from the Agile Manager game with close to 500,000 decision records from 1,144 players in over 9,000 game sessions. Extensive experiments demonstrate the validity and efficiency of computational complex crowdsourcing task allocation strategy learned from human participants.

The paper can give a better task allocation strategy in the crowdsourcing systems.

The purpose of this paper is to study the participation behaviors in the context of crowdsourcing projects from the perspective of gamification.

This paper first proposed a model to depict the effect of four categories of game elements on three types of motivation based upon several motivation theories, which may, in turn, influence user participation. Then, 5 × 2 between-subject Web experiments were designed for collecting data and validating this model.

Game elements which provide participants with rewards and recognitions or remind participants of the completion progress of their tasks may positively influence the extrinsic motivation, whereas game elements which can help create a fantasy scene may strengthen intrinsic motivation. Besides, recognition-kind and progress-kind game elements may trigger the internalization of extrinsic motivation. In addition, when a task is of high complexity, the effects from game elements on extrinsic motivation and intrinsic motivation will be less prominent, whereas the internalization of extrinsic motivation may benefit from the increase of task complexity.

This study may uncover the motivation mechanism of several different kinds of game elements, which may help to find which game elements are more effective in enhancing engagement and participation in crowdsourcing projects. Besides, as task complexity is used as a moderator, one may be able to identify whether task complexity is able to influence the effects from game elements on motivations. Last, but not the least, this study will indicate the interrelationship between game elements, individual motivation and user participation, which can be adapted by other scholars.

Ensuring quality is one of the most significant challenges in microtask crowdsourcing tasks. Aggregation of the collected data from the crowd is one of the important steps to infer the correct answer, but the existing study seems to be limited to the single-step task. This study aims to look at multiple-step classification tasks and understand aggregation in such cases; hence, it is useful for assessing the classification quality.

The authors present a model to capture the information of the workflow, questions and answers for both single- and multiple-question classification tasks. They propose an adapted approach on top of the classic approach so that the model can handle tasks with several multiple-choice questions in general instead of a specific domain or any specific hierarchical classifications. They evaluate their approach with three representative tasks from existing citizen science projects in which they have the gold standard created by experts.

The results show that the approach can provide significant improvements to the overall classification accuracy. The authors’ analysis also demonstrates that all algorithms can achieve higher accuracy for the volunteer- versus paid-generated data sets for the same task. Furthermore, the authors observed interesting patterns in the relationship between the performance of different algorithms and workflow-specific factors including the number of steps and the number of available options in each step.

Due to the nature of crowdsourcing, aggregating the collected data is an important process to understand the quality of crowdsourcing results. Different inference algorithms have been studied for simple microtasks consisting of single questions with two or more answers. However, as classification tasks typically contain many questions, the proposed method can be applied to a wide range of tasks including both single- and multiple-question classification tasks.

Today’s software engineers often work in teams to develop complex software systems. Therefore, successful software engineering in practice require team members to possess not only sound programming skills such as analysis, design, coding and testing but also soft skills such as communication, collaboration and self-management. However, existing examination-based assessments are often inadequate for quantifying students’ soft skill development. The purpose of this paper is to explore alternative ways for assessing software engineering students’ skills through a data-driven approach.

In this paper, the exploratory data analysis approach is adopted. Leveraging the proposed online agile project management tool – Human-centred Agile Software Engineering (HASE), a study was conducted involving 21 Scrum teams consisting of over 100 undergraduate software engineering students in multi-week coursework projects in 2014.

During this study, students performed close to 170,000 software engineering activities logged by HASE. By analysing the collected activity trajectory data set, the authors demonstrate the potential for this new research direction to enable software engineering educators to have a quantifiable way of understanding their students’ skill development, and take a proactive approach in helping them improve their programming and soft skills.

To the best of the authors’ knowledge, there has yet to be published previous studies using software engineering activity data to assess software engineers’ skills.

The dropout rates of open e-learning platforms are often cited as high as 97%, with many users discontinuing their use after initial acceptance. This study aims to explore this anomaly through the lens of affordances theory, revealing design–reality gaps between users' diverse goals and the possibilities for action provided by an open IT artefact.

A six-month case study was undertaken to investigate the design implications of user-perceived affordances in an EU sustainability project which developed an open e-learning platform for citizens to improve their household energy efficiency. Thematic analysis was used to reveal the challenges of user continuance behaviour based on how an open IT artefact supports users in achieving individual goals (e.g. reducing energy consumption in the home) and collective goals (lessening the carbon footprint of society).

Based on the findings, the authors inductively reveal seven affordances related to open e-learning platforms: informing, assessment, synthesis, emphasis, clarity, learning pathway and goal-planning. The findings centre on users' perception of these affordances, and the extent to which the open IT artefact catered to the goals and constraints of diverse user groups. Open IT platform development is further discussed from an iterative and collaborative perspective in order to explore different possibilities for action.

The study contributes towards research on open IT artefact design by presenting key learnings on how the designers of e-learning platforms can bridge design–reality gaps through exploring affordance personalisation for diverse user groups. This can inform the design of open IT artefacts to help ensure that system features match the expectations and contextual constraints of users through clear action-oriented possibilities.

The coronavirus disease 2019 (COVID-19) pandemic has led to a global digitalization of organizational activities: the pandemic forced people and organizations to profoundly review values, purposes and norms. However, the research on how digital technologies impact human relationships and interactions at work results fragmented. Still, the importance of understanding which behaviors and norms enhance social interactions and organizational performances in digital environments remains critical, especially after COVID-19 advent. Therefore, this study explores how human relationships change in a wholly digital environment and what to expect for the new normal.

The study first explores the research gap through a systematic literature review to clearly understand what emerged so far. Second, through semi-structured interviews and a focus group, an empirical analysis was conducted.

Findings suggest that both work and emotional dimensions are crucial to nurturing human relationships in a digital environment. More precisely, the study unveils the need for innovative leaders to review their approaches to communication and the work experience and consider the emotional dimension in terms of community purpose and individual well-being, while identifying rituals as an overlapping tool. Finally, the authors propose a parallelism between these results and the agile revolution to inspire leaders to rethink their leadership and behaviors getting closer to the agile approach, which may represent a valuable way to rethink human relations in our professional environment.

The paper sheds light on an ongoing phenomenon that touches the lives of each organizational actor. The two-step structure hopes to provide both a structured base of the knowledge developed to date, proposing a systematic view of what has been studied since the outbreak of the pandemic to date and to provide insights for future developments.