Where is the student who was here? Gamification as a strategy to engage students

Vanessa Itacaramby Pardim (School of Economics, Business, and Accounting, University of Sao Paulo (FEA̙USP), Sao Paulo, Brazil)
Luis Hernan Contreras Pinochet (Paulista School of Politics, Economics and Business, Federal University of Sao Paulo (UNIFESP), Osasco, Brazil)
Adriana Backx Noronha Viana (School of Economics, Business, and Accounting, University of Sao Paulo (FEA̙USP), Sao Paulo, Brazil)
Cesar Alexandre de Souza (School of Economics, Business, and Accounting, University of Sao Paulo (FEA̙USP), Sao Paulo, Brazil)

International Journal of Information and Learning Technology

ISSN: 2056-4880

Article publication date: 14 March 2023

Issue publication date: 17 March 2023

657

Abstract

Purpose

Education is undergoing digital transformation intensified by COVID-19. In this context, gamification is an attractive alternative based on the use of elements of the games with educational purposes. However, it keeps the educational content to be learned as a central element without neglecting the “fun,” which contributing to engaging students. Therefore, this study aims to analyze the factors that affect students' engagement in an undergraduate course of Business Administration with gamified activities in remote education.

Design/methodology/approach

The authors collected data through a survey available to students of the administration course at a private university in São Paulo (n = 671). This study used a quantitative approach, using SEM with PLS estimation and with the support of other analytical techniques.

Findings

The results support all the hypotheses formulated. Those with the associated construct “competition” obtained the most robust relationships, which denotes that competition is an essential element in gamification. Despite being supported by the results, “network exposure” influencing engagement is one point of attention to improving teaching strategies.

Research limitations/implications

Graduate schools could implement this type of gamified activity, evaluating whether students enrolled in higher degrees would willingly engage in a learning activity considered “less serious.”

Practical implications

Higher education institutions can benefit from this study by understanding that gamification is presented as an active methodology that increases students' engagement in teaching.

Originality/value

This research addressed gaps in the factors that affect students' engagement with gamified activities, proposing an alternative theoretical model to those present in the literature.

Keywords

Citation

Pardim, V.I., Contreras Pinochet, L.H., Viana, A.B.N. and Souza, C.A.d. (2023), "Where is the student who was here? Gamification as a strategy to engage students", International Journal of Information and Learning Technology, Vol. 40 No. 2, pp. 177-192. https://doi.org/10.1108/IJILT-05-2022-0122

Publisher

:

Emerald Publishing Limited

Copyright © 2023, Emerald Publishing Limited


Introduction

Education is undergoing digital transformation intensified by COVID-19. Institutions at different levels of education were forced to enter the world of remote education, which is not new. However, that brought even more challenges for those involved, not only from a technological perspective due to the insufficiency or lack of adequate infrastructure but, above all, the need to give opportunities for an attractive and engaging learning environment for students (Tan et al., 2018; Nieto-Escamez and Roldán-Tapia, 2021). In literature, we find different ways to make learning more engaging, and one of them is using games. Adopting games, especially digital games, as a vehicle for learning, is known as game-based learning (Tan et al., 2018).

The types of existing games are commercial games and serious games. Commercial games are designed for the primary purpose of entertainment, but can also be used for educational purposes, provided that their use is controlled (Connolly et al., 2012; Filippou et al., 2018). The main aspect that makes games attractive in education is their playful character, as they are fun and enjoyable to use. This can lead to students' engagement with their learning process, an essential element in remote education. This engagement is vital in classes that lecture through video conferencing platforms (Google Meet, Zoom, etc.), a recurring practice before the pandemic (Sailer et al., 2017; Subhash and Cudney, 2018). Sometimes there is a low level of interactivity in these digital learning environments since students adopt a passive posture and become a spectator of their learning process. The silent presence in this class format (camera and audio turned off and inactivity in the chat) is a cause for concern for teachers since the low level of engagement can compromise academic performance (Nieto-Escamez and Roldán-Tapia, 2021).

Tan et al. (2018) state that learning through games provides an emotion not present in traditional education. The challenge is to manipulate games to meet an educational purpose since it is not easy to align the purpose of a commercial game to learning objectives, and it is easy to develop a serious game, depending on the cost involved. Thus, gamification is an attractive alternative since it uses elements, mechanics and logic of games but keeps educational content as a central element without leaving aside the fun (Fu et al., 2009; Filippou et al., 2018; Roy and Zaman, 2018) to engage students (Rojas-López et al., 2019; Ding et al., 2018). Engagement is students' affective, cognitive and behavioral involvement in learning activities (Kahu, 2013; Angelino et al., 2021).

Considering that gamification can take various forms by combining different elements of the games, it would be imprudent to study the factors that lead to engagement as a generic construction (Sailer et al., 2017). Despite its importance and several studies on the subject (Hamari et al., 2016; Zainuddin et al., 2020; Bai et al., 2020), it is still necessary to conduct research that seeks to understand the factors that affect engagement with gamified activities, to improve the understanding of how it develops (Hamari et al., 2016), which makes this study timely and meaningful.

This article addresses existing gaps in the factors that affect the engagement of students with gamified activities, one of the most important constructs, but which still requires studies that seek to understand how it develops, especially in the context of digital learning, as is the case of remote education (Hamari et al., 2016; Schöbel et al., 2021).

This article aims to analyze the factors that affect students' engagement in an undergraduate course of Business Administration with gamified activities in remote education. These factors are considered one of the most important to effectively use gamification in the learning process (Schöbel et al., 2021) in higher education. This study's goal is to examine if gamification can be a useful method to enhance the engagement of students (Szendrői and István, 2022). Therefore, the researchers' questions addressed in this paper are:

RQ1.

What factors affect the engagement of students with gamified activities?

RQ2.

How does gamification promote learning and improves students' academic performance?

Understanding this phenomenon is important because it can reduce many students' demotivation when submitted to a traditional teacher-centered teaching and learning process. This work can help higher education institutions and their actors, especially management courses, explore the potential that gamification presents to promote learning that engages and improves students' academic performance.

This article contributes to the literature by proposing a theoretical model for understanding gamification applied in the classroom context of higher education. In addition, it contributes to fostering the engagement of university students in the Business Administration course but is not limited to this level of education. Although made possible during the pandemic, the initiative can be applied in the “new normal” scenario in courses offered in face-to-face, distance, or hybrid models. Therefore, it was necessary to understand the models that stand out in the literature and serve as guides of practical applications of gamification for the development of activities (Hamari and Koivisto, 2015; Suh and Wagner, 2017; Filippou et al., 2018; Park and Kim, 2020).

Literature review and construction of the theoretical model

Gamification and remote higher education

Games as a source of fun are prevalent among people of different age groups and gender (Bai et al., 2020). As for its emergence, although uncertain, there is evidence dating back hundreds of years before the Christian era, but it gained prominence in the early 2000s due to the rise of digital games (Deterding et al., 2011).

Gamification emerges as the use of game design elements in a context other than play but in real situations such as work, crowdsourcing, data collection, health, marketing and education, among others, to motivate specific behaviors (Seaborn and Fels, 2015; Suh and Wagner, 2017; Sailer et al., 2017; Subhash and Cudney, 2018; Schöbel et al., 2021).

Rojas-López et al. (2019) share the same understanding of gamification. They add that in the context of education, introducing elements, design and context of play in the design of the learning process of students, regardless of the area of knowledge or level, not only aims at constructing knowledge, but also developing transversal skills and attitudes such as collaboration, self-regulation of learning and creativity.

In addition to allowing them to learn from their mistakes, the feedback is immediate. The definition presented omitted the objective of gamification that, according to Deterding et al. (2011) and Sailer et al. (2017), is based on four semantic components: game (rules-based and goal-oriented), elements (adoption of part of the characteristics, rather than developing a complete game, such as serious games), design and context (not game-related, regardless of intent to use).

Werbach and Re (2014) claims that what characterizes gamification is the selection, application, implementation and integration of game design elements rather than using them in isolation. For example, Kahoot, Socrative, or Quizizz does not mean it is a gamified activity (Sanchez et al., 2020). In this study, we used Quizizz, which, similarly to Kahoot, is a game-based learning tool used to foster the motivation and engagement of students with gamified activities, promoting learning, empathy and the ability to work as a team (Tan et al., 2018; Zainuddin et al., 2020).

However, applying tools is insufficient to achieve an effective gamification “fun” experience (Szendrői and István, 2022). Thus, the element of surprise must complement these elements to achieve motivation and, consequently, engagement in the participation of members.

Engagement in gamified activities

Students engaged in gamified activities developed through e-quizzes can experience the feeling of personal presence as in the real world (Calleja, 2007; Malliet, 2006; Ros et al., 2020). Therefore, a proper design, with tools that entertain, can provide students confidence in conducting the activity. Student engagement and context are positively related to performance (in the case of this article, the score, ranking and podium), which are positively related to learning (Grau-Valldosera and Minguillón, 2014; Domínguez et al., 2013; Ros et al., 2020).

Engagement occurs through the fun provided by the game mechanics (Filippou et al., 2018). It is considered an attribute that influences students' ability to learn and academic performance, and they tend to perform better results in gamified activities than those in traditional activities (Hamari et al., 2016). Engaged students pay more attention, are more interested in the content and materials provided and are curious to learn more (Rahman et al., 2019). Finally, students engaged in the dynamics of the competition process and, consequently in studies, dedicate more time to developing skills that will be useful at later moments in life, which is one of the objectives of the formal educational process (Filippou et al., 2018).

Following are the antecedent engagement factors with gamified activities we present to clarify which elements are more related to engagement issues within a perspective of competition between groups of students in the game.

Antecedent factors of student engagement with gamified activities

Anxiety (AN)

Adopting technologies to promote gamified activities can cause negative emotional effects in students that arise, not only during interaction but when the idea of having to interact in a new process begins. Anxiety and similar emotional states can adversely affect the experience of use and productivity, learning, social relationships and well-being (Saadé and Kira, 2009). Thus, anxiety related to the use of technologies, in the case of this article Quizizz, is the feeling of fear or apprehension when using or considering using them. Anxiety in studies of information systems issues has been seen as a personality variable that influences the use of the system. Thus, due to the use of technological resources, a highly anxious person would be at a significant disadvantage compared to his peers, affecting his learning process (Saadé and Kira, 2009).

Experience (EX)

This construct was adapted from Soto-Acosta et al. (2014), who analyzed the experience of using the internet. However, in this article, the experience is in gamification. In both cases, there is a need to identify the individual's ability developed from using various services that provide conditions for their learning (Nysveen and Pedersen, 2004). Students' experience with gamified activities is crucial to understand the perceptions, attitudes and behavior in this environment. Specifically, experience contributes to the more effective use of tools, leading more experienced users to demonstrate more positive attitudes when using technologies (Chen and Macredie, 2005; Soto-Acosta et al., 2014). Individuals who consider themselves empowered to use e-quizzes in gamified activities may like to explore technologies, while their attitudes may undergo significant changes (López-Nicolás et al., 2008). Thus, gamification through these digital resources can provide a successful experience through the “fun” that the game phenomenon can provide (Koivisto and Hamari, 2014; Szendrői and István, 2022) (Koivisto and Hamari, 2014; Szendrői and István, 2022).

Competition (CO)

Through competition, students can feel that they interact with other colleagues and develop the team's strategy with autonomy to self-regulate in performing the activities (Deterding et al., 2011; Liu et al., 2013; Hamari et al., 2016). Empirical evidence shows that competition in an autonomous context makes participants resilient to the feeling of failure, leading them to develop a sense of self-efficacy and encouraging them to engage in an immersive process to achieve higher scores. This study proposes that the experience (Soto-Acosta et al., 2014) moderates the relationship between competition and engagement. The increase in experience stimulates the desire to overcome challenges by expanding engagement, which can bring negative feelings. This is explained by the fact that games require individuals to work as a team to win. Good results mean the recognition of peers and those who lead the game, which will trigger a sense of personal fulfillment, raising the level of motivation between individual and group. The above led to the formation of the following hypotheses:

H1a.

Competition positively affects engagement.

H1b.

Competition positively affects immersion.

H1c.

Immersion-mediated competition positively affects engagement.

H1d.

The moderating effect of the experience negatively affects the relationship between competition and engagement.

Network exposure (NE)

The exposure in the network considers the perception of the magnitude of the phenomena and the space destined to the network as relevant. The space that the individual is inserted in the network can affect the amount of social activity conducted. The number of people who use a service, in this case, a gamified activity, can be considered important because there is a concentration of social interaction (Baker and White, 2010; Lin and Lu, 2011). This research considers that the effects of network exposure on engagement are mediated by operationalized social factors, which in this study is characterized by the competition of students to achieve higher scores that lead them to a good ranking at the end of the process. It provides valuable benefits (Mäntymäki and Islam, 2014) and, consequently, can lead to greater engagement with gamified activities (Hamari and Koivisto, 2015). The number of students and working groups who compete for a better position in the ranking and know to occupy a place on the podium can make the gamified activity more attractive and competitive. However, it can arouse aspects related to emotions, such as anxiety. Thus, this study proposes that anxiety (Saadé and Kira, 2009) moderates the relationship between network exposure and competition since competition requires greater integration between peers for the activity to occur in the network. Therefore, the following hypotheses arise:

H2a.

Exposure to the network positively affects the competition.

H2b.

Exposure to the network positively affects engagement.

H2c.

Exposure in the network mediated by competition positively affects engagement.

H2d.

The moderating effect of anxiety negatively affects the relationship between network exposure and competition.

Immersion (IM)

Within gamified learning, immersion occurs when the student is absorbed by the activity (Goethe, 2019). If a gamified activity creates these conditions, it is reasonable to expect the student to consider the helpful tools in the evaluation process. It will allow him to be completely absorbed into something pleasant. In summary, immersion will influence engagement to determine whether students understand that gamified activity is useful (Filippou et al., 2018). Therefore, we develop the following hypothesis:

H3.

Immersion positively affects engagement.

Figure 1 presents the theoretical model proposed in this research and the direct, indirect (mediated) and moderating relationships between the constructs.

Method

Participants and data collection

We collected data via an electronic questionnaire made available to students of the Business Administration course who participated in the gamified activity at a private university in São Paulo, Brazil. Before the collection, we performed a pre-test with the class students (n = 80). We obtained 821 valid questionnaires, and convenience is the sample's selection criterion. In the treatment and purification of data, the research instrument first used the missing data control, making it mandatory to fill out all the questionnaire items. Subsequently, we utilized Mahalanobis distance (D2) to remove 150 outliers, resulting in a final sample of n = 671 respondents. The sample consisted of 39.3% (n = 264) men and 60.7% (n = 407) women. Both groups with an average age of 25 years.

Research instrument

The antecedent factors of engagement were selected from a bibliometric analysis and a preliminary meta-analysis to identify which constructs would be more related to engagement issues within the perspective of competition between groups of students in the game. The scales selected for this study were adapted from studies validated in the literature and had practical relevance in aspects adherent to this study. The aspects are engagement (Filippou et al., 2018), anxiety (Saadé and Kira, 2009), competition (Suh and Wagner, 2017), network exposure (Hamari and Koivisto, 2013), experience (Soto-Acosta et al., 2014) and immersion (Fu et al., 2009; Filippou et al., 2018).

We structured the research questionnaire into 26 items, and each variable was measured using a 7-point Likert-type scale (1-totally disagree to 7-totally agree). All operationalizations of psychometric constructs were adapted from previously published sources and underwent a reverse translation process, validated by three senior researchers in the area and can be observed in the Table A1 (see Supplementary file). The research used a quantitative approach, with multivariate analysis, using the Structural Equation Modeling (SEM) technique, specifically the Partial Least Squares estimation (PLS).

Results

Measurement

We evaluated the measurement model for the entire sample to operationalize the convergent and discriminating validity of the constructs. We examined the convergent and discriminant validities for each of the 26 indicators. This phase excluded three variables from the Engagement construct (EN2, EN3 and EN6) that presented factor loadings lower than 0.5. As Table S1 (see Supplementary file) demonstrates, all seven constructs exceeded the minimum required level of criteria in terms of Cronbach's Alpha (CA), Composite Reliability (CC) and Average Variance Extracted (AVE). As a result, all reported CA were above 0.70, and the CC scores were higher than 0.70, indicating that all constructs were satisfactory in the reliability analysis (Hair et al., 2012).

The AVEs for each construct were greater than 0.50, indicating that convergent validity was also considered acceptable (Hair et al., 2012). We evaluated discriminant validity by examining factor loadings and correlations between constructs. The square root of the AVE for each construct was considered greater than each correlation between construct coefficients, confirming discriminant validity.

Structural model

We calculated the coefficients of determination (R2) to estimate the predictive accuracy of the structural model (Hair et al., 2012) and to explain the effects of each construct [engagement: R2 = 0.794 considered high; competition: R2 = 0.500 and immersion: R2 = 0.376 considered moderate] based on Chin (1998) criterion. Bootstrapping is a nonparametric method that includes many new samplings (in this case, 5,000) to evaluate the form of the sampling distribution of a statistic (Chin et al., 2003). For relationships, we utilized bootstrapping to calculate the evaluations of the statistical paths with the t-test. In this study, the authors performed a resampling process to evaluate whether the effects of direct, indirect (mediating) and moderating relationships were significant (Hair et al., 2012). Table S2 (see Supplementary file) presents the results of the partial least squares (PLS) technique, identifying that all hypotheses (direct, indirect (mediated) and moderators) were accepted.

Indirect relationships indicated that the mediating effects were partial, considering that both direct and indirect relationships were significant. In Figure S3 (see Supplementary file), anxiety moderated the association between network exposure and competition when analyzing the moderation. The greater the anxiety, the greater the negative effect of network exposure in competition (β = −0.264; p < 0.001). Similarly, in Figure S4 (see Supplementary file), the experience moderated the association between competition and engagement so that the greater the experience, the greater the negative effect of competition with engagement (β = −0.140; p < 0.001).

Discussion

This study's findings indicate that the students of the business administration course engaged (dependent variable) with the gamified activity developed in the discipline “Organizational Transformation Management” that involved the game design elements, such as scoring, ranking, podium, choice of cards, extra challenges and competition (see Figure S1 in the Supplementary file). The authors created the proposed activity to generate new forms of interaction with students to seek engagement in the remote teaching environment (see Figure S2 in the Supplementary file). This interaction is beneficial since silence prevailed along with a high rate of non-participation in the non-gamified activities and perceived apathy during attempts to discuss throughout the classes.

As a result, all hypotheses with direct, indirect (mediated) and moderate relationships converged with this purpose. We observed that the hypotheses associated with the construct “competition” obtained higher standardized coefficients (β) values (H1a, H1b and H2a). This result denotes that competition is an essential element in the gamification process, providing visible incentives for positive behaviors on the part of students regarding their learning process, and this finding is in line with other studies on the subject (Deterding et al., 2011; Zainuddin et al., 2020).

In addition to the construction of knowledge, competition also contributes to the development of transversal skills and attitudes, such as collaboration, stimulated by the adoption of the ranking and podium not individually but in groups (Rojas-López et al., 2019). However, although all hypotheses have been accepted, H1b (β = 0.613; p < 0.001) emphasizes understanding the process of engagement with gamification. H1b presents a high difference concerning β, indicating that competition is positively and strongly influencing immersion in activities within the gamified context and, consequently improving its performance (Liu et al., 2013). For example, in class 6, one of the highest scores in the game, in a non-gamified activity that preceded the first round of the Quiz and developed individually, of the 165 students, only 123 participated, and 42 had to be replaced. In the gamified activity that occurred the following week, 160 students participated; only five had to be replaced, reinforcing students' engagement with the activity.

Of all the hypotheses, H2b (β = 0.083; p < 0.05) with the path “network exposure engagement”, even though accepted, had the lowest significance. This can be explained by the fact that these students develop this activity remotely and through mobile phones. This process makes it difficult, for example, to access the classroom environment (Google Meet) and the development of an activity that, even as a team, requires each student's engagement in the team's achievements (Baker and White, 2010; Lin and Lu, 2011; Hamari and Koivisto, 2015).

We observed interesting results in the hypotheses that moderated this research, anxiety and experience. In the case of anxiety (H1d), we detected that during the development of the gamified activity, the more exposed in the network, the higher the level of competition due to the clear need to participate in the activities. Even so, we observed a negative feeling due to the feeling of failure that can occur in the dissemination of the ranking/podium, especially if the group is in the last position. This generates a feeling that the student/group may not be able to achieve the expected results, so we created the letters and extra challenges to keep alive the hope that new positions can be achieved (Saadé and Kira, 2009).

Similarly, the experience (H2d), moderating the relationship between competition and engagement, brought a negative feeling because the student, with each new round, expanded his experience, which can reduce the feeling of initial euphoria with the innovative aspect proposed in the gamified activity (López-Nicolás et al., 2008).

Conclusions

Theoretical and practical implications of research

This article reports the possibility of analyzing the factors that affect the engagement of students of the administration course with gamified activities in remote education. This research indicates that students prefer using gamified activities in this context of classes via the conference platform. This was reflected in the increased participation via chat (Google Meet) and activities during classes when we applied gamified activities. The theoretical model of the research identified constructs considered favorable to understanding the process of engagement in classes.

When compared with other approaches, we observed that the implications of measuring preference for gamification in education helped lay a foundation for broader implementations of gamification in learning and teaching practice. Thus, this work contributes by producing a theoretical model different from those present on the subject. For example, Hamari et al. (2016) assessed engagement as being affected only by challenge and skill, and the study by Bai et al. (2020), who sought to understand only the effect of ranking (absolute and relative) on engagement, but whose hypotheses were not supported. Finally, to prove this study, it is from the literature to advance and be a predictor of engagement in the study by Filippou et al. (2018). Therefore, the innovation in this article is outside the proposed model and brings advances to the implementation of gamification.

Gamification applied in education should be considered and explored. In remote education, people are increasingly present in environments where technology and digital media stand out; therefore new approaches and strategies must be adopted to engage students. This is primarily because many students are discouraged by more “conservative” methodologies. Thus, gamification brings new ways to enable the learning process actively, breaking with traditional models centered on the teacher and placing the student at the center of this process.

Besides being the protagonist, the student learns from the errors throughout the process, no longer needing to wait until the end to evaluate contents worked on throughout the semester. Another important aspect is that gamification contributes to student retention by involving them in a range of activities that use elements from game design, making learning pleasurable and challenging. This learning style causes students to reach a state of flow, which leads them to immerse themselves within the context of the learning experience. Thus, higher education institutions and their actors can benefit from this study by understanding that gamification is presented as an active methodology that increases students' engagement in teaching.

Limitations and suggestions for future research

There is room for future research to validate students' preference for gamified activities. First, this strategy could be applied in disciplines with different characteristics within the same course to verify the adequacy, leading to greater student engagement. In addition, adopting it in several disciplines in the same semester could cause an opposite effect by the saturation of the model.

The article does not propose to discuss the attributions of the students and the teacher but to present the process of analyzing each of the causal relationships of the proposed model inserted in the proposal to highlight the “engagement” of students with the implemented gamification strategies. Therefore, we suggest future research using the proposed model. Furthermore, we suggest that other personality constructs (e.g. psychological disorders, personality traits, etc.) be tested and validated in future models.

It is crucial to identify to what extent active methodologies can be employed as part of a decision of each teacher independently or whether it should be something interdisciplinary articulated during the planning of the semester. This will allow them to identify if the constructs of this study apply to a wide range of students with different sets of skills and academic interests depending on the nature of diverse disciplines. Second, more complex models, for example, with the insertion of more game features and the creation of a smart software/application, could be explored and examined. Finally, this type of gamified activity could be adapted to graduate school, evaluating whether students enrolled in higher degrees would moderate students' willingness to engage in a learning activity that can be considered “less serious.”

Figures

Theoretical model

Figure 1

Theoretical model

Format of activities

Figure S1

Format of activities

Playful stimuli

Figure S2

Playful stimuli

Interaction H1d graph (slope analysis)

Figure S3

Interaction H1d graph (slope analysis)

Interaction H2d graph (slope analysis)

Figure S4

Interaction H2d graph (slope analysis)

Convergent and discriminating validity

ConstructsCACCAVECorrelation of constructs
(1)(2)(3)(4)
(1) Competition0.8780.9250.8060.898
(2) Engagement0.9030.9390.8380.8240.915
(3) Network Exposure0.8600.9050.7050.6380.6500.839
(4) Immersion0.9180.9390.7550.6130.7290.5550.869

Note(s): The square roots of AVEs represent the italic diagonal elements in the matrix correlation

PLS technique results in hypotheses

HypothesisPathsβBootstrapping (5,000 samples)Standard deviationt-test
Direct relationships
H1aCompetition Engagement0.3630.3610.0487.522***
H1bCompetition Immersion0.6130.6130.03218.964***
H2aNetwork Exposure Competition0.4880.4870.03314.930***
H2bNetwork Exposure Engagement0.0830.0840.0273.064**
H3Immersion Engagement0.2710.2700.0328.600***
Indirect relationships (Mediations)
H1cCompetition Immersion Engagement0.1660.1650.0208.214***
H2cNetwork Exposure Competition Engagement0.1770.1760.0276.446***
Moderations
H1dMod (Experience)*Competition Engagement−0.140−0.1410.0226.426***
H2dMod (Anxiety)*Network Exposure Competition−0.264−0.2650.0367.303***

Note(s): **p < 0.05; p < 0.001

Description of model scales

Construct (References)ItemItem descriptionFactor loadingsMean (SD)
Anxiety
Adapted from Saadé and Kira (2009)
AN1I am apprehensive about participating in the game0.8564.741(1.996)
AN2It scares me to think I might not be able to participate in the game if my Internet connection fails0.8695.259(1.844)
AN3I hesitate to participate in the game to make mistakes and not respond correctly to the alternatives0.7093.659(2.354)
AN4The competition between the groups in the game is somewhat intimidating for me0.7003.741(2.304)
Competition
Adapted from Suh and Wagner (2017)
CO1In the game, I can compete with other groups0.9416.402(1.141)
CO2In the game, I can compare the performance of my group with the other0.9476.344(1.157)
CO3In the game, I can threaten the status of other groups with the active participation of my group0.7975.848(1.445)
Network Exposure
Adapted from Hamari and Koivisto (2013)
NE1I have many colleagues who are following my group's performance in the game0.8345.642(1.558)
NE2My group follows my activities in the game0.8735.924(1.410)
NE3I am aware of the performance of the other groups in the game0.8165.702(1.459)
NE4I have been very related to colleagues in the game0.8336.148(1.314)
Experience
Adapted from Soto-Acosta et al. (2014)
EX1I consider myself highly skilled at using the technological tools of the game0.9075.979(1.261)
EX2I consider myself knowledgeable of good strategies to participate in the game0.9305.833(1.319)
EX3I know about the game more than most classmates0.6804.675(1.862)
EX4I know how to find out the answers in the game0.6004.568(2.047)
Immersion
Adapted from Fu et al. (2009) and Filippou et al. (2018)
IM1I don't understand the time passing while I'm participating in the game0.7685.779(1.629)
IM2I forget what's around me while I'm playing the game0.8975.556(1.590)
IM3I temporarily forget the everyday worries while I'm participating in the game0.9075.614(1.544)
IM4I can get involved in the game0.9065.939(1.305)
IM5I feel emotionally involved in the game0.8595.705(1.508)
Engagement
Adapted Filippou et al. (2018)
EN1I want to complete the game0.9066.306(1.214)
EN2I want to take advantage of all the possibilities that the game offers**
EN3I found the game satisfying**
EN4I was focused during the rounds of the game0.9146.191(1.213)
EN5I felt that time passed fast during the game0.9276.310(1.209)
EN6I was excited during the rounds of the game**

Note(s): *Items deleted in the model adjustment phase

Supplementary file

Presentation of the ‘Chuchu Game’

The gamified activity analyzed in this study was developed in the discipline “Organizational Transformation Management”, the curricular component of the Administration course, at a private university in São Paulo (n = 671).

The gamified activity ‘Chuchu Game” was developed over six classes (see Figure S1). This aspect is at the level of principles and heuristics of the game design by Deterding et al. (2011). Students from 8 classes (5 at night and 3 in the daytime) participated in this study; the same teacher taught the discipline in all classes. Considering the course's offer period, the second half of 2020, the classes taken remotely due to the COVID-19 pandemic, based on the Google Meet conferencing platform.

Quizizz was used to enable the strategy adopted in making available to students a gamified activity. Through this tool, the student can track their performance through the recording, tracking and maintenance of the individual points obtained by correcting the proposed question and the speed with which they managed - time restriction - aspect related, respectively, to the design patterns of the game interface and design mechanics (Deterding et al., 2011). The teacher can also generate reports and export the scores through a spreadsheet.

A limitation of this tool is the number of groups that can be created free of charge, 8 in total, which was insufficient for the number of students, approximately 170, divided, on average, into 20 groups, which made it impossible for the calculate of the score of each group to be performed automatically by the tool. In addition, the game had additional activities such as the choice of cards and extra challenge. So, the formula adopted was Round = Σ of the score of the three students with the highest score + the score of all members +200 points if the group is complete + card score (when applicable) + extra challenge score (these last two elements tied to the level of the game model). The same formula was used in each of the rounds, the difference was that from the second onwards, there was the accumulation of the score of the previous round. This scoring system allowed working with groups in varying sizes (minimum of 6 and maximum of 15 members, but without this, it represented any burden on the score).

The moment of the assembly of the groups was the most delicate because the students had to leave a passive and solitary posture and exercise, among others, the ability to work as a team with people they did not know and with whom they had not yet interacted. To bring the students closer together and, at the same time, to assemble the groups, a case study was applied to interact and get to know each other. At the time of the composition of the groups, a Padlet has used in which a student was a representative, and the other ones were part of the group, freely, if it did not exceed the maximum number (n = 15) and, mainly, that no group was less than six members. Thus, providing a moment for them to build group identity was possible.

Students were creating strategies with their group mates during each round to improve collective performance, culminating, for example, in creating WhatsApp groups to exchange ideas during the rounds. The moment of the ranking/podium presentation was long-awaited by the students. It occurred through the projection of two slides: (1) with the podium, which contained the name of the groups that occupied the first three positions and then the table with the overall score of all groups in the room (from highest to lowest score), what, according to Deterding et al. (2011), would be related to the design patterns of the game interface.

The groups that occupied the top three positions in the first and second rounds acquired the right to choose a different card in each round. In the last (3rd round), all groups acquired the right to choose a card. This additional activity was incorporated as a playful stimulus (see Figure S2) – each card with its distinctive characteristic could lead to a new rearrangement of leadership between the groups since they could draw a card that increased the points of the group itself or the points of the colleagues. It is in line with what Werbach (2014) puts because, in this proposal, gamification was not restricted to the introduction of Quizizz, but rather the selection, application, implementation and integration of game design elements such as cards, ranking, podium and extra challenge.

The instant feedback of the achievements in each of the rounds of the “Chuchu Game” allowed monitoring of the students' understanding throughout the classes in a context of formative evaluation that indicates that the evaluation occurs in the interaction between student-teacher, student-content and student-student (Zainuddin, 2020).

The sample consisted of 39.3% (n = 264) men and 60.7% (n = 407) women. Both groups with an average age of 25 years. After the homogeneity of the regression parameters was confirmed, a covariance analysis (ANCOVA) was conducted to verify whether the classes (cluster variable) would have an effect on the overall score obtained after the gamified activity and whether three other variables (comparison – comparing with more traditional classes, how much gamified activities favor learning the educational contents worked in class; level of complexity – to answer questions; and engagement – variable dependent on the structural model) would influence as covariates. ANCOVA revealed that there is an effect of covariates on the score [comparison: F(1, 661) = 6.350; p = 0.012; complexity level: F(1, 661) = 18.102; p < 0.001; engagement: F(1, 661) = 12.454; p < 0.001]. In addition, there is an effect of the classes on the score, after the control for the effect of the covariates - comparison, level of complexity and engagement [F(6, 661) = 7.819; p < 0.001]. The covariates that were present in the model and that were evaluated in the values (comparison (x = 6.50); complexity level (x = 5.31); engagement (x = 6.27); with an error bar of 95% confidence interval).

Another point to highlight is that all the mean scores in the gamified activity were considered adequate due to their self-regulation within the collaborative learning process. This gamified activity had weight three over the total grade in the discipline and classes 1 (x1= 13,779.93) and 6 (x6= 13,661.71) were the highest scores. Classes 7 (x7= 11,665.40) and 8 (x8 = 12,131.90) were allocated on Fridays and felt motivated by this type of strategy, thus enabling the retention of students in the remote class via Google Meet. Another important fact is that the overall average of the 8 classes was 8.7 (with six the minimum score required for approval), and the worst average was 7.96.

The scenario was different in the first half, which began in person and then went to the remote format. The average of the classes was 7.0, but only after several opportunities to spare the activities lost throughout the semester and by applying extra activities to leverage the grades. What was evident at the end of the semester was the difficulty of the students to remain engaged during the classes, since, in this new format for them, the fact that they could have camera and microphone turned off, but still obtain presence because they were logged into the system, made participation during classes dispensable. Added to this are the possibility of resuming the lost activities and the absence of team activities due to adaptation. In addition, an intergroup analysis (t-test with independent samples) was carried out with the means of the constructs selected in the model with the variable gender control category (1-male and 2-female), the following results were obtained: students belonging to the male public are more anxious (t(669) = 3.171; p = 0.002) and experienced (xan=4.61; t(699) = 2.712; p = 0.007; xex=5,43) than the female. When analyzing competition, network exposure and immersion, it is observed that both male and female groups present equivalent behaviors. Finally, the female group showed greater engagement with gamified activity (t(669) = 2.113; p = 0.035; xen=6.34).

Appendix

References

Angelino, F.J.de A., Loureiro, S.M. and Birlo, R.G. (2021), “Analysing students' engagement in higher education through transmedia and learning management systems: a text mining approach”, Innovation and Learning, Vol. 30 No. 4, pp. 484-502.

Bai, S., Hew, K.F. and Huang, B. (2020), “Is gamification ‘bullshit’? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts”, Educational Research Review, Vol. 30, 100322.

Baker, R.K. and White, K.M. (2010), “Predicting adolescents' use of social networking sites from an extended theory of planned behaviour perspective”, Computers in Human Behavior, Vol. 26 No. 6, pp. 1591-1597.

Calleja, G. (2007), “Digital game involvement”, Games and Culture, Vol. 2 No. 3, pp. 236-260.

Chen, S.Y. and Macredie, R.D. (2005), “The assessment of usability of electronic shopping: a heuristic evaluation”, International Journal of Information Management, Vol. 25 No. 6, pp. 516-532.

Chin, W.W. (1998), “The partial least squares approach for structural equation modeling”, in Marcoulides, G.A. (Ed.), Modern Methods for Business Research, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, pp. 295-336.

Chin, W.W., Marcolin, B.L. and Newsted, P.R. (2003), “A partial least squares latent variable modeling approach for measuring interaction effects: results from a Monte Carlo simulation study and electronic-mail emotion/adoption study”, Information Systems Research, Vol. 14 No. 2, pp. 189-217.

Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T. and Boyle, J.M. (2012), “A systematic literature review of empirical evidence on computer games and serious games”, Computers and Education, Vol. 59 No. 2, pp. 661-686.

Deterding, S., Dixon, D., Khaled, R. and Nacke, L. (2011), “From game design elements to gamefulness: defining gamification”, Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, New York, NY, Tampere, pp. 9-15.

Ding, L., Er, E. and Orey, M. (2018), “An exploratory study of student engagement in gamified online discussion”, Computers and Education, Vol. 120 No. 1, pp. 213-226.

Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C. and Martínez-Herráiz, J.J. (2013), “Gamifying learning experiences: practical implications and outcomes”, Computers and Education, Vol. 63, pp. 380-392.

Filippou, J., Cheong, C. and Cheong, F. (2018), “A model to investigate preference for use of gamification in a learning activy”, Australasian Journal of Information Systems, Vol. 22, pp. 1-23.

Fu, F.L., Su, R.C. and Yu, S.C. (2009), “EGameFlow: a scale to measure learners' enjoyment of e-learning games”, Computers and Education, Vol. 52 No. 1, pp. 101-112.

Goethe, O. (2019), “Immersion in games and gamification”, Gamification Mindset, Springer International Publishing, pp. 107-117.

Grau, J. and Minguillón, J. (2014), “Rethinking droput in online higher education: the case of the universitat oberta de Catalunya”, International Review of Research in Open and Distance Learning, Vol. 15 No. 1, pp. 290-308.

Hair, J.F., Sarstedt, M., Ringle, C.M. and Mena, J.A. (2012), “An assessment of the use of partial least squares structural equation modeling in marketing research”, Journal of the Academy of Marketing Science, Vol. 40 No. 3, pp. 414-433.

Hamari, J. and Koivisto, J. (2013), “Social motivations to use gamification: an empirical study of gamifying exercise”, Proceedings of the 21st European Conference on Information Systems, Utrecht, pp. 5-8.

Hamari, J. and Koivisto, J. (2015), “‘Working out for likes’: an empirical study on social influence in exercise gamification”, Computers in Human Behavior, Vol. 50, pp. 333-347.

Hamari, J., Shernoff, D.J., Rowe, E., Coller, B., Asbell-Clarke, J. and Edwards, T. (2016), “Challenging games help students learn: an empirical study on engagement, flow and immersion in game-based learning”, Computers in Human Behavior, Vol. 54, pp. 170-179.

Kahu, E.R. (2013), “Framing student engagement in higher education”, Studies in Higher Education, Vol. 38 No. 5, pp. 758-773.

Koivisto, J. and Hamari, J. (2014), “Demographic differences in perceived benefits from gamification”, Computers in Human Behavior, Vol. 35, pp. 179-188.

Lin, K.-Y. and Lu, H.-P. (2011), “Why people use social networking sites: an empirical study integrating network externalities and motivation theory”, Computers in Human Behavior, Vol. 27, pp. 1152-1161.

Liu, D., Li, X. and Santhanam, R. (2013), “Digital games and beyond: what happens when players compete”, MIS Quarterly, Vol. 37 No. 1, pp. 111-124.

López-Nicolas, C., Molina Castillo, F. and Bouwman, H. (2008), “An assessment of advanced mobile services acceptance: contributions from tam and diffusion theory models”, Information and Management, Vol. 45 No. 6, pp. 359-364.

Mäntymäki, M. and Islam, A.K.M.N. (2014), “Social virtual world continuance among teens: uncovering the moderating role of perceived aggregate network exposure”, Behaviour and Information Technology, Vol. 33 No. 5, pp. 536-547.

Malliet, S. (2006), “An exploration of adolescents' perceptions of videogame realism”, Learning, Media and Technology, Vol. 31 No. 4, pp. 377-394.

Nieto-Escamez, F. and Roldán-Tapia, M.D. (2021), “Gamification as online teaching strategy during COVID-19: a mini-review”, Frontiers in Psychology, Vol. 12, p. 648552.

Nysveen, H. and Pedersen, P.E. (2004), “An exploratory study of customers' perception of company web sites offering various interactive applications: moderating effects of customers' internet experience”, Decision Support Systems, Vol. 37 No. 1, pp. 137-150.

Park, C. and Kim, D. (2020), “Exploring the roles of social presence and gender difference in online learning”, Decision Sciences Journal of Innovative Education, Vol. 18 No. 2, pp. 291-312.

Rahman, R.Ab., Ahmad, S. and Hashim, U.R. (2019), “A study on gamification for higher education students’ engagement towards education 4.0”, Lecture Notes in Networks and Systems, pp. 491-502.

Rojas-López, A., Rincón-Flores, E.G., Mena, J., García-Peñalvo, F.J. and Ramírez-Montoya, M.-S. (2019), “Engagement in the course of programming in higher education through the use of gamification”, Universal Access in the Information Society, Vol. 18, pp. 583-597.

Ros, S., Gonzalez, S., Robles, A., Tobarra, L.L., Caminero, A. and Cano, J. (2020), “Analyzing students' self-perception of success and learning effectiveness using gamification in an online cybersecurity course”, IEEE Access, Vol. 8, pp. 97718-97728.

Roy, R.V. and Zaman, B. (2018), “Need-supporting gamification in education: an assessment of motivational effects over time”, Computers and Education, Vol. 127, pp. 283-298.

Saadé, R.G. and Kira, D. (2009), “Computer anxiety in e-learning: the effect of computer self-efficacy”, Journal of Information Technology Education, Vol. 8 No. 1, pp. 177-191.

Sailer, M., Hense, J.U., Mayr, S.K. and Mandl, H. (2017), “How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction”, Computers in Human Behavior, Vol. 69, pp. 371-380.

Sanchez, D.R., Langer, M. and Kaur, R. (2020), “Gamification in the classroom: examining the impact of gamified quizzes on student learning”, Computers and Education, Vol. 144, pp. 1-16.

Schöbel, S., Saqr, M. and Janson, A. (2021), “Two decades of game concepts in digital learning environments – a bibliometric study and research agenda”, Computers and Education, Vol. 173, pp. 1-23.

Seaborn, K. and Fels, D.I. (2015), “Gamification in theory and action: a survey”, International Journal of Human-Computer Studies, Vol. 74, pp. 14-31.

Soto-Acosta, P., Molina-Castillo, F.J., Lopez-Nicolas, C. and Colomo-Palacios, R. (2014), “The effect of information overload and disorganisation on intention to purchase online”, Online Information Review, Vol. 38 No. 4, pp. 543-561.

Subhash, S. and Cudney, E.A. (2018), “Gamified learning in higher education: a systematic review of the literature”, Computers in Human Behavior, Vol. 87, pp. 192-206.

Suh, A. and Wagner, C. (2017), “How gamification of an enterprise collaboration system increases knowledge contribution: an affordance approach”, Journal of Knowledge Management, Vol. 21 No. 2, pp. 416-431.

Szendrői, L. and István, S. (2022), “Implementing gamified teching: exploring the effects of gamification and personal types in an economics course”, International Journal of Game-Based Learning, Vol. 12 No. 1, pp. 1-19.

Tan, D., Ganapathy, M. and Singh, M.K.S. (2018), “Kahoot! It: gamification in higher education”, Pertanika Journal of Social Science and Humanities, Vol. 26, pp. 565-582.

Werbach, K. and Re (2014), “Defining gamification: a process approach”, International Conference on Persuasive Technology, Persuasive Technology, LNCS 8462, Sprinter International Publishing, pp. 266-272.

Zainuddin, Z., Shujahat, M., Haruna, H. and Chu, S.K.W. (2020), “The role of gamified e-quizzes on student learning and engagement: an interactive gamification solution for a formative assessment system”, Computers and Education, Vol. 145, pp. 1-15.

Corresponding author

Vanessa Itacaramby Pardim can be contacted at: vanessa.itacaramby@usp.br

Related articles