Gamification and its challenges in e-learning: a case study of computer science learners in KKHSOU

Abstract

Purpose

The purpose of the case study is to investigate the perception of computer science learners at Krishna Kanta Handiqui State Open University (KKHSOU) regarding the use of gamification and to identify the challenges in implementing gamified teaching-learning processes in open and distance learning (ODL) institutions.

Design/methodology/approach

The case study was carried out by observing the views of the learners on the use of gamification in distance learning and studying the participation level of computer science learners before and after the gamification environment was added to a specific topic through the learning management system (LMS). The pre-game and post-game effects on learners' participation against various factors using t-tests were also investigated in this study.

Findings

The study reveals that gamification had a positive and highly significant influence on the learners' participation and engagement in the e-content provided via the LMS. The post-gamification mean score (5.62) was higher than the pre-gamification mean score (2.36), and the overall paired t-test (5.301) value was significant at the 1% level of significance. Some of the challenges faced during the study were communication with the learner, lack of technical skills of the learner, duration of study, etc.

Originality/value

This study will help in understanding the perception and effect of gamification on computer science learners in ODL mode in India. The study succeeded in demonstrating that the use of gamified elements results in increased learner participation.

Citation

Barua, A.M. and Bharali, S.S. (2023), "Gamification and its challenges in e-learning: a case study of computer science learners in KKHSOU", Asian Association of Open Universities Journal, Vol. 18 No. 3, pp. 233-245. https://doi.org/10.1108/AAOUJ-01-2023-0001

1. Introduction

Gamification is basically the application of game design techniques and elements in non-game contexts and activities (Deterding et al., 2011). The term “gamification” was initially coined by British computer programmer Nick Pelling in 2002 but became widely used in 2010 (Tóth and Tóvölgyi, 2016; Kamasheva et al., 2015). Werbach et al. (2012) added to the definition of gamification that it is used to engage the users or alter their behavior. When preparing a gamified platform, we should keep in mind that the creators are not making an isolated game but only using game elements to motivate the learners to learn in a fun-induced environment. Gamification has been used in several different domains, such as business, marketing, mobile applications etc. but its utility and importance in the educational environment have increased in recent years.

The main goals of applying the gamification concept in the distance learning process are to increase the learners' motivation, engagement and involvement in the learning activities, support behavioral changes and socialize (Aziz Hussin et al., 2021; Strmecki et al., 2015). When learners enjoy the learning experience, they will be more motivated to learn and actively participate in their learning process. Current studies suggest that a positive attitude is developed among learners who use gamification as a learning tool where a learner can play according to a predefined scenario and can also monitor the result of their decision-making on goal realization and learning outcomes (Bernik et al., 2017). Introducing gaming components as a teaching method in e-learning allows learners to have a repetitive chance, and learners may engage themselves for an extended period, which makes the learning most effective.

Several researchers have found that gamification can enhance user engagement, motivation and participation and improve users' performances like decision making, problem solving and critical thinking (Pakinee and Puritat 2021; Hussein et al., 2019; Puritat, 2019 a, b). The learner's perception of e-learning content will be greatly influenced while using gaming elements. The e-learning contents help to promote full-time education and improve the curriculum (Saleem et al., 2021).

Amidst the coronavirus disease 2019 (COVID-19) pandemic, educational institutions underwent a shift from traditional face-to-face teaching to e-learning methods, aiming to provide knowledge to learners. Consequently, teachers faced the challenge of ensuring that their students were informed and actively engaged in the e-learning platforms. With the wide spread usage of emerging technology like the web, social media and mobile phones in day-to-day life, the importance of e-learning frameworks has been expanded as a huge number of learners can directly access web resources at a minimal cost (Saleem et al., 2021).

1.1 Gamification: a theoretical support

A lot of research work in the relevant area has been done to achieve better effective learning by using gamification in education. All the elements of computer games may not be suitable for e-learning systems. According to Bernik et al. (2017), a pedagogical design based on gamification places greater emphasis on students' motivation in the learning process and increases students' willingness to learn and their engagement with online learning materials, which considerably reduces the subjective influence of teachers on students. Their study showed that the most commonly used pedagogical features are discussion forums, questionnaires, quizzes, assignments, bonus materials and feedback. Game-based learning has improved students' performance and increased the speed of responses (Attali and Arieli-Attali, 2015). Pakinee and Puritat (2021) and Smiderle et al. (2020) highlighted that the application of gamification in e-learning systems enhanced the students' engagement and their learning behavior based on basic personality traits. Moreover, games are considered highly interactive resources that fulfill learning goals. Pacheco (2013) found the importance of educational games through the learning management system (LMS) in promoting emotional engagement to motivate students and make the teaching and learning process more interesting and effective. According to Pakinee and Puritat (2021), gamification of e-learning serves as a tool to engage students in distance learning; however, to enhance learning, the presence of a teacher is also necessary during the teaching process.

Muntean (2011), in their study, proposed that metrics like page views per visitor, time spent on the site, total time per user, the frequency of visit, participation and conversions can be used while analyzing the engagement of learners. Aziz Hussin et al. (2021), in their paper, proposed a gamified educational course in an open and distance learning (ODL) environment by considering three learning theories, namely behaviorism, cognitivism and constructivism. Their studies proved that the fun learning approach could offer a better learning experience to the students. However, the way instructions are presented in game-based learning is much more important in learning outcomes, as otherwise it can lead to a decline in performance, participation and effort (Kalogiannakis et al., 2021). Ahmed and Asiksoy (2021), in their study, investigated the effects of gamified flipped learning (GFL) on the students' innovation skills and self-efficacy for a virtual physics laboratory course. They claimed that the students had a positive perception of gamification. Ghai and Tandon (2023), in their study, investigated the role of gamification and instructional design to enhance the usability of online learning. The researchers in their study claimed that gamified learning is a key enabler of motivation, engagement and user experience, along with instructional design.

Based on the various studies on gamification, factors like engagement, participation, attention, motivation and the performance level of learners are considered. There are limited studies highlighting the challenges faced during the implementation of game elements for distance learners. This study involves polling students who were enrolled in the computer science program at Krishna Kanta Handiqui State Open University (KKHSOU) in 2021. The responses to the survey's questions about gamification and the e-learning framework were carefully scrutinized. The study also assessed how well the participants could use the LMS. The e-contents were carefully created with both non-gamified and gamified learning environments, including self-assessment exercises. The study also looks at how incorporating game components affects learners' participation and engagement levels.

The research objectives of this study are as follows:

RO1.

To study the perception of computer science learners of KKHSOU regarding the use of gamification.

RO2.

To study the participation level of computer science learners of KKHSOU before and after the implementation of game elements in LMS.

RO3.

To identify the challenges in the teaching-learning process based on this study.

2. Methodology

The study was conducted among the learners enrolled in the first semester of academic session 2021–2022 in the Master of Science in Information Technology (MSc IT) and Bachelor of Computer Application (BCA) programmes of KKHSOU. The total number of learners enrolled in these two programmes during 2021–2022 was 150, out of which 31 (21%) were in the MSc IT programme and 119 (79%) were in the BCA programme. This study aims to analyze the learners' responses to the use of gamification in distance learning. A survey questionnaire was distributed via a Google link to the email IDs of the learners. The questionnaire consisted of two sections: (A) demographic profiles and (B) learners' opinions on the use of gamification in distance learning. For section (B), learners were asked to answer whether they strongly agree, agree, are neutral or disagree or strongly disagree with the statements given in the questionnaire regarding gamification.

In the study, both gamified and non-gamified environments were provided for the computer science learners using the LMS of the university, which is built using Moodle 4.0. The e-learning content was provided to the same learners through the LMS of the university on the topic “Array,” which is part of the course “Data Structure Through C Language” in their second semester. This study aims to investigate the effectiveness of introducing gamification in e-learning content. The study of RO2 was conducted in two phases. In the first phase, the participation level of the learners for non-gamified e-contents was observed. In the second phase of the study, the participation level of the learners for the same e-contents after the addition of gamified elements was observed. The participation and popularity of the different game elements were also monitored along with the t-test results in the pre-post-gamification framework. The five main steps followed for RO2 are shown in Figure 1.

In the first step, the e-contents, including videos on the topic “Array”, were developed. In the second step, quizzes were created to assess learning outcomes in the non-gamified version. In the third step, the learners were invited to login to this course in the LMS to view the e-contents and attempt the quizzes. The learners were sent invitations to join the course through email messages, messages in WhatsApp groups and phone calls. The time period allotted for this first phase of activities was 15 days, from July 1 to 15, 2022. In the fourth step, the gamified elements are created and added to the course. Next, the learners were asked to access the new gamified environment, and the time period given for this second phase of activities was from July 16 to 31, 2022. In the fifth step, records of the learner's activities and participation details for one month were collected from the LMS and then analyzed.

3. E-learning platform design

This study uses the KKHSOU LMS called “eBidya: LMS of KKHSOU,” created using Moodle 4.0, through which the e-contents of various courses can be offered with learning resources and activities. Moodle 4.0 is an open-source e-learning platform that is suitable for gamification as it allows teachers to implement game elements in the learning process.

3.1 Learning curriculum

The curriculum was created by considering three aspects, namely time, material and assessment of the learners. The e-materials in the course can be divided into four categories: topic, lecture videos, quizzes and game elements. Four topics were chosen for this study based on the concept of “Arrays,” along with two video lectures and four quizzes. For the gamified version, four game elements were incorporated, including questions about the selected topic “Arrays.”

Table 1 shows the distribution of the two time periods for gamified and non-gamified environments for the time period from July 1 to 15, 2022 and the time period from July 16 to 31, 2022 for learners' self-assessment.

3.2 Implementation of gamified design

The implementation of gamified design depends on the games that are chosen and the positives that can be achieved from the implementation of these games. These elements play an important role in generating effective e-learning as well as enhancing learners' interest throughout the learning process.

In this study, the game elements are used for a learner's self-assessment in a way that helps them remember the important terms and the meaning of those terms. The game elements that we have chosen to apply to the LMS for our study are Hangman, Hidden Picture, Crossword and Snakes and Ladders. All these games help the learner to self-assess themselves in a fun way by playing games that are very popular and with which they may already be familiar. Details of the game elements are given in Table 2.

4. Results and findings

4.1 Findings of RO1

A survey questionnaire was given out to students enrolled in the BCA and MSc IT programmes during the study's initial phase. There were 42 responses, and 31 of them were from the BCA programme, while 11 were from the MSc IT programme. A total of 29 respondents to the survey were students between the ages of 18 and 30, 9 were students between the ages of 31 and 45 and 4 were students between the ages of 46 and 60; however, there were no students over the age of 60. When asked if they were familiar with the word “gamification” in distance learning, the study found that 26 respondents (61.9%) were in favor of it and 16 respondents (38.1%) were against it. Additionally, the survey gathered details regarding the participants' individual encounters with gamified components in online courses. Results showed that while 16.7% of respondents had used gamified components in their distance learning courses, 83.3% of respondents had not come across gamification in open-distance e-learning. Participants were also asked if they had ever used LMS software, such as Moodle, and the results showed that 85.7% had never used the LMS and 14.3% had.

The survey also included questions on gamification in distance learning, for which respondents had to answer in a five-level Likert scale comprising positive and negative responses like “strongly agree,” “agree,” “neutral,” “disagree” and “strongly disagree.” Different gamification queries were asked in the survey, and the percentages for each response are shown in Figure 2. The result shows that 83.3% of respondents agree or strongly agree, while 11.9% of respondents were neutral when asked if gamification in distance education will make learning less stressful. Furthermore, 80.8% of the respondents agreed or strongly agreed while 14.2% were neutral when asked if gamification in distance education will make learning more fun. When the respondents were asked if gamification in distance education will help to achieve effective learning, 78.5% of the respondents agreed or strongly agreed, while 19.0% were neutral on this question. The results are shown in Figure 2.

The result shows that 85.7% of respondents agreed or strongly agreed, while 11.9% of respondents were neutral when asked if gamification in distance education will improve the learning process. Additionally, 78.5% of respondents agreed or strongly agreed, while 19.0% were neutral when asked if gamification in distance education will help in retaining the attention of students. Next, 85.6% of respondents agreed or strongly agreed, while 11.9% were neutral when asked if gamification in distance education will help the students to learn the topics better. The results are shown in Figure 3.

When the respondents were asked if gamification in distance education will help increase the engagement level of learners, 85.7% of respondents answered that they agreed or strongly agreed, while 11.9% were neutral. Moreover, 83.3% of respondents agreed or strongly agreed, while 14.2% were neutral when asked if gamification in distance education will help increase active participation from students. Additionally, 85.6% of respondents agreed or strongly agreed, while 11.9% were neutral when asked if gamification in distance education will help increase the motivation of students. The results are shown in Figure 4.

4.2 Findings of RO2

The present study is divided into two time groups, i.e. before and after the addition of game elements. The study reveals that the participation and engagement of learners in the “Data Structure through C Language” course made available in the LMS have increased significantly after the addition of game elements. The results observed for the learners of the BCA and MSc IT programmes are summarized in Table 3.

It can be observed from Table 3 that the participation increased after implementing the game elements in the LMS course for both the BCA and MSc IT programmes. The number of participants increased from 19 to 38 for Topic 1, 12 to 31 for Topic 2, 8 to 18 for Topic 3 and 7 to 16 for Topic 4, as can be observed from Table 3. The increasing trend of learners' participation can also be seen for the lecture video materials provided on the LMS platform. The results showed that the total number of participants for both the BCA and MSc IT programmes had been increased from 12 to 28 for the first video lecture and from 4 to 16 for the second lecture video materials. Similarly, it was found from the study that the total number of participants in the non-gamified quizzes also increased after incorporating the game elements in the learning platform. Learners' participation had increased from 13 to 23 for Quiz 1, from 9 to 26 for Quiz 2, from 8 to 23 for Quiz 3 and from 7 to 17 for Quiz 4 as can be observed from Table 3. The results also show that after the introduction of the game elements, the participation of the learners in all the different e-contents like topics, lecture videos and quizzes increased.

Figure 5 provides a pictorial response to the learners' participation in the four game elements. It has been observed that out of the four games, Hangman was the most popular, as it was accessed by 29 participants. The next most popular game was Hidden Picture, which had 22 participants. The third game was a crossword game, which was accessed by 18 participants. The last game element was Snakes and Ladders, which was accessed by 19 participants.

In the present study, for both groups, i.e. pre- and post-gamification, when a learner accessed and engaged with any of the components or metric of the e-content provided through the LMS, a score of 1 was accorded. Contrarily, when a learner does not access and engage with the components of the e-content provided through the LMS, a score of 0 is accorded. As the total number of components studied was 10; hence, the total score for learners' participation and engagement fell within the range of 0–10, where a higher value indicates higher participation and engagement. Further, we have studied the effects of gamification on learner participation against factors like gender, age group, the choice of programmes and study centre. Thus, the present study has been conducted using the framework of pre- and post-game learners' engagement, broadly adhering to the principles of quasi-experimental design.

It was observed from Table 4 that the post-gamification mean score (5.62) was higher than the pre-gamification mean score (2.36). The paired t-test provided a value of t-statistics (5.301), which was significant at the 1% level of significance. This implies that the gamification had a positive and highly significant influence on the learners' participation and engagement in the e-contents provided via the LMS.

• (1) Gender

The t-test results further show that for male learners, there was a highly significant difference in learner participation between pre-game and post-game results. The calculated t-value (4.663) for male learners was significant at 1% level of significance, whereas the calculated t-value (2.520) for female learners was significant at 5% level of significance. This suggests a plausible male bias in gamification, wherein male learners are likely to be more attracted to games and also toward trying out new tools employed in teaching-learning processes.

• (2) Age group

The t-test results also show that for learners within the age group of 18–30 years, the post-game mean score of participation and engagement of the learners was significantly greater than the pre-game mean score. The calculated t-value (6.428) for learners within this age group was found to be significant at the 1% level of significance. Quite revealingly, the calculated t-value (1.679) for learners belonging to the age group 31–45 years and that of the learners belonging to the age group 46–60 years (2.990) were found to be statistically insignificant. This goes on to indicate that gamification has a distinct and greater effect among the younger age groups in terms of participation and engagement.

• (3) Study centre

The calculated t-value (2.095) for learners belonging to the city study centre (CSC) and also the t-value (2.274) for learners belonging to regional centre (RCs) had been found to be statistically insignificant when compared with the pre- and post-gamification mean scores attained by the learners. However, the t-value (6.544) for learners in other study centre (OSC) shows that there was a highly significant difference in learner participation between pre-game and post-game survey results. This may be because the learners in the CSC and in the RC were more likely to be exposed to regular online or face-to-face counseling classes, but the learners in most of the OSCs might not have the same benefits, and hence, they are more interested in learning through gamification tools.

• (4) Programme choice

The t-test results again reveal that for the BCA program, there was a highly significant difference in learner participation between pre-game and post-game mean scores attained by the learners. The calculated t-value (6.44) for BCA learners was found to be significant at the 1% level of significance. This may be because the BCA learners were freshly enrolled and were relatively young and had limited knowledge of the chosen topics. Thus, gamification can be instrumental in increasing the engagement of learners. Quite contrarily, in the case of the MSc IT programme, the t-test result (1.206) shows that there was no significant difference in learner participation between pre-game and post-game scores. This is because the learners of the MSc IT programme were presumably more familiar with the topic or material chosen for this study as they have prior knowledge of the same during their already completed programmes.

This result underlines the fact that gamification alone may not be effective for all sorts of programmes; rather, gamification has to be supplemented by innovative and interesting courses and topics to attract greater participation and engagement of learners.

4.3 Challenges of the study

The study provided some insight into the challenges of effective use of gamification in ODL mode. These challenges are listed below.

• (1) Communication with learners

A few learners remained unreached even when different communication mediums like phone calls, emails and WhatsApp messages were used to communicate with the learners.

• (2) Limited interest and motivation

During the study, it was found that learners had limited interest and lacked proper motivation to access the e-materials in the LMS.

• (3) Digital training required

Learners belonging to different age groups have different levels of skill in handling new technologies. Most of the learners are new to LMS, and hence the application of a gamified environment in addition to LMS poses new technical difficulties. Digital training is required to bridge this gap between technical skills and the ability of the learners.

• (4) Duration of study

One of the future challenges will be to compare the success of this pilot gamification study with future studies after increasing the duration of the study from one month to a whole academic semester.

• (5) Isolated learning

Another significant challenge lies in addressing the isolation faced by ODL learners during their learning journey. The absence of face-to-face interactions with educators and regular interactions with peers hinders the learner's engagement and motivation, potentially leading to reduced participation and a sense of disconnection from the learning process.

• (6) Environmental issues

Another challenge that was identified in this study was the impact of natural calamities like floods, which are specific to KKHSOU. Since KKHSOU is located in Assam, which is prone to massive floods every monsoon, the learners from these floods-affected regions of Assam are deprived of having the same access to various facilities for ODL learning. In a few cases, financial constraints also hamper learners in their journey in the ODL mode.

Addressing these issues and finding effective solutions in the context of open-distance education remain crucial for improving learner engagement and participation.

5. Discussions and implications of the study

This study investigates the perception of computer science learners enrolled in the BCA and MSc IT programmes toward gamification in distance education. The study also measures the impact of the addition of gamification on their LMS courses by monitoring the participation level of learners in the pre-and post-gamification frameworks. It is essential to recognize that several factors play an important role in the impact of gamification on the study. The duration of the study (one month) may be one of the factors that play an important role as mentioned in the challenges. In this study, since the course belongs to the second semester and the period of study was during the initial phase of the semester, the number of participants was limited. Also, the selected course was practical based, so the confidence level of the learner is another important factor. Similarly, the level of instructor perception also had an effect on the study.

Gamification for imparting education is a new and innovative approach for learners in many universities. Looking at the increase in the number of learners in different e-contents of the LMS course, gamification could be adapted as an additional teaching mechanism in addition to the conventional teaching methods that are already employed. This strategy will be fruitful in engaging more learners as it helps imparting subject knowledge in a fun way. This could also further help to increase the participation, motivation and performance levels of the learners.

6. Conclusion and future scope

Gamification integration in online learning is an effective tactic for raising student engagement in distance learning. Several goals can be accomplished by introducing game aspects into the learning process, including enhanced student retention rates. These goals include heightened learner engagement, increased motivation and an enjoyable setting. This study demonstrates how gamification may be successfully applied and accepted as an effective teaching strategy for students studying computer science, leading to more engaging online learning environments. The study's conclusions mark a significant advancement in the implementation of gamified environments within the LMS to improve teaching-learning in online education. The findings of this study also provide insightful information for future research and act as a manual for creating successful gamified systems for distance learning.

The study conducted is experimental in nature and offers numerous opportunities for future research. Subsequent studies may explore a broader range of game elements, exceeding the current four elements used. While this study focused on computer science students, it was evident that their technical knowledge regarding the LMS access was inadequate. Future investigations could be conducted after these learners have gained more experience handling the LMS, enabling a comparison between the outcomes of different studies.

To expand the scope, further research could consider the entire semester's timeframe as specified in the university's academic calendar. Additionally, the study's diversity could be enhanced by including learners from various subject areas and departments within the same university. Furthermore, a cross-university study would be immensely valuable in understanding the impact of gamification on the teaching-learning process across different subjects. Such an approach would provide a more comprehensive and holistic view of the effects of gamification in the context of education.

Figures

Figure 1

Research flow (RO2)

Figure 2

Gamification in distance education makes learning less stressful, more fun and effective

Figure 3

Gamification in distance education helps to improve the learning process, retain attention and learn better

Figure 4

Gamification in distance education helps to increase engagement level, active participation and motivation

Figure 5

Number of participants who accessed the different game elements

Table 1

Time distribution for self-assessment

Time period	Gamified or non-gamified environment
1st July – 15th July 2022	Non-gamification version
16th July – 31st July 2022	Gamified version

Source(s): Table by authors

Table 2

Gamification elements in the learning management system

Game elements	Description
Hangman	It is a guessing game where the learners attempt to build the missing word by guessing the spelling of the word. They can win the game by correctly guessing all the letters of the word
Hidden picture	It is puzzle game. Here when the learner gives the correct answer, a part of the hidden picture is shown. When all the questions are answered correctly then the whole hidden picture will become visible
Crossword	It is a word game in which learners needs to fill up boxes that go across and down pattern by entering words that are the answers to the given questions
Snakes and Ladders	It is a board game where, when the learner correctly answers the question, they can then roll the dice. The point on the board moves forward depending on the value of the dice and the goal is to reach the final square on the board

Source(s): Table by authors

Table 3

Participation level of learners before and after addition of game elements

E-Contents name		BCA		M.Sc.IT
		Before game elements	After game elements	Before game elements	After game elements
Non-gamified environment	Topic 1	13	30	6	8
	Topic 2	8	26	4	5
	Topic 3	5	14	3	4
	Topic 4	3	11	4	5
	Video Lecture 1	7	23	5	5
	Video Lecture 2	2	13	2	3
	Quiz 1	9	14	4	9
	Quiz 2	6	17	3	9
	Quiz 3	6	18	2	5
	Quiz 4	5	12	2	5
Gamified environment	Game 1 (Hangman)	–	23	–	6
	Game 2 (Hidden Picture)	–	19	–	3
	Game 3 (Crossword)	–	15	–	3
	Game 4 (Snakes and Ladders)	–	15	–	4

Source(s): Table by authors

Table 4

t-test values for pre-game and post-game learner participation

	Mean
Factors	Pre-game	Post-game	N	t-value	p-value
Overall**	2.36	5.62	42	−5.301	0.000
Gender
Male**	2.14	5.40	35	−4.663	0.000
Female*	3.43	6.71	7	−2.520	0.045
Age group
18–30**	2.00	5.26	29	−6.428	0.000
31–45	2.56	5.56	9	−1.679	0.132
46–60	4.50	5.50	4	−2.990	0.784
Study centre
CSC	2.81	5.63	16	−2.095	0.540
RC	4.14	5.43	7	−2.274	0.063
OSC**	1.32	5.68	19	−6.544	0.000
Programme
M.Sc.IT	3.18	5.27	11	−1.206	0.256
BCA**	2.06	5.74	31	−6.44	0.000

Note(s): ** indicates significant at 1% level of significance

* indicates significant at 5% level of significance

Source(s): Table by authors