Fun, experience or education? Learning efficiency – virtual reality lesson vs traditional lesson

Abstract

Purpose

The aim of the article was to compare the learning outcomes of the same content in the form of a traditional analogue lesson and in the form of a virtual reality (VR) lesson with the use of head-mounted display (HMD).

Design/methodology/approach

The study included one biology lesson conducted in 4 groups of a dozen people and one biology lesson in VR carried out individually on 75 people. The respondents completed the knowledge test, a questionnaire before and after the class regarding the attitude to new technologies, and feelings after the lesson. The researchers used detailed observation sheets (subjects' behaviour and the dynamics of the lesson). The obtained results were analysed statistically through lesson type (traditional/VR), respondent type (technology enthusiast/non-enthusiast) and question type. The Mann–Whitney U test, t-student and chi-squared (?²) test were used.

Findings

The average of the overall results in the knowledge test was similar in both groups (16 points; ±SD 2.13), slightly better for the analogue group and for the non-enthusiast group. It was found that VR hinders the acquisition of knowledge by tech enthusiasts, who perceive it primarily in the play paradigm. However, it encourages the learning of technology sceptics, who quickly discover a passion for exploring the virtual world. It was clearly indicated, quantitatively and qualitatively, how the technology modalities directly influenced the learning outcomes.

Originality/value

The article offers fresh insights into how students' perceptions of the educational process can be transformed through the integration of VR. The compelling findings and nuanced analysis provide a robust foundation for exploring new frontiers in educational technology.

Citation

Brylska, K., Gackowski, T., Kwiatkowska, A. and Dudziak-Kisio, M. (2024), "Fun, experience or education? Learning efficiency – virtual reality lesson vs traditional lesson", Information Technology & People, Vol. 37 No. 8, pp. 216-234. https://doi.org/10.1108/ITP-08-2022-0631

1. Introduction

We live in an era of great significance, where the impact of media and new technologies on human cognitive functions is rooted in a peculiar paradox. On the one hand, the social fabric is predominantly shaped by media, undergoing an extensive mediatization process. Active participation in various communities – be it in business, politics or even within families – has become impossible without engaging with media to some extent (although this influence can vary depending on factors such as age, place of residence, education, wealth, etc.). On the other hand, it is well understood that cognitive skills can be potentially limited by excessive exposure to media, especially new forms. This limitation manifests as increased fatigue, distraction, poorer memory retention and heightened susceptibility to various influences from individuals and entities (Whelan et al., 2017; Karr-Wisniewski and Lu, 2010; Luo et al., 2022; Schmitt et al., 2021; Maass et al., 2011; Ophir et al., 2009; Kuhlmann et al., 2014; Metag and Arlt, 2016; Schumann, 2018; Strömbäck et al., 2020). If this paradox is considered within the context of education, which is arguably one of the most crucial cognitive tasks for individuals, particularly the youth, the magnitude of the challenge ahead can be appreciated. Researchers are tasked with addressing questions such as whether to incorporate new technologies into the learning process, and if so, to what extent. They grapple with how to do it effectively and safely, which technologies to employ and the degree of customization for users. We believe that this manuscript has the potential to contribute to this vital and intricate discussion, marking the next step in the quest for answers to these profound questions.

To ensure conceptual clarity, it is crucial to define the key terms relevant to this article. Virtual reality (VR), as defined by Coquillart et al. (2011, p. V), refers to “interactive human-computer-mediated simulations of artificial environments”, which, we can add, are three-dimensional and can be interacted by a person using special equipment, such as a head mounted display (HMD). Virtual technologies (VTs), as argued by Martín-Gutiérrez et al. (2017), encompass a broader range of technologies beyond VR, including augmented reality (AR) and, at times, mixed reality (MR). Therefore, VTs is a more comprehensive term than VR, covering a wider spectrum of technologies that can provide a sense of immersion.

2. State of the art

The basic assumption behind the hypothesis about the effectiveness of using new technologies in education can be based on John Dewey’s assumption that effective learning is experiential (1938). VTs are emerging as a modern tool, and perhaps even a sophisticated method of acquiring knowledge, implementing various experiences and not inferior to a traditional lesson. The hypothesis about the effectiveness of VTs in education stems from the theory of motivation, especially interest theory and self-efficacy theory (Mayer, 2008; Schiefele, 2009; Bandura, 1977; Schunk, 1991). It can be assumed that the mechanisms of control, challenges, curiosity, cooperation and competition allow us to engage more thoroughly in the experience, and thus better remember not only the impressions but also the information that emerged during the VR experience.

The research conducted thus far confirms a certain relationship between VTs and several important learning outcomes, three of which seem to be crucial. The first is the improvement of students' academic performance, as well as their motivation and satisfaction (Harris and Reid, 2005; Kerawalla et al., 2006; Sotiriou and Bogner, 2008; Di Serio et al., 2013; Martín-Gutiérrez and Meneses Fernández, 2014; Bacca et al., 2014; Holley et al., 2016; Allcoat and von Mühlenen, 2018; Lege and Bonner, 2020; Gim et al., 2022; Rojas-Sánchez et al., 2023). The second is the development of the social and teamwork skills of students (Kaufmann et al., 2005; Martín-Gutiérrez et al., 2010; McGovern et al., 2019). The third is the development of psychomotor and cognitive skills (Feng et al., 2008). These findings demonstrate the validity of scientifically exploring VTs as an educational tool.

Moreover, VTs promote decision-making in experience, which facilitates autonomous exploration and learning by doing, in action. Real-time interaction enables instant visualization of results based on which students can make further decisions and improve learning efficiency (Kotranza et al., 2009). In addition, VTs empower a constructivist approach to learning – students can freely contact virtual objects and other students, explore and receive feedback, make decisions and co-create the educational space (also places inaccessible in real life). We already know that generative learning, combining VR/AR and traditional methods of internalizing knowledge and processing it, gives better educational results (Parong and Mayer, 2018). And last but not least, these technologies are more and more easily available and at the same time they are becoming an increasingly natural part of the environment of the youngest generations, so-called digital natives (Prensky, 2001; Martín-Gutiérrez et al., 2017).

At this juncture, it is imperative to delineate the persisting limitations associated with the utilization of VR in education. Firstly, a logistical constraint pertains to the temporal and financial investment required for the implementation and maintenance of these technologies, coupled with the challenge of integrating diverse devices while ensuring stability – an issue that may lead to frustration among students and educators (Dunleavy et al., 2009; Squire and Jan, 2007; Klopfer and Squire, 2008). Resistance among teachers is also observable, stemming from a shift in the learning paradigm, necessitating departure from their professional comfort zones and posing difficulties in aligning individual student needs with a standardized technological solution, such as the content of VR applications (Klopfer and Squire, 2008). A strategic challenge lies in the cognitive overload experienced by students due to the abundance of information, stimuli, and the simultaneous requirement to operate devices and complete tasks, inducing multitasking-induced overload and confusion (Dunleavy et al., 2009; Moreno and Mayer, 2004; Fassbender et al., 2012). Furthermore, tasks in a VR application may demand students to apply and synthesize various complex skills – such as collaboration, spatial navigation, problem-solving and technology handling – skills that students often lack (Kerawalla et al., 2006; Klopfer and Squire, 2008; Squire and Jan, 2007; Dunleavy et al., 2009; Fromm et al., 2021; Zhao et al., 2021).

Cognitive confusion becomes problematic when reality and virtual elements mix – students may lose track of where the game ends and reality begins. This, in turn, causes a loss of control, disrupts the learning process and may threaten physical safety (Klopfer, 2008; Dunleavy et al., 2009). Digital natives are not always competent in using new technologies, especially in educational settings (Margaryan et al., 2011). This conclusion underscores the significance of the ability to navigate technology, a factor that significantly influences the learning process (Tammy Lin, 2017). There are also known limitations to using these technologies for younger children: they are still developing hand-eye coordination and balance, and too early exposure to visual electronic stimuli can even harm them (Yamada-Rice et al., 2017). Finally, we cannot forget about cultural limitations; the respondents in the research projects executed so far are usually people from one cultural environment, which may be determinant (Tammy Lin, 2017), especially in terms of the language of VR experience (Fassbender et al., 2012).

In conclusion, the extant research offers compelling evidence in favour of VTs, particularly VR, as a promising tool for education. Nevertheless, the cited works underscore the imperative for rigorous scientific exploration, as our understanding of its efficacy remains fragmented and requires supplementation, particularly concerning the transmission of theoretical knowledge (as the development of manual skills in VR is well-researched and yields unambiguous results). The experiment detailed in this article addresses this research imperative.

3. Methodology

3.1 Aim and hypotheses

The article aims to check the effectiveness of teaching methods using VR technology as opposed to the traditional analogue lesson that we are dealing with during school education. Therefore, the focus of the article was on verifying two main research questions: (Q1) whether and, if so, how do the results of the students' knowledge test differ after the VR lesson and after the analogue lesson? and (Q2) whether and, if so, how do the results of the knowledge test of technology enthusiasts and other students differ?

Analogous to the research questions, we formulated the following hypotheses:

(H1).

The results on the knowledge test will be significantly different between the group participating in analogue lessons and VR lessons.

(H1a).

The results on the knowledge test will be better for the group participating in VR lessons than for those in analogue lessons.

(H2).

The results on the knowledge test will be significantly different between the technology enthusiasts and non-enthusiasts groups.

(H2a).

Technology enthusiasts will achieve better results in the test than the non-enthusiasts group.

3.2 Methods

The study was based on a combination of the experiment of conducting a series of biology lessons in the traditional form of an analogue lesson (analogue groups) and in a VR environment lesson (VR groups) with a knowledge test and questionnaires before and after the study. The proper study included one biology lesson conducted in four groups of nine or ten people each (39 in total) and one biology lesson in VR carried out individually on 36 people.

A pilot study was also carried out to refine the study and reduce its weaknesses. The pilot study involved conducting one analogue lesson (for a group of eight people) to refine the lesson scenario and the lesson observation sheet. It also included conducting nine VR experiments using the appropriate application to practice the experience pattern and the VR lesson observation sheet. The pilot study revealed the need to add a brief (approximately 4 min) introduction to the VR lesson, where the participant could learn to operate the equipment by exploring a different application.

In addition, a control group study has been conducted to verify the level of biological knowledge of an average social sciences or humanities student, without attending any lesson. The control group consisted of 69 social sciences or humanities students (54 women and 15 men), aged 19–22 years. The control group passed only the knowledge test, with no additional questionnaires. The students completed the test on computers under the supervision of the researcher.

3.3 Participants

The selection of the respondent group was purposive. It comprised undergraduate students from the faculties of social sciences and humanities, which, to some extent, reduced the probability of prior knowledge about the material covered in the lessons. The assumption was that these individuals had limited or no familiarity with the biological topics discussed, and their responses to the final test questions would be solely based on information obtained during the experimental lessons. Moreover, participants were questioned about their knowledge of biology in the pre-lesson questionnaire, aiming to potentially identify and exclude individuals with a more advanced understanding of biology from the experiment.

Another criterion for selecting participants in the study was language. Polish needed to be the first (native) language of respondents, thanks to which it was possible to rule out the language barrier in learning about strictly biological terms. What is more, in order to exclude respondents who potentially could deceitfully or without diligence answer the questions, respondents who indicated that they did not know what VR is were rejected from the study. The same selection criteria have been used to compose the control group in the study.

In the screening survey, 115 individuals participated. After applying the selection criteria (native Polish speakers, lack of advanced biological knowledge, basic understanding of what VR is), a total of 75 people qualified for the experiment. This group consisted of 42 females and 33 males, with ages ranging from 19 to 24 years (mean age 20.60 ± SD 1.01 years). Among them, 39 students (28 female, 11 male) participated in the analogue lessons and 36 students (14 female, 22 male) participated in the VR lessons. There was no remuneration for participating in the study. All participants gave written informed consent.

3.4 Procedures, materials and tools

The study consisted of a series of measurements divided into two main parts: analogue lessons and VR lessons.

The substantive starting point for our experiments was the VR application which served as the educational experience for our respondents (the VR lesson). The VR experiments were carried out on the VR Arena in Laboratory of Media Studies at the University of Warsaw, using the Oculus Rift VR Headset, touch controllers (enabling the subject to perform simple tasks and move around virtual space without having to move in reality) and a camera. We utilized a free Polish-language educational VR application developed by Bayer and Immersion companies, which consisted of three substantive parts related to the structure of the human body: the “Eye” module, the “Intestines” module and the “Heart” module (A preview recording presenting screen captures of the application can be found on https://www.youtube.com/watch?v=6-fEfTTqQEU). Figures 1–4 showcase screen captures from different segments of the application.

The user progressed through successive parts of the application, and in each of them, an off-voice explained various substantive threads. Simultaneously, the user received vocal instructions regarding the possibilities of actions in a given part of the application using controllers held in their hands (e.g. lifting bacteria in the intestine or following a blood vessel). In this way, the user could acquire knowledge and virtually observe various pathological conditions inside the human body. The experience did not have a game-like character – the user did not earn points or levels, nor did they compete with others. The primary goal of the application was to equip the user with knowledge in the field of human health through an immersive, engaging experience.

The VR lesson lasted on average less than 14 min (between almost 12 and 18.5 min), respondents could spend as much time in VR as they needed. Of course, it should be noted that this is the duration of the experiment itself, without adding the training time in another application before the actual lesson and the time of putting on, calibrating and removing the equipment. Respondents in VR groups did not have the opportunity to repeat the material, they only received a lecture in the form of a voice-over with a 3D visualization and a few game-related tasks.

The analogue lessons were conducted in a traditional manner and strictly adhered to the content of the VR application, staying within its limits. This was facilitated by a meticulously crafted script outlining the identical scope of knowledge covered in both lesson types (focussing on the eyes, intestines and heart). A biology teacher from the high school, duly prepared for the role, delivered the analogue lessons. The lesson plan involved brief lecture segments, utilizing a few PowerPoint slides interspersed with auxiliary graphics displayed on the teacher-led presentation. These segments were complemented by exercises for students, who independently filled out pre-prepared worksheets (printed on paper). Each analogue lesson across the four groups lasted approximately 40 min (between 32 and 45 min) and followed a consistent structure. The conditions were similar to a traditional classroom lesson – the respondents were seated at the desks, the teacher at the desk in front of the room.

The respondents in both groups completed the same knowledge test and two types of questionnaires: before and after the lesson.

The pre-lesson questionnaire included demographics (gender, age, faculty and BA/MA programme student was participating), the section regarding respondents' knowledge and experiences related to VR and the use of electronic devices (three closed-ended questions and three open-ended questions). The next part consisted of questions assessing participants' background knowledge and exposure to topics related to human anatomy, cardiovascular diseases and eye diseases, including their educational experiences and self-assessment of anatomical knowledge (four closed-ended questions and one using a Likert scale). The final section focused on respondents' attitudes toward new technologies, featuring 24 statements where respondents expressed their stance on a five-point Likert scale, ranging from “definitely no” to “definitely yes”, 12 statements presented an enthusiastic attitude toward new technologies, while the other 12 presented the opposite perspective.

The post-lesson questionnaire consisted of three open-ended questions (identifying which elements of the lesson were memorable to the student, which ones they liked the most, and what changes they would make to the lesson) and closed-ended questions in the form of affirmative statements. Students were instructed to indicate on a five-point Likert scale the degree to which these statements applied to them. In reference to these statements, respondents evaluated their experiences during the lesson, paying attention to the clarity of the material, the pace of information delivery, concentration, as well as the impact on their interest in the topic and the desire for further knowledge on the discussed subject. Following the VR lesson, students were presented with a set of 22 questions, while the analogous lesson prompted responses to 16 questions. Some questions were repetitive, while others were specific to the nature of each type of lesson.

The knowledge test comprised 20 single-choice questions, with each question having only one correct answer among the four provided options. The test exclusively covered the material discussed during both VR and analogue lessons, as the scope of the material in both sessions was identical. Students were allotted 20 min to complete the test.

The materials used by researchers were lesson scenarios and detailed observation sheets referring to the subjects’ behaviour and the course and the dynamics of the lesson, fulfilled during the experiments. Analogue lessons were recorded using a voice recorder (to avoid intimidating students and the teacher with a camera), while VR lessons were recorded on video. For each type of lesson, an observation sheet was prepared, filled in live during the experiment (research team member participated in each conducted lesson, both traditional and VR). If necessary, the sheet was supplemented based on the lesson recording. The observation sheet for analogue lessons, in addition to obvious metadata such as date and duration, included ten points related to student engagement, signs of interest or potential boredom, the lesson element that generated the most interest, comments and remarks made by students during the lesson, teacher’s actions (whether actively engaging in discussion, evaluating student work, correcting incorrect answers, and maintaining the overall lesson atmosphere) and the general teacher–student relationship. Additional researcher notes could also be added. The VR lesson observation sheet, in addition to metadata, included a list of 15 categories for evaluating the respondent’s behaviour in the VR experience. These categories included various aspects, such as respondent’s caution, exploratory activity, signs of nervousness, handling of equipment, asking questions, following instructions, maintaining balance, difficulties with controllers, revisiting previous locations and reporting any inconveniences. The researcher marked this behaviour by selecting a description from a predefined cafeteria for each question and by adding their own comments and observations.

The information recorded in the observation sheets was used to contextualize the quantitative results obtained, as indicated in subsequent sections of the manuscript.

4. Results

4.1 Results of the knowledge test

The basic way to check the effectiveness of teaching methods with the use of VR technology was to compare the results of the final knowledge test conducted after the VR lesson and after the traditional analogue lesson as well. The test consisted of 20 closed questions (one of the four answers was correct), each for 1 point to be obtained. A total of 75 participants completed the knowledge test (42 women and 33 men). The average of the overall results was 16 points (± SD 2.13), which means that the surveyed students gave an average of correct answers at the level of 80%. A Mann–Whitney U test indicated that there was no significant sex difference in relation to the result of the knowledge test (p = 0.669, α = 0.05).

The respondents from analogue lesson (n1 = 39) obtained slightly better results (Table 1). The average result in their group was 16.36 (± SD 2.21), the result above average of this group was obtained by 21 respondents – 53.85%. In this group the weakest result (9 points) and the highest ones (2 respondents – 5.13% – obtained the maximum points) have been noticed (see Figure 5).

The average result in the VR lesson (n2 = 36) was 15.61 (± SD 2.00), and the result above average of this group was obtained by 22 respondents – 61.11% (with overrepresentation of 16 points which was obtained by 30.56% of VR group respondents). The weakest result in this group was 10 points, while the best ones were 19 points (2 respondents – 5.56%). The VR experiments were therefore conducive to obtain an average result.

The students participating in the analogue lesson scored slightly better in the knowledge test, but the differences, however, turned out to be statistically insignificant (Mann–Whitney U test: p = 0.092, α = 0.05; t-student: p = 0.130, α = 0.05), so the H1 and the H1a have been falsified. The standard deviation in the scores was similar, indicating that belonging to the analogue/VR group does not significantly differentiate the test score (see Figure 5).

Nevertheless comparing the results of individual responses (Table 1) brings some new interesting threads to the analysis. Respondents in the analogue group (n1 = 39) gave 81.79% of correct answers and obtained better results in 12 questions while in the VR group (n2 = 36) it was 78.06% and only 6 questions with better results (and 2 draws) than the previous group.

It should be noted that the difference in the results of these two groups did not vary significantly, while the digestion time of these lessons differed more than twice. Moreover, the modality of the VR lesson created a direct issue for low VR group scores. The three lowest rates of correct answers in the VR group (q16 = 28%, q12 = 44%, and q4 = 50%) were obtained for topics that appeared during unclear and stressful situations for the respondents, i.e. moments when the lecturer was discussing the topics from those questions, and the respondent found themselves in a new stimuli situation. An example would be the transition from one organ to another in the virtual world. The respondent was given a task to perform but did not know the instructions for this task, which came a moment later. Probably, in this way, the respondent, trying to orient themselves in the new situation, did not pay enough attention to the voice of the lecturer for a while. Thus, respondents in the VR group scored much better for questions concerning moments in the VR experience where the voice-over was not interrupted by any task, allowing the respondent to leisurely observe the interior in which they were located.

It also needs to be noticed that the mean value obtained on the knowledge test of the control group was 10.87 points (± SD 2.57), while, as a reminder, it was 16.36 (± SD 2.21) in analogue group and 15.61 (± SD 2.00) in VR group. It means that the control group without any biology lessons achieved clearly worse on the test results, and more respondents were unsure of their choices. The increase of knowledge of the respondents after the lessons given can be assessed at the level of almost 50%. What is more, the spread of the results for the surveyed students from the control group was from 3 to 15 points but the highest number of respondents achieved the results of 9 and 11 points ( 13 and 12 students, respectively). They also gave an average of correct answers at the level of 54.35%; the best results were obtained by q20 (91.30% of correct answers), q17 and q18 (86.96%), while the poorest result was obtained for q16 (11.59%) and q1 (18.84%).

A Mann–Whitney U test indicated that there was a significant difference in relation to the result of the knowledge test between the analogue and control group (p < 0.001, α = 0.05) and between VR and control group (p < 0.001, α = 0.05).

4.2 Technology enthusiasts and non-enthusiasts groups

The last one stage of data analysis was to divide the respondents into two groups: enthusiasts of new technologies and the remaining respondents (for the sake of simplicity, they can be called non-enthusiasts). For the purposes of this study, we define an “technology enthusiast” as an individual who expresses a positive attitude towards new technologies, declares a willingness to explore and use them, demonstrates readiness to accept new devices and software, recognizing the benefits arising from them. A non-enthusiast, on the other hand, is a respondent not declaring such characteristics.

The respondents were categorized using pre-lesson questionnaire, comprising (among others, as indicated in section 3.4.) 24 questions on a five-point differential scale (from “definitely no” to “definitely yes”). A total of 12 questions expressed high enthusiasm for new technologies (e.g. “Buying the latest versions of devices such as a computer, tablet, smartphone is important to me”, “VR is the future”) and 12 questions indicating distance or concerns towards technology (e.g. “VR is impractical and unnecessary”; “I would feel foolish wearing VR goggles in front of other people”). The enthusiast group was selected from respondents who answered “rather yes” or “definitely yes” to at least 50% of the “enthusiastic” questions. From among the group of the remaining respondents, it is possible to distinguish a group of technological “sceptics”, whose responses were at most 40% “enthusiastic”. The division criterion was therefore rigid and sharp.

The results on the knowledge test for those two groups were very similar. The average result of enthusiasts (n3 = 34) was 15.91 (± SD 2.34) and for non-enthusiast (n4 = 41) the result was 16.07 (± SD 1.96) – so the H2 and the H2a have been falsified. In analogue lesson group, enthusiasts (n5 = 14) obtained 16.61 (± SD 2.23) average results and for non-enthusiasts (n6 = 25), it was 16.15 (± SD 2.21). In VR lesson, enthusiasts (n7 = 20) get average results of 15.26 (± SD 2.32) and for non-enthusiasts (n8 = 16) it was 15.87 (± SD 1.54). Non-enthusiasts in the analogue group fare relatively best, while enthusiasts in the VR group fare relatively the worst (Figure 6). It is worth noting that the standard deviation in the group of non-enthusiasts of new technologies (especially in the VR group) clearly differs, which indicates that members of this group were the closest to each other.

The differences in the studied lessons were demonstrated not only in the final test results but also in the indicative post-lesson questionnaire (with responses on a 5-point differential scale, to which points from 1 to 5 were assigned in the analysis). Thanks to this survey, we could see some differences in the feelings of the respondents after the lessons (see Table 2).

In the comparison, the respondents from the VR group noticed that they liked more the form of the lesson they participated in (3.58 mean in VR group and 2.82 in analogue group) and would more like to take part in a lesson of the same form, but with a different subject (3.67 mean in VR group and 2.13 in analogue group); they also felt a smaller drop in concentration (1.56 mean in VR group and 2.87 in analogue group), fewer people lost interest in the topic of the lesson (0.81 mean in VR group and 1.85 in analogue group), and they hardly found the lesson boring (0.25 mean in VR group and 1.67 in analogue group). All of these were consistent with the researchers' observations that, for example, about halfway through the analogue lessons, the students began to get bored and yawn, often looking out of the window.

It is also worth looking at more specific observations. After the analogue lesson, enthusiasts rated the lesson as boring to a greater extent (mean 1.71, ±SD 1.33) than after the VR lesson (mean 0.30, ±SD 0.80), during the lesson they noticed a greater decrease in concentration (mean 3.07, ±SD 1.21) than after the VR lesson (mean 1.60, ±SD 1.19), and also to a lesser extent they would take part in a lesson of the same form but with a different subject (mean 2.14, ±SD 1.56) than in the VR group (mean 3.70, ±SD 0.57).

We also carried out the chi squared (χ²) test for this data, and it turned out to be significant – so there is a relationship between some answers of the enthusiasts and the participation in the analogue or VR lesson (Q5: p = 0.001; Q6: p = 0.002; Q7: p = 0.015; Q8: p = 0.013, α = 0.05). Similar results have been obtained in the non-enthusiasts group in two types of lessons (Q1: p = 0.001; Q5: p = 0.000; Q6:p = 0.043; Q7: p = 0.000; Q10: p = 0.011, α = 0.05).

The last undoubtedly different groups noticed were “top enthusiasts” and “sceptics”. The group of “top enthusiasts” included 17 people (11 men and 6 women), 9 of them participated in the VR lesson (4 people declared that they have VR headset at home). In this group, 3 people were insecure and reluctant to explore throughout the experience, while 3 others were willing to explore. In addition, no one from this group asked about the equipment, did not report any somatic complaints, and 7 people asked at least once for additional instructions (perhaps it was a matter of the weakness of the application in which the lesson took place).

Therefore, it was not a homogeneous group, and the attitude to new technologies did not necessarily directly affect the behaviour in VR. Among the elements that people from this group remembered and liked the most are, of course, immersive and interactive elements (the fact that they could, for example, virtually “pick up” a bacterium or “catch” a vitamin with the help of controllers). One of the weaknesses of this type of lesson was the inability to go back to previous information, which meant that some information was missing when the subjects were engaged in an interactive task. This observation also confirmed unquestionably low scores on several questions in the final knowledge test in the VR group.

The group of “sceptics” included 16 people (5 men and 12 women), five of them took part in the VR lesson. In the VR experiment, they spent a little more time than the others – average time in the experiment was 15 min 5 s (while the average of all measurements was 14 min 9 s). None of the people in this group asked about the equipment, and all asked for additional instructions for tasks in VR. Among the impressions after the lesson, it is worth mentioning that the advantage of immersion was emphasized, especially the elements related to fun such as some mini-games. Moreover, no person commented on what would change in such a VR lesson, which can be read as this group was less critical of VR than the group of enthusiasts. Interestingly, the remaining 11 people after the analogue lesson, when they were asked what could be changed in this lesson, they demanded more dynamics and more visuality. It means that if they could improve analogue lessons (but theoretically it was a better type of lesson for this group), it would probably be heading towards lessons to VT.

5. Discussion

The results of the study do not allow for unequivocal confirmation of the hypotheses; in general, the differences in the test scores are not statistically significant for either the analogue – VR groups or the enthusiast – non-enthusiast groups. This finding aligns with the conclusions of Hamilton et al.'s (2021) literature review, which suggests that research results on the effectiveness of learning in VR remain ambiguous, especially in the area of science-based cognitive studies. As the authors state, most studies found advantages of VR learning over traditional methods, but “this was the case particularly when the subject area was highly abstract or conceptual, or focused on procedural skills or tasks” (Hamilton et al., 2021, p. 21).

However, in the study presented in this manuscript, some noteworthy trends were observed. Firstly, students after the VR lesson achieved similar results on the knowledge test as students after the analogue lesson, although the VR lesson was shorter and became a new and challenging experience for almost all participants. This provides a strong premise for confirming the effectiveness of education in such a modern formula. Such an assumption was confirmed in two similar studies, the subject of which was biological knowledge: Maresky et al. (2019) and Liou and Chang (2018) showed that immersion in the VR environment through HMD is more conducive to the assimilation of anatomical knowledge than a traditional lesson.

On the other hand, the advantages of an analogue lesson are undeniable – students are accustomed to this format, and the teacher adds excellent value (Ostrander et al., 2018). According to the Cognitive Theory of Multimedia Learning proposed by Mayer (2009, 2014), the traditional lesson provides a chance for more effective learning because it does not cause cognitive overload, as VR does. Furthermore, we observed that the fundamental strength of an analogue lesson was the teacher and the individual work of students on worksheets. Secondly, we noted that predispositions and personal characteristics turned out to be key – it can be assumed that a good student will always earn an A. Therefore, VR could be analysed as a valuable tool for students with specific needs and cognitive abilities. Moreover, we already know that combining VR with other teaching methods and techniques brings promising results (e.g. Fogarty et al., 2017; Johnston et al., 2018).

The experimental procedure revealed several conditions and assumptions that impact both data collection and its analysis. In the learning domain, a significant assumption was the inability to obtain answers to any questions asked during both lessons. In the case of a VR experiment, this is natural – the lesson’s scope is confined to the thematic content of the application. To maintain the same scope of classes, we also applied this assumption in the analogue lesson – the teacher could not respond to the students' questions, and if she did, it was done without additional knowledge presentation. This is an unnatural situation, causing a certain difficulty for the students and posing an immense challenge for the teacher. However, it is noteworthy that only students in the analogue lesson sought additional information; this did not occur during the VR lesson. This phenomenon may indicate a different approach of students to both forms of studying. During an analogue lesson, they are naturally oriented towards learning, so if something is unclear or interesting, they ask questions. In a VR experiment, participants are focused on experience and entertainment. Activity similar to the traditional question in VR experience may involve more active exploration of the virtual world, searching for or following an interesting object. However, in our experiment, only a few participants took such actions.

Interesting conclusions can be drawn in the area of technology skills. Despite the young age of the respondents and their claims of at least general knowledge of new technologies, most of the respondents – a generation of so-called digital natives – encountered difficulties in using VR equipment freely. In the pilot VR experiments, the team found that the respondents' ignorance of VR technology was a significant challenge, as for a significant part of them, our experiment marked their first contact with VR. During the pilot lessons, users “entered” the application immediately after putting on the headset, without prior training. This led to uncertain movement in the application, difficulties with the correct execution of the lecturer’s instructions or reluctance to use the controllers' functions (e.g. teleportation). In such situations, respondents often sought help from the researchers supervising the experiment, necessitating their intervention. Moreover, the lack of prior acquaintance with VR could negatively impact the respondents' acquisition of knowledge (Margaryan et al., 2011; Tammy Lin, 2017). Instead of focussing on the lesson, they directed their attention to familiarizing themselves with the functions of the controllers and headset or adapting to the virtual environment. To effectively address this problem, we utilized The Lab application, which served as a short warm-up for the respondents before the actual experience.

The third area concerns the assessment of the effectiveness of the learning process, prompting the question: How to test students' knowledge? Traditionally, the exam format aligns precisely with the lesson format – an analogue lesson concludes with a traditional test completed on a sheet of paper. In our experiment, following the lesson in which students listened to the teacher, completed paper worksheets and viewed the 2D presentation, they had to take a written test on the computer. The same exam format was prepared for students after the VR lesson, which essentially constituted a 3D experience. However, it can be assumed that it might be more effective if the exam took place within the learning environment. Moreover, tests are typically conducted after a certain cycle of learning, as the teaching, learning and knowledge verification process traditionally spans a more extended period. An additional consideration is retention tests – if we test students more than once, when should we conduct the test after the experiment (Parong and Mayer, 2018; Radianti et al., 2020; Çankaya, 2019)? Predicting the results of such a retention test is challenging, but it can be assumed that if educational activity is associated with emotions, it may contribute to better content retention (Steidl et al., 2011). This leads to another significant challenge – Should we evaluate the knowledge acquired in VR (which is necessarily procedural) through a test (Ozuru et al., 2013; Kamińska et al., 2019; Hamilton et al., 2021)? Furthermore, the question remains as to whether the results will be replicable for other subjects (e.g. not biology but history). Thus, are the characteristics of the taught content important, and is there any content more suitable for presentation in the form of a VR experience than others?

Based on these results, several practical recommendations can be proposed. VR emerges as an interesting technology that could serve as a valuable module in the educational process. It is likely to be effective as a tool for encouraging students to engage with a subject and deepen their knowledge, as it allows for the visualization of complex concepts in an understandable and attractive manner. However, since this technology is not yet widely present in traditional education, it would be beneficial to start by familiarizing students with VR and its capabilities to fully optimize its use in learning. Additionally, considering the leading role of teachers in the educational process, who also provide a certain degree of individualized learning (e.g. through questions and answers during lessons or individual consultations), VR should currently be treated as an additional, supportive option for both students and teachers in their work.

Certain limitations of the study have been identified, both at the methodological and procedural levels. It appears that it might be more effective to pre-divide the group into enthusiasts/non-enthusiasts before the experiment and select respondents to create more numerous groups. Additionally, retention tests would likely reveal more differences in the studied groups. At the organizational level, anonymized marking of respondents even during the analogue lesson could be beneficial for a more in-depth analysis of the results. Moreover, a more challenging test would probably reveal more individual differences (however, its scope was constrained by the limited content of the application).

The data presented raise several strategic questions regarding this area of research. Is this technological revolution changing the learning paradigm? What should be the relationship between content in the form of VR and in the form of analogue in the learning process – to what extent can VR serve as the “vestibule” of a traditional lesson, and when and how can it be an independent educational tool, supplemented with traditional methods at best (to work better and allow for generative learning)? Should VTs focus on imparting skills rather than knowledge (e.g. procedural-practical knowledge, analytical and problem-solving skills, or soft skills, aligning with the broader shift in education)? On the other hand, what is the role of the teacher – to be a mentor or become a guide to new technologies, an expert in learning techniques? And finally, who will the student become – an autonomous, fast-learning subject or a technology-complementing object? Are there – and what are their characteristics – people predestined to learn mainly with VTs, including VR? It seems that research in the area discussed will have to deal with these questions sooner rather than later.

6. Conclusion

In conclusion, the study revealed both opportunities and limitations associated with using VR as a technology for learning. On the one hand, due to its leading applications – fun, gaming, and entertainment activities – VR can effectively encourage knowledge acquisition (especially among tech enthusiasts) and present content in an engaging manner. On the other hand, the inherent characteristics of this technology, such as full immersion causing cognitive overload, impose significant limitations on its reliability as a tool for knowledge acquisition. The study’s results suggest that a primary challenge in effectively utilizing VR in the learning process is to ensure students become deeply familiar with the technology, enabling them to fully exploit its potential. Finally, the experiment’s outcomes highlight the essential and pivotal role of the teacher, who remains a crucial resource in the traditional education model, often determining student engagement and the success of the knowledge acquisition process.

Figures

Figure 1

In this interface, the user selects the organ to which he would like to move

Figure 2

User is in the heart. There is an ongoing task in which he must activate within 30 s as many blood cells as possible

Figure 3

In this part of the lesson, the user needs to pick up one beneficial and adverse bacterium to look at it closely

Figure 4

For this task, the user sees how a person suffering from AMD sees. He needs to perform such simple operations as turning off the TV

Figure 5

Scoring distribution in the knowledge test of the analogue group and the VR group

Figure 6

Scoring distribution in the knowledge test of the enthusiasts and non-enthusiasts in analogue and VR groups

Table 1

Comparison of correct answers between analogue and VR groups

	Analogue group (n = 39)		VR group (n = 36)
Question number	Number of correct answers in the group	Percentage of correct answers in the group	Number of correct answers in the group	Percentage of correct answers in the group
1	31	79%	27	75%
2	28	72%	25	69%
3	39	100%	31	86%
4	26	67%	18	50%
5	33	85%	27	75%
6	31	79%	33	92%
7	33	85%	36	100%
8	38	97%	34	94%
9	31	79%	35	97%
10	28	72%	32	89%
11	18	46%	20	56%
12	33	85%	16	44%
13	26	67%	24	67%
14	33	85%	28	78%
15	36	92%	33	92%
16	26	67%	10	28%
17	38	97%	31	86%
18	37	95%	36	100%
19	34	87%	30	83%
20	39	100%	36	100%

Source(s): Authors’ own work

Table 2

Comparison of means (and standard deviations) between respondents in the analogue and VR groups in the post-lesson questionnaire

Statements in the post-lesson questionnaire	Analogue group (n = 39)			VR group (n = 36)
	All (39)	Enthusiast (14)	Non-enthusiast (25)	All (36)	Enthusiast (20)	Non-enthusiast (16)
Q1: I enjoyed the lesson in which I participated**	2.85 (0.87)	2.71 (10.07)	2.88 (0.78)	3.58 (0.79)	3.40 (0.99)	3.81 (0.40)
Q2: I knew the topic discussed in the lesson before	3.41 (0.76)	3.71 (0.47)	3.40 (0.71)	2.38 (1.25)	2.40 (1.23)	2.38 (1.36)
Q3: During the lesson I lost interest in the discussed topic	1.75 (1.26)	2.14 (1.35)	1.68 (1.18)	0.80 (1.04)	0.85 (1.14)	0.75 (1.00)
Q4: I understood the issues presented during the lesson	3.46 (0.70)	3.50 (0.65)	3.44 (0.77)	3.36 (0.67)	3.25 (0.64)	3.50 (0.73)
Q5: The lesson I attended was boring*/**	1.60 (1.10)	1.71 (1.33)	1.64 (0.99)	0.25 (0.64)	0.30 (0.80)	0.19 (0.40)
Q6: During the lesson I noticed a drop in my concentration*/**	2.75 (1.24)	3.07 (1.21)	2.76 (1.16)	1.55 (1.14)	1.60 (1.19)	1.50 (1.15)
Q7: I would like to take part in a lesson conducted in this way again, but on a different subject*/**	2.17 (1.24)	2.14 (1.56)	2.12 (1.13)	3.69 (0.61)	3.80 (0.57)	3.69 (0.70)
Q8: The lesson irritated me and spoiled my mood*	0.60 (0.85)	0.71 (0.97)	0.60 (0.87)	0.13 (0.48)	0.10 (0.45)	0.19 (0.54)
Q9: During the lesson I felt focused on the subject	2.07 (1.09)	1.64 (1.15)	2.20 (1.00)	2.66 (1.02)	2.70 (1.17)	2.63 (0.89)
Q10: I was interested in the topic of the lesson**	2.15 (1.06)	2.50 (1.34)	2.44 (0.92)	2.80 (1.24)	2.80 (1.15)	2.81 (1.42)
Q11: The pace of communicated information was too fast	1.12 (1.15)	0.79 (0.97)	1.24 (1.23)	1.58 (1.13)	1.45 (1.15)	1.75 (1.18)
Q12: The lesson encouraged me to advance my knowledge in the fields of biology and anatomy	1.90 (1.20)	2.29 (1.44)	1.68 (0.95)	2.00 (1.17)	2.00 (1.17)	2.00 (1.26)

Note(s): *Enthusiast p < α = 0.05; **Non-enthusiast p < α = 0.05

Source(s): Authors’ own work

Conflict of interest: The authors declare that they have no conflict of interest.