How do students perceive their own and their peers' progress in e-learning?
International Journal of Information and Learning Technology
ISSN: 2056-4880
Publication date: 4 November 2020
Abstract
Purpose
Interest is currently growing in open social learner modeling (OSLM), which means making peer models and a learner's own model visible to encourage users in e-learning. The purpose of this study is to examine students' views about the OSLM in an e-learning system.
Design/methodology/approach
This case study was conducted with 40 undergraduate students enrolled in advanced programming and database management system courses. A Likert-type questionnaire and open-ended questions were used to obtain the students' views. System usage data were also analyzed to ensure the richness and diversity of the overall data set.
Findings
The quantitative data of the students' views were analyzed with descriptive statistics; the results are presented as graphics. The qualitative data of the students' views were examined by content analysis to derive themes. These themes are organized into four subtopics: the students' positive views, their negative views, their improvement suggestions and their preferences about using similar OSLM visualizations in other e-learning systems. The students' subjective views are discussed in the context of their recorded interactions with the system.
Research limitations/implications
Competition due to seeing peer models was considered by participants both as positive and negative features of the learning system. So, this study revealed that, the ways to combine peer learner models to e-learning systems that promote positive competition without resulting social pressure, still need to be explored.
Practical implications
By combining open learner models with open peer models, OSLM enhances the learning process in three different ways: it supports self-regulation, encourages competition and empowers self-evaluation. To take advantage of these positive contributions, practitioners should consider enhancing e-learning systems with both own learner and peer model features.
Originality/value
Despite increasing interest in OSLM studies, several limitations and problems must be addressed such as sparsity of data and lack of study of different contexts and cultures. To date, no published study in this area exists in Turkey. The purpose of this study is to fill this gap by examining OSLM features in an e-learning system from the perspectives of Turkish students by using both their system interaction data and their subjective views.
Keywords
Citation
Somyürek, S., Brusilovsky, P., Çebi, A., Akhüseyinoğlu, K. and Güyer, T. (2020), "How do students perceive their own and their peers' progress in e-learning?", International Journal of Information and Learning Technology, Vol. 38 No. 1, pp. 49-74. https://doi.org/10.1108/IJILT-05-2020-0073
Download as .RISPublisher
:Emerald Publishing Limited
Copyright © 2020, Emerald Publishing Limited
1. Introduction
In recent years, several major trends have shaped global education. Among them, massive online open courses (MOOCs), flipped classrooms, mobile learning and open educational resources are directly or indirectly related to online training (Brown et al., 2020). Because online training expands educational opportunities beyond traditional formats and borders, this approach is becoming increasingly popular. Online training facilitates the completion of professional certificates for various undergraduate and postgraduate programs, provides in-demand skills, knowledge, competencies via specific courses and supports face-to-face learning by offering additional activities. Recently, online training also has become a necessity rather than a choice for many people due to the global pandemic of COVID-19, which has led several countries to announce cancellations of in-person classes, lectures and seminars, and to move most educational activities online.
According to the Horizon Report 2020, certain emerging technologies and practices related to online training, such as adaptive learning systems and learning analytics (Brown et al., 2020), may improve learning under particular conditions. While adaptive learning systems and learning analytics (LA) have different goals and attempt to support the learning process in different ways, both areas pay increased attention to visualization of student information using skill meters, graphs, word clouds and network diagrams (Bull et al., 2016). The main purpose of adaptive learning systems is to personalize the learning process for each individual user (Brusilovsky, 2001). Modeling a learner through a structured representation of that learner's knowledge, misconceptions and difficulties (Bull, 2004) is the core element of these systems (Brusilovsky et al., 2015). In traditional adaptive learning systems, the learner model is an internal mechanism that is used to make correct inferences about the student and to provide appropriate content and navigation adaptations for that student. However, an increasingly popular research stream in the area of adaptive learning systems researchers demonstrated an strong positive value of making learner models visible and explorable by the learners through open learner models (OLMs) (Bull, 2004).
Presenting student information in visual form through various kinds of dashboards has been also an important direction of research in the area of LA. While these dashboards were originally focused on stakeholders who are responsible for managing of the learning process such as instructors or parents, a more recent, but increasingly active stream of research examined the value of presenting LA data to the learners themselves through student facing LA dashboards (Bodily et al., 2018). As stressed by Bodily et al. (2018) in their analysis of similarities and differences between OLMs and student facing LA dashboards, although OLMs are based on user modeling and student facing LA dashboards are based on data-driven decision-making, they are very similar. They summarize these similarities and differences as following: (1) OLM research includes more system evaluation through usability tests, perception surveys and randomized control trial experiments than LA dashboards research; (2) Behavioral metrics about learner such as discussion board views, page views, number of assignments submitted are used more extensively in LA dashboards; (3) Visualization of assessment data are used more extensively in OLM; (4) Both OLM and LA work included opportunity for comparison with peers or a standard for the course. However, a higher number of OLM tools that include such functionality were existed as opposed to LA dashboards. (5) The systems more frequently allow the learner to interact with the OLM in some way in OLM research than LA dashboards research. Bull (2016) suggested combining the power of OLMs with LA dashboards.
The OLM is considered to have a deep impact to raise awareness in e-learning because of its strong psychic-pedagogical foundation (Ferreira et al., 2019). Positive results of OLM have been reported in empirical studies, such as the promotion of the cognitive, metacognitive and motivational components in learning (Ferreira et al., 2019; Hooshyar et al., 2019). In the most recent review of the OLM field, Bull (2020) also highlighted results of earlier studies that demonstrated advantages of allowing students to use visual OLM format to compare themselves with peers. In this paper, we refer to an extended version of the OLM that allows viewing OLMs of peers or peer communities as open social learner models (OSLM). Among other values, Bull pointed out that because accessing peer models offers users more data than the learner's own model, this helps the students to set goals and collaborate; it also helps parents and instructors to more effectively support learning. According to prior studies, some other positive effects of the OSLM compared to the OLM are enhanced engagement (Brusilovsky et al., 2015; Guerra et al., 2016), greater efficiency (Brusilovsky et al., 2015; Guerra et al., 2016), improved self-assessments (Kerly and Bull, 2008; Somyürek and Brusilovsky, 2015; Suleman et al., 2016; Somyürek et al., 2020), enhanced effectiveness (Guerra et al., 2016) and increased persistence (Barria-Pineda et al., 2018).
Despite increasing interest in OLM studies, several limitations and problems must be addressed. According to the literature review conducted by Jivet et al. (2018), one of the most important shortcomings in these studies is a sparsity of data; they advised that “complementing the feedback gathered through self-reports with usage data… will provide more credibility to the results.” Fournier et al. (2011) stated that qualitative data could be used to support quantitative data, to permit deeper analyses of learning. Thus, researchers should collect both qualitative and quantitative evidence for the most useful results in future OLM studies. As well as other qualitative data, the students' perceptions toward OSLM features are needed in order to understand the potential value of making learner's own and peer models visible for learning and to enhance e-learning systems. Since perception is directly affecting human behavior, it is so important to be aware of student perceptions about specific features and functions of learning systems. As another issue, extending the OLM with peer models can be useful, but “what is effective in one context or in some cultures may not work as well in others” (Bull, 2020). We also should remember that students' cultural characteristics may affect their views about the OSLM, and/or may vary the effects of the OSLM on their behaviors and emotional states. Hooshyar et al. (2020) revealed that most of the OLM / OSLM related studies were conducted in the UK, the USA, Australia and New Zealand. To our knowledge, no study on the OSLM has thus far been conducted in Turkey. So, conducting this research in Turkey may contribute to the literature to investigate the effects of culture on perspectives of students on OSLM features in an e-learning system.
1.1 Research questions
How do the students evaluate the e-learning system including OLM and OSLM features?
What are the students' views about the e-learning system including OLM and OSLM features?
2. Method
This research was carried out as a case study that allowed examining students' views about the OSLM in an e-learning system with an in-depth and holistic perspective. “The case study offers a means of investigating complex social units consisting of multiple variables of potential importance in understanding the phenomenon.” (Merriam, 2009, p. 50).
2.1 Participants
The participants were students in the Computer Education and Instructional Technologies (CEIT) department of a public university in Turkey. In total, 22 students were enrolled in advanced programming, and 28 students were enrolled in the database management system course. Of these, 10 students never used the e-learning system, and 15 did not answer the questionnaire. Hence, for our system usage analysis, we analyzed the data of 40 students, and for the subjective views analysis, we analyzed 35 students' responses to our Likert-type questionnaire. Of those 40 students, 19 (47.5%) were female, and 21 (52.5%) were male. The participants were coded as “P1, P2,…, P40” to associate them with their quoted statements while maintaining confidentiality.
2.2 The e-learning system
2.2.1 Content
The e-learning system was developed by a research team at a university in the USA, and the content and interfaces then were translated into Turkish. The e-learning system allows access to three types of interactive learning content for Structured Query Language (SQL): examples, animations and questions.
Examples: Worked examples designed as a piece of SQL code are provided to demonstrate how to solve a problem. For example, Figure 1 shows a worked example from the topic Table Creation; it illustrates an SQL command to create a table using SQL.
When a student clicks on any line in the example, he/she can view the explanation about the SQL command on that line. For example, when the student clicks the second line, he/she can see the explanation of the following SQL clause:
ogr_no VARCHAR(20) NOT NULL,
Column named ogr_no, which is character domain type, has 20-digit space and does not allow NULL value.
In the body of the CREATE TABLE clause, the columns of the newly created table are defined. Firstly, every column in the table must have a unique name and domain name is to identify the kind of data that the column stores. Lastly, required data are to determine whether the column contains required data and prevents NULL values from appearing in the column; otherwise, NULL values are allowed.
Animations: Animations are used to explain the worked examples more dynamically. These animations demonstrate the execution of the SQL commands step-by-step. Students can control the flow of the animation using the navigational controls to play, stop or resume the animation. Students can also use these controls to replay important parts as often as they require (see Figure 2).
Questions: Questions are used as practice activities that offer learners a chance to apply their SQL knowledge. As shown in Figure 3, the task is provided at the top of the screen, and the related database schema information is shown at the bottom; the latter includes tables, columns and sample records. The students are required to write an SQL command to complete the given task. While preparing this, they can practice by clicking on the “Open SQL-Lab” button to input their command and view the result, including any errors. If they wish to submit their final answer, they click on the “Send answer” button.
2.2.2 Visualization of the OSLM
The visualization of a learner's model in the e-learning system is implemented using skill meters. The skill meters are presented as grids that display the progress of the student, the class average and comparison of the student's progress against the class average for each topic. As shown in Figure 4, the OSLM includes three rows.
The first row, “Me,” displays the students' progress in the topics. The color of the grids ranges in a continuum from gray to green. The intensity of the green indicates the degree of completion of the topic. Darker green means more progress has been made in the topic. This first row could be called an OLM.
The third row, “Group,” displays the aggregated peer model. Darker blue indicates greater class progress in that topic.
The second row, “Me vs group,” displays the learner's progress compared to the average of the group (i.e. the class). Darker green indicates that the learner has progressed further than the group; darker blue means that the group has progressed further than the targeted learner; gray means equal progress.
The visualization of the learner model also presents the course structure and provides navigational support because it shows the SQL topics covered in the course. The course content is grouped under 18 subtopics, including Table Creation with SQL and Derived Relations and Views. When a student clicks on any grid cell (topic), he/she can see the contents (examples, questions, and animations) of that topic (Figure 5).
The students also can see peer models in the class by clicking the “load other students” button. The individual peer models are presented as separate rows and are anonymous. The student also can see her/his rank in these ordered peer models (Figure 6).
2.3 Procedure
This study was conducted for eight weeks in the advanced programming and database management system courses in a public university. In the first week, the researchers introduced the system in a live demonstration and explained how to use it. Students could use the e-learning system to study course-related tasks during last hours of face-to-face class or to repeat and practice in the course topics during extra-curricular hours online. The students were encouraged to use the e-learning system, which included several examples and practice activities. However, they were not forced to use the system. At the end of eight weeks, each participant was asked to complete two different instruments to evaluate the e-learning system.
2.4 Instruments
In this study, data were collected with three different instruments: questionnaire, structured interview form and system log data. Firstly, to assess the students' views about the OSLM in this e-learning system, a questionnaire was developed by the researchers. The questionnaire was designed with four sections, including 33 five-point Likert-type statements overall. The first section includes six items to assess the students' acceptance of the e-learning system. The second section includes eight items to assess their views about the OLM. The third section includes four items about the OSLM. The final section contains 15 items directed at assessing the students' acceptance of each type of content (examples, animated examples, quizzes). The students were asked to rate each statement on a scale ranging from 1 to 5 (strongly agree, agree, neutral, disagree, strongly disagree).
Secondly, to understand the students' views about the e-learning system in more detail, structured interview form including four open-ended questions was developed. This additional instrument was used to obtain a deeper insight into the students' views compared to the Likert-type questions. These open-ended questions focus on positive and negative aspects of the e-learning system, the students' improvement suggestions regarding the system, and their preferences about using similar OSLM visualizations in other e-learning systems, such as Edmodo or Moodle.
Finally, the log data obtained from the students' interactions with the system was used to support their views data. The log data include several metrics, such as the students' total number of logins in the system, their total number of peer model views, and their total number of accesses to examples, animations and questions.
2.5 Data analysis
Triangulation was used to overcome problems concerning the single data collection method and to ensure the diversity of the data. First, we used descriptive statistics to analyze the students' answers in the Likert-type questionnaire. The frequencies of the students' ratings were calculated, and stacked graphs were used to visualize these results. Second, content analysis was conducted to examine the students' answers to the open-ended questions. Content analysis provides a systematic way to analyze the large amounts of text data by searching for and classifying meaningful structures in the content. For this purpose, the researchers read the answers to the open-ended questions carefully, and then they created meaningful codes to categorize the texts, and finally grouped similar codes under derived themes. For the qualitative data, the first analysis was repeated after three weeks, and then the consistency of these two analyses was evaluated. To analyze the system log data, descriptive statistics and correlation analysis were used.
3. Results
To answer the first research question “How do the students evaluate the e-learning system including OLM and OSLM features?”, the quantitative results of the students' views about the learning system obtained through Likert-type questionnaire were used. These results were presented under the following subtopics, students' acceptance of the e-learning system, their views about the OLM, their views about the OSLM, and their acceptance of the different content types. To answer the second research question “What are the students' views about the e-learning system including OLM and OSLM features?”, the qualitative results of the students' views about the learning system obtained through open-ended questions were used. The results of independent content analyses of each of four open-ended questions were presented in following subsections: the students' positive views, their negative views, their improvement suggestions, and their preferences about the use of similar OSLM visualizations in other e-learning systems. In these subsections, identified themes derived from the analyzed, coded and combined the students' views were presented. Log-data analysis was also used to support in some of subresearch questions.
3.1 Students evaluations on the e-learning system including OLM and OSLM features
3.1.1 The students' acceptance of the e-learning system
The first subtopic, on the students' acceptance of the system, was assessed using six items in the questionnaire (Figure 7). The majority of the students agreed or strongly agreed that (I5) they liked to use the system (n = 29, 82.86%). About three-quarters of the students agreed or strongly agreed that (I3) the system was easy to use (n = 25, 71.43%), and (I6) they would like to use it in other courses (n = 27, 77.14%). More than half the students agreed or strongly agreed that (I1) using the system improved their academic performance in the course (n = 22, 62.86%); (I2) using the system, it was easier for them to study the course material (n = 22, 62.86%); and (I4) they could use the system without the need to be told how it functions (n = 22, 62.86%). Overall, the students accepted the e-learning system positively. However, there were also some neutral evaluations and a few (maximum six out of 35 students) negative evaluations. Among these six items, the most positive ratings were assigned to the fifth item, “like to use the system.”
3.1.2 The students' views about the OLM
The second subtopic is related to the results of the students' ratings of eight items about the OLM in the e-learning system (Figure 8). A majority of the students agreed or strongly agreed that (I7) it was useful to see their progress in the e-learning system (n = 31, 88.57%), (I9) seeing their progress in the tool motivated them to work on the available resources (n = 29, 82.86%), (I11) the interface helped them to identify their weak points (n = 29, 82.86%), (I13) it was useful to see their progress for each topic (n = 33, 94.29%), and (I14) it was useful to see their progress within different content types (n = 32, 91.43%). About three-quarters of the students agreed or strongly agreed that (I8) they liked the interface in general (n = 27, 77.14%), (I10) the interface helped them to understand how the class content is organized (n = 27, 77.14%), and (I12) the interface helped them to plan their classwork (n = 26, 74.29%). Overall, the students' views were positive concerning their individual learner model in the e-learning system. The most positive ratings were assigned to the thirteenth and fourteenth items, on the usefulness of the system to allow students to see their progress for each topic and for different content types.
3.1.3 The students' views about the OSLM
The third subtopic is focused on the results of the students' ratings of four items about the OSLM in the e-learning system (Figure 9). A majority of the students agreed or strongly agreed that (I17) comparing their progress with the rest of the group helped them to identify their weak points (n = 29, 82.86%), and (I18) using green and blue colors in different intensities to show their progress and the class's progress was easy to understand (n = 30, 85.71%). Approximately three-quarters of the students agreed or strongly agreed that (I15) the ability to see the progress of the rest of the group made the system more valuable to them (n = 26, 74.29%), and (I16) the ability to see the progress of the rest of the group motivated them to use the system more frequently (n = 25, 71.43%). These results indicate that the students' views about accessing the peer models in the e-learning system were generally positive. The most positive ratings were assigned to the seventeenth and eighteenth items, on the ease of use of the OSLM and its usefulness to them in identifying their weak points.
We also conducted statistical analysis on their log data to see whether the students' efforts to see the progress of the rest of the group caused them to use the system more frequently, as they stated in their subjective ratings. For this purpose, correlation analysis was conducted, which examined their clicks on the “load other students” button and other usage data. We found that there was a high positive correlation between the total number of clicks on the “load other students” button and the total number of logins to the load numbers of the system (r = 0.744; p < 0.01). Moderate positive correlations were found between the click numbers on the “load other students” button and the total number of second attempts for the questions (r = 0.655; p < 0.01), animated example lines (r = 0.684; p < 0.01), visited topics (r = 0.582, p < 0.01), and interactions with the learning content (i.e. unique animations (r = 0.623, p < 0.01), unique examples (r = 0.561, p < 0.01) and unique questions (r = 0.486, p < 0.01)). When we only focused on the questions, moderate positive correlations were found between the click numbers on the “load other students” button and successful question attempts (r = 0.513; p < 0.01), and low positive correlations between successfully answered different questions (r = 0.486; p < 0.01), successful answers in third attempts (r = 0.442; p < 0.01) and successful answers in first attempts (r = 0.461; p < 0.01).
All these positive correlations indicate that the students' objective usage data supports their subjective views about the OSLM features that made the system more valuable and motivated them to use the system more frequently.
3.1.4 The students' acceptance of the different types of content
The fourth and final subtopic focuses on the students' ratings of five items (applied for each content type) on their acceptance of the different types of content (i.e. the examples, animated examples and quizzes) (Figure 10). A majority of the students agreed or strongly agreed that (I19) the examples were a valuable learning activity type (n = 33, 94.29%), (I20) the examples helped them in learning the course concepts (n = 32, 91.43%), (I22) it was easy to use the examples (n = 33, 94.29%), and (I23) they would recommend using the examples to other students (n = 33, 94.29%). Approximately three-quarters of the students agreed or strongly agreed that (I21) they enjoyed working with the examples (n = 27, 77.14%). These results indicate that the students generally had positive views about the examples. The most positive ratings were assigned to the nineteenth, twenty-second and twenty-third items, about seeing the examples as valuable, their ease of use, and recommending them to others. Among these five questions, the least positive ratings were assigned to the pleasure of working with the examples.
As shown in Figure 11, a majority of the students agreed or strongly agreed that (I24) the animations were a valuable learning activity type (n = 34, 97.14%), (I25) the animations helped them to learn the course concepts (n = 32, 91.43%), (I26) they enjoyed working with the animations (n = 30, 85.71%), (I27) it was easy to use the animations (n = 34, 97.14%), and (I28) they would recommend using the animations to other students (n = 33, 94.29%). The most positive ratings were assigned to the twenty-fourth and twenty-seventh items, about seeing the animations as valuable and their ease of use. These results further indicate that the animations were more positively rated compared to the examples, and more students stated that they enjoyed using them.
In Figure 12, we can see a majority of the students agreed or strongly agreed that (I29) the questions were a valuable learning activity type (n = 31, 88.57%), (I30) the questions helped them in learning the course concepts (n = 31, 88.57%), (I32) it was easy to use the questions (n = 29, 82.86%), and (I33) they would recommend using the questions to other students (n = 29, 82.86%). Approximately, three-quarters of the students agreed or strongly agreed that (I31) they enjoyed working with the questions (n = 25, 71.43%). The most positive ratings were assigned to the twenty-ninth and thirteenth items, about seeing the questions as valuable and their usefulness for learning the course concepts. Although the students generally reacted positively toward the questions, fewer students assigned positive ratings to all five statements compared to the other two content types.
According to the number of the students' positive ratings, the order (from most to least positive) of their ranking of the types of content is the animations, the examples, and then the questions. When we checked the students' log data for their usage of these contents, we found the following. The average number of visits to various examples was 26.58 (42.19%; there were a total of 63 examples), the average number of visits to animations was 19.76 (34.67%; there were a total of 57 animations), and the average number of solved questions was 16.30 (33.27%; there were a total of 49 questions). The questions were the least used content type, as we had expected from the subjective ratings. However, though the animations were rated more positively than the examples by a few of the students, we found that their usage was less. To examine this situation, only the usage data of those students who reported enjoying the animations more than the examples were analyzed. According to that criterion, we found that their average usage of the animations was greater than their average usage of the examples (see Table 1). The opposite situation was found regarding those students who more highly rated their enjoyment of the examples than the animations.
3.2 Students' views about the e-learning system including OLM and OSLM features
3.2.1 The students' positive views
The students' positive views about the e-learning system were analyzed and grouped under 11 themes: competition, understanding one's own progress, the opportunity to practice, relative assessments, consolidation, ease of use, usefulness, fun to use while learning, seeing the course structure, appealing quality and permanency of learning (see Figure 13).
3.2.1.1 Competition
Most of the students (n = 27) stated that they preferred to use the system actively because the OSLM interface creates competition among the students. Those students appreciated being able to see their own and their peers' progress juxtaposed.
Some students offered the following views:
I was even more ambitious when I saw my friends ahead of me (P19).
Since it created a competitive system, it encouraged my active use (P10).
When you see that everyone is doing learning activities, you should do it too (P11).
Seeing the class members' progress, in addition to my own level of progress, made me ambitious (P25).
3.2.1.2 Understanding one's own progress
In total, 13 students stated that being able to see their own level of progress is one of the most pleasing features of the system. They liked to evaluate their own work and learning progress.
A student claimed that:
Thanks to this interface, I can easily see the topics that I was missing, I left in the middle, or I had difficulty with (P16).
Another student said:
I planned my learning process better as I saw how far I was progressing (P19).
One other stated:
I understood what topic I was missing because the system allowed me to see my progress. I could look at the topics again whenever I wanted (P24).
3.2.1.3 The opportunity to practice
This was another positive feature in the opinions of several students (n = 10). They liked solving questions by writing SQL statements.
Students indicated the following views about the opportunity to practice:
Thanks to this system, I had the opportunity to practice with the codes I learned in the course before and improved my SQL coding skills (P2).
I learned easier since I could apply several examples (P32).
Practicing is quite useful (P29).
3.2.1.4 Relative assessments
The third most frequently (n = 10) mentioned positive view about the system was the opportunity to make relative evaluations. These students liked to learn their progress by comparing themselves against their peers.
For example, one student said:
I was able to learn my status and rank in class (P6).
Another student stated:
When I saw the class ahead, I understood I have to study faster (P29).
Another one claimed:
The system was useful because we can also see class members' progress in general (P16).
3.2.1.5 Consolidation
One of the most positive features according to four students was knowledge consolidation. They liked to reinforce prior knowledge using the system. Some of the students stated:
It enabled me to better understand the SQL topics in the course and to consolidate them with applications (P25).
We can reinforce the knowledge we learned in face-to-face class hours through this system, which can be accessed from anywhere on the Internet (P13).
3.2.1.6 Ease of use
Some students (n = 4) also liked the ease of use of the system. Perceived ease of use is one of the two primary factors that influence an individual's intention to use a system/technology. Some of their views were:
The system was easy to use and understand (P38).
The animations and examples were very descriptive and very easy to use (P3).
3.2.1.7 Usefulness
Four students' positive views about the system were focused on the system's usefulness. According to the technology acceptance model, perceived usefulness shows “the degree to which a person believes that using a particular system would enhance his or her job performance” (Davis, 1993, p. 477). This is one of the two primary factors that influence an individual's intention to use a system/technology.
3.2.1.8 Fun to use while learning
Three students' positive views about the system were related to how fun it was to use. They stated two different reasons for having fun when they were learning. These reasons were seeing different colors in the grids that represent their progress in topics and learning with both examples and questions to practice.
3.2.1.9 Seeing the course structure
Some students (n = 3) stated that seeing the organization and sequencing of the course content is one of the most positive aspects of the e-learning system. Though the course syllabus was distributed at the beginning of the term, the students liked to see all the topics clearly posted when they log into the system. Some of their statements were:
It was one of the best features of the system to see all the topics, from the first to the last (P5).
Being able to see how many topics are to be covered in the course is so useful (P23).
3.2.1.10 Appealing quality
Two students stated that the included features made the e-learning system appealing to them. One student claimed that the OSLM made the system more appealing, and other thought the general features made the system appealing. They said:
It was very attractive to see the difference in our studying process compared to our friends (P30).
The system was attractive, so I used it actively (P7).
3.2.1.11 Permanency of learning
Two students (n = 2) stated that using the system facilitated more permanent learning. They thought interacting with different content types and reinforcing materials helped them to internalize information taught in the face-to-face course.
One student commented:
Using different content types, such as examples, animations, and questions, led to more effective, faster, and permanent learning. (P9)
Another student stated:
Working with this system made the knowledge in the course more permanent. (P10)
3.2.2 The students' negative views
It is noteworthy that the students' negative views were much fewer in number than their positive ones. Only 11 negative views were obtained from the students about the system. When these few comments were grouped, competition, demotivation and anxiety were derived as themes (see Figure 14).
3.2.2.1 Demotivation
Some students (n = 6) claimed that when students see that their progress is generally behind that of the class, they may feel demotivated. So, they thought displaying insufficient progress ratings might be a demotivator. For example, one student said:
If I have not spent much time in the system, or I am having difficulty with the topics and at the end in the rankings, my heart would be broken. (P16)
The motivation of students who do not understand the topics or have difficulties in answering questions may be negatively affected. (P18)
3.2.2.2 Competition
Though competition was perceived as a positive feature by most of the students, some of them (n = 4) thought the opposite. The following comments are some of the students' negative views about competition:
Problems may arise among friends because of competition. (P28)
Overly ambitious students may be negatively affected by seeing more successful peers. (P27)
3.2.2.3 Anxiety
One student stated that the OSLM features might cause anxiety for students:
Students may feel like a failure when they see everybody is successful and they are not. (P11)
3.2.3 Improvement suggestions
As shown in Figure 15, four themes (visual design, content, the OSLM and additional tools/functions) were derived from the students' improvement suggestions.
3.2.3.1 Visual design
The most frequently (n = 11) stated improvement suggestions about system were related to its visual design features. Several students claimed the design is too plain and that the colors should be changed (n = 10). One of them stated that the font size should be changed in addition to the color (n = 1). Some of the students said:
I would suggest creating a more attractive interface by adding some visual items that cover the entire page. (P38)
Using more vivid colors as opposed to the current colors would be beneficial (P9).
3.2.3.2 Content
Seven students made suggestions about improving the content in the e-learning system. These suggestions generally focus on providing tutorials, adding more and more varied examples, improving the content with narratives and videos, providing a feedback mechanism and offering a more detailed help feature to explain the OSLM. Some of their statements were:
It would be nice if there were audio narrations about the examples and animations. (P1)
I would like to see the captured video tutorials about the topics. (P10)
The questions should be improved. When I give a wrong answer, I would like to see the clues first and then the correct answer. (P29)
More examples and improved animations could be added. (P40)
A more detailed guide about the how the shades of the colors are determined would be helpful (P9).
3.2.3.3 The OSLM
Four suggestions were focused on the OSLM. Two students stated that they would like to see real student names in the OSLM, and the other two students expressed the exact opposite view. Their statements were:
I would like to see the names of my classmates in the peer models, so that we could understand which students are better than others (P19).
I would like to remove all class progress and related features from the system. (P23)
I think that just aggregated class progress is enough, and each student's progress should be removed from the system (P12).
When the log data were analyzed, we found that P12 and P23 used the system only a few times; therefore, they were both below the class average in usage and also at the bottom of the rankings in the peer models. P12 visited just one example and one animated example for one topic and did not solve any questions. Similarly, P23 visited two examples and one animated example for one topic and did not solve any questions. Therefore, their suggestions, one about removing all the OSLM features and the other about removing peer models, are likely due to the fact that their progress was very low compared to others in the class, and they did not like to see this. On the other hand, according to the interaction data, P19 browsed all 63 examples and 21 animated examples, visited all 18 topics and solved 21 different questions. So, P19 was above the average of the class and even at the top of the rankings. In this case, we surmised that P19 wanted others to see P19's name at the very top of the rankings in the peer models. Interestingly, the other student (P32), who suggested showing the learners' names in the peer models, was below the average of the class in usage and at the bottom of the rankings. This student visited eight examples and four animated examples for three topics and did not solve any questions. Under these circumstances, this student's suggestion about including real names was surprising.
Since these views were obtained from only a few students' responses to the open-ended questions, to see a more general tendency, we also examined the answers offered for the OSLM questions by those students who had top rankings and those who were at the bottom of the rankings in the peer models. To do this, we clustered the users according to their system usage, with more usage representing a better peer model and less usage representing a worse peer model. We obtained three clusters: high, low and medium. We then found that 25% of the higher performers disagreed with the Likert-scaled statement that “the ability to see the progress of the rest of the group makes the system more valuable for them.” Among the lower performers, 38.5% disagreed with that statement. But we also found that 62.5% of the higher performers and 61.6% of the lower performers agreed with the statement. Thus, more than half of the students believed that the OSLM makes the system more valuable, regardless of their performance, but more students with poor performance did not find the OSLM valuable.
3.2.3.4 Additional tools/functions
Two students suggested that additional tools or functions could be added to the system. One of these is related to scaffolding, and the other is a rewards system. These two students said:
It might be better if we could contact our instructor through this system when we have difficulty. (P25)
A rewards system could be added for successful students (P27).
3.2.4 The students' preferences about using similar OSLM visualizations in other systems
A majority of the students (n = 38) stated that they would like to use similar OSLM visualizations, so they could they see their and peers' progress in other in other systems such as Edmodo or Moodle. Two students stated that they would not like to use similar systems. The reasons for the students' preferences were also examined. Their positive views were grouped into three themes: understanding one's own progress, competition and other (see Figure 16). Only one theme, competition, was derived from the students' negative views. Nine students did not specify any reason for their opinions.
3.2.4.1 Understanding one's own progress
Nine students stated that they would like to use similar OSLM visualizations because it helped them to understand their own progress. Actually, most of the comments were similar to the comments about assessing one's own progress within the students' positive views about the e-learning system (above). One variance is that here they emphasized the importance for their improved learning performance of seeing visualized presentations of their progress in different learning management system (LMS)-related activities, especially assignments. Some of their comments were:
It is important to see our progress in our assignments. Then we could plan our studying according to this knowledge. (P17)
It will be useful to see my progress in the course in different activities that I would like to use. (P24)
In this way, I can identify and complete my missing assignments (P13).
3.2.4.2 Competition
The students' positive views on this point also were similar to those in the competition theme within their positive views about the system (above). Additionally, here they stated their desire to see similar social features in other online learning systems especially for assignment-related activities. Some of their comments were:
An interface that shows who submitted the assignment and who did not would be good to see. (P32)
We can see what others are doing in the system and, by nature, we do not want to accept being one of the low performers. In this way, we can see our progress ranking and try to increase it. (P35)
Two students identified competition as a reason that they would not like use OSLM visualizations in other systems. One of them said:
Since it will cause competition among the students, I would not like to use similar OSLM visualizations. (P12)
3.2.4.3 Other
The other 12 reasons were irrelevant to the OSLM features. They were more about the benefits of work examples, the animations, etc. Since these views were already presented in the section on positive views about the e-learning system, they are not re-explained here.
4. Discussion and conclusion
The OSLM has become a popular topic for research due to its potentials for visualizing information that guides individuals, peers and instructors to see and compare students' progress and academic weaknesses, so that adaptions can be made (Ferreria et al., 2019) and student motivation can be increased (Hsiao and Brusilovsky, 2017).
This study utilized both qualitative and quantitative data to explore students' views about an e-learning system which includes OLM/OSLM features. The quantitative results indicate that the participating students generally assessed the e-learning system positively. Their views are discussed under four subtopics: their acceptance of the e-learning system, their views about the OLM, their views about the OSLM and their acceptance of the different content types.
Most of the students accepted the e-learning system positively, but there were also some neutral and a few negative opinions. A majority of the students agreed or strongly agreed that they liked using the system. About three-quarters agreed or strongly agreed that the system was easy to use and they would like to use it in other courses. The general usability and usefulness of the e-learning system were also evaluated positively in a previous study conducted by researchers at a large university in the USA. In that study, a majority of the students were Chinese, with a few students of other ethnicities. Our study included Turkish students in a CEIT department in Turkey. Although there were cultural and domain differences between the students, we found that positive reactions to the overall system were numerous in both test groups.
The students' views about seeing the individual learner model in the e-learning system were also positive. The most positive ratings were assigned to the students' ability to see their progress for each topic and for different content types. Feedback from other studies, obtained using a Likert-type questionnaire (Brusilovsky et al., 2015; Bull and Britland, 2007; Bull and Mabbott, 2006; Bull et al., 2007; Guerra et al., 2016), indicate similar findings that most of the students appreciated being able to see their progress in different online systems and found that useful.
Our results also indicate that the students were pleased by their ability to access peer models in the e-learning system. The most positive ratings highlighted the ease of use of the OSLM and its assistance in helping them to identify their weak points. We conducted statistical analysis on log data to see whether the students' efforts to see the progress of the rest of the group caused them to use the system more frequently, as they stated in their subjective ratings. The results show several positive correlations between peer model viewing activity and system engagement. Those students who wondered about their classmates' progress and most frequently clicked the “load other students” button to see peer models also browsed the animation examples more frequently, and more importantly, they continued to try to solve SQL questions even when their first attempts were wrong. This result suggests that viewing others' progress induced the students not to give up but rather to keep trying to find the correct answer; it also encouraged them to engage more with the system. The log data analysis supports our findings that the OSLM features made the system more valuable and motivated the students to use it more frequently. Subjective views expressed in previous studies are further supported by this finding of generally positive views about the OSLM features and its usefulness (Brusilovsky et al., 2015; Bull and Britland, 2007; Bull and Mabbott, 2006, Bull et al., 2007; Guerra et al., 2016).
The results of the students' ratings for their acceptance of each type of content indicate that they generally liked the examples, animated examples and quizzes. The most positive ratings about the examples were that the students found them valuable, easy to use and would recommend them to other learners. Similarly, the most positive ratings about the animations relate to their views that the animations were valuable and easy to use. The most positive ratings about the questions were that they were valuable and helped the students to learn the course concepts. Furthermore, our results indicate that the animations attracted the most positive ratings; the examples were more positively rated compared to the questions; and especially, more students stated that they enjoyed using the animations. When we checked all the students' log data for usage of these contents, the questions were the least used content type, as we had expected after reviewing the subjective ratings. However, though the animations were rated more positively than the examples, we found their usage frequency to be less. But the data of those students who claimed they enjoyed the animations more than the examples shows that their usage frequencies for the animations were greater than for the examples.
The qualitative results of the students' views about the e-learning system were discussed under three subtopics: the students' positive views, their negative views and their improvement suggestions. Our analysis of their positive views allowed us to derive 11 categorization themes: competition, understanding one's own progress, the opportunity to practice, relative assessments, consolidation, ease of use, usefulness, fun to use while learning, seeing the course structure, appealing quality and permanency of learning. Competition was a primary motivating factor to use the system, followed by understanding one's own progress. That competition was the primary motivating factor to use the system is not surprising because social comparison is considered a driving force behind competition among peers (Festinger, 1954).
Most of the derived themes are related to the OSLM and OLM features of the system. Similar views about the positive aspects of these features were also found in prior studies that used open-ended questions as an assessment tool. Those earlier studies did not group the students' views under themes, but we found that the students' quoted remarks were especially related to three of the themes in our study: competition (Bull and Britland, 2007; Bull and Mabbott, 2006; Bull et al., 2007), understanding one's own progress (Bull et al., 2007) and relative assessments (Bull and Britland, 2007; Bull et al., 2007).
It is noteworthy that the students' negative views were much less numerous than their positive ones. Only 11 negative views were obtained; these were grouped under three themes: demotivation, competition and anxiety. These views were all focused on the peer models and their potentially negative effects on the students' affective states. Some students worried that low performers might lose their motivation to study when they see they are at the very end of the ratings. This result can be explained by the theoretical underpinnings of the social comparison theory, which emphasizes that comparisons made in an effort geared toward self-improvement may result in more negative feelings about the self when comparisons are against higher achievers (Guyer and Vaughan-Johnston, 2018), due to feelings of failure and the threat to self-integrity (Muller and Fayant, 2010). Competition was also a negative factor in the views of some students. Their comments highlighted excessive ambition, which may result in problems for some students due to their friendship relations or inability to tolerate more successful peers. The negative effects of comparing oneself to successful peers were also reported in studies that targeted MOOC settings, wherein student drop rates were incrementally and closely associated with comparisons to peers with higher-rated excellence in their performances (i.e. the more highly-rated the peer/s, the greater the chance for lower-rated students to drop out) (Rogers and Feller, 2016). Thus, while designing an e-learning system which promotes social comparisons and competition, such negative effects should be understood and hopefully mitigated by providing information about less successful peers as well as more successful ones (Rogers and Feller, 2016). But this is still a very challenging issue; on the one hand, promoting positive competition brings out the best of students' abilities and provokes them to demonstrate their maximum potential, while on the other hand, resulting social pressure may lead students to focus on the ratings of their peers excessively and sometimes detrimentally.
Bull and Mabbott (2006) obtained similar results about the negative aspects of the OSLM features. Students' statements in their study likewise could be grouped under two themes in this study: demotivation and anxiety.
Four themes were derived from the students' improvement suggestions that relate to the visual design, content, OLM and additional tools/functions. The most frequently stated improvement suggestions are related to the visual design features and content. We obtained several specific suggestions, such as using brighter colors, improving the content by providing tutorials, adding more and more varied examples and enhancing the content with narratives and videos. We also believe that two additional tools or function suggestions are important to improve the system. One of these is to introduce a rewards system, which is an important gamification strategy in e-learning. Since the students were members of the Y Generation, who usually enjoy playing games, it is not surprising that they suggested a rewards mechanism. The other suggestion is scaffolding, which refers modeling or to the teacher providing help to demonstrate how to solve a problem. Scaffolding is one of the most important elements of a learning process, according to several learning theories, such as the sociocultural theory of Vygotsky or the problem-based learning theory of Jonassen.
Four students' suggestions about the system focused on the OSLM, specifically the peer models. Two students stated that they would like to see real student names in the OSLM; two other students said they do not want to see peer models. According to the interaction data, we found that both the students who did not like peer models at all were lower performers. One of those who wanted to see names in the peer models was one of the very best performers. We surmised that student performance affects their thoughts about the peer models. Our study results contradict those reported in a previous study conducted by Bull et al. (2007). Bull et al. (2007) found that there were no clear differences between the preferences of the stronger and weaker students for including real names in their learner models. Moreover, in that study a student with a higher-rated performance stated a preference to use the peer models anonymously so as not to be thought of as “showing off.” Since the relevant views in our study were obtained from only a few students' comments taken from the open-ended question responses, we additionally compared the answers provided for these OSLM questions between a group of students who were at the top of the rankings vs a second group who were at the bottom in the peer models. We found that more than half the students stated that the OSLM makes the system more valuable, regardless of their performance ratings, but more students with poor performance ratings did not find the OSLM valuable. So, we may deduce that the difference may not be due merely to performance (since not all the low performers commented the same way) but an added factor must be the students' individual differences, such as their mastery goal orientations or their levels of competitiveness. Cultural differences (in Turkey vs the UK, for example) or domain variances (e.g. education vs. engineering) may also affect these differences.
A majority of the students stated that they would like to use similar OSLM visualizations in other e-learning systems, such as Edmodo or Moodle. The two most-stated reasons for this preference include understanding one's own progress and competition. The same two themes were also derived from the students' positive general views about the e-learning system. But here, the students' comments differed in that they emphasized their desire to see their progress in different LMS-related activities, such as assignments.
In conclusion, the results of this study indicate that the OSLM is valuable because it supports both self-regulation (by providing opportunities for students to monitor their own progress) and competition and comparison (due to their ability to see their peers' progress). Therefore, we suggest that practitioners should use e-learning systems, which include OLM and OSLM features together, to take advantage of their positive contributions to the learning process.
5. Limitations and suggestions concerning future research
There are some limitations in this study concerning the use of the system, its contents and scope. In this study, the e-learning system was provided as an additional (and optional) medium to support face-to-face course activities; as a result, students used it only voluntarily. Also, the system includes several worked examples and SQL problems for practice, but no tutorials about the topics. Another limitation is that the content of the system focused exclusively on SQL, whereas the database management systems course in which it was used includes five main component areas of the database development process, namely, requirement analysis, conceptual modeling, logical modeling, physical modeling, and queries. Similarly, the advanced programming course (in which the same SQL-only system was used) includes content on PHP and database operations. So, in both courses approximately only one-third of the overall course content overlaps with the system content. In addition, both courses are oriented toward technical content, so we could observe results only in that context. Finally, our sample sizes were relatively small, and the convenience sampling method was used to recruit students. In follow-up studies, researchers should consider using more comprehensive samples from different domains and even from different cultures. This will help researchers to obtain more varied data, more generalizable results, and the ability to more competently adjust the OSLM in different contexts. Analyzing and adapting to the participating students' individual characteristics, such as their mastery goal orientations or their levels of competitiveness, also may be useful to understand the effects of individual differences on their perceptions or usages of different OSLM features. In this way, learner system design can be customized to suit differing user characteristics.
Figures
Figure 1
A sample screenshot of the e-learning system. The system interface and all learning content were available in both Turkish and English. While the students in classes used Turkish version to make the nature of the interface and learning content clear, this and following figures present the English version
Figure 7
The students' acceptance of the e-learning system [1]
Usage data for the animations and examples
| Students | Visits | Line actions | Completing percentage | |||
|---|---|---|---|---|---|---|
| Animations | Examples | Animations | Examples | Animations | Examples | |
| Enjoyed the animations more | 24.50 | 24.00 | 75.00 | 71.00 | 42.24% | 41.38% |
| Enjoyed the examples more | 29.25 | 33.00 | 162.00 | 207.00 | 46.43% | 52.38% |
Note
“For this and the following Likert-type figures, the questions pertaining to the numbered items are located in the Appendix.”
Questionnaire
Part 1. General system acceptance
Using practice system improves my academic performance in the course
By using practice system, it is easier for me to study the course material
Overall, I believe that practice system is easy to use
I can use practice system without needing to be told how it functions
Overall, I like using practice system
If practice system is available in other courses, I will use it
Part 2. OLM features
(7)In general, it was useful to see my progress in practice system
(8)In general, I liked the interface of practice system
(9)Seeing my progress in the tool motivated me to work on the available resources
(10)The interface helped me to understand how the class content is organized
(11)The interface helped me to identify my weak points
(12)The interface helped me to plan my class work
(13)It was useful to see my progress for each topic
(14)It was useful to see my progress in different content types
Part 3. Social features
(15)The ability to see the progress of the rest of the group makes practice system more valuable for me
(16)The ability to see the progress of the rest of the group motivates me to use the system more frequently
(17)Comparing my progress with the rest of the group helped me to identify my weak points
(18)Using green and blue colors in different intensities to show my progress and the class's progress was easy to understand
Part 4. Acceptance of the different types of content
(19)The examples were a valuable learning activity type
(20)The examples helped me in learning the course concepts
(21)I enjoyed working with the examples
(22)It was easy to use the examples
(23)I would recommend using the examples to other students
(24)The animations were a valuable learning activity type
(25)The animations helped me in learning the course concepts
(26)I enjoyed working with the animations
(27)It was easy to use the animations
(28)I would recommend using the animations to other students
(29)The questions were a valuable learning activity type
(30)The questions helped me in learning the course concepts
(31)I enjoyed working with the questions
(32)It was easy to use the questions
(33)I would recommend using the questions to other students
References
Barria-Pineda, J., Guerra-Hollstein, J. and Brusilovsky, P. (2018), “A fine-grained open learner model for an introductory programming course”, Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, pp. 53-61.
Bodily, R., Kay, J., Aleven, V., Jivet, I., Davis, D., Xhakaj, F. and Verbert, K. (2018), “Open learner models and learning analytics dashboards: a systematic review”, Proceedings of the 8th International Conference on Learning Analytics and Knowledge, ACM, pp. 41-50.
Brown, M., McCormack, M., Reeves, J., Brook, D.C. and Grajek, S. (Eds) (2020), EDUCAUSE Horizon Report: Teaching and Learning Edition, EDUCAUSE, Louisville, CO.
Brusilovsky, P. (2001), “Adaptive hypermedia”, User Modeling and User-Adapted Interaction, Vol. 11 Nos 1-2, pp. 87-110.
Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V. and Durlach, P.J. (2015), “Open social student modeling for personalized learning”, IEEE Transactions on Emerging Topics in Computing, Vol. 4 No. 3, pp. 450-461.
Bull, S. (2004), “Supporting learning with open learner models”, Planning, Vol. 29 No. 14, pp. 1-15.
Bull, S. (2016), “Negotiated learner modelling to maintain today's learner models”, Research and Practice in Technology Enhanced Learning, Vol. 11 No. 1, p. 10.
Bull, S. (2020), “There are open learner models about!”, IEEE Transactions on Learning Technologies, Vol. 3 No. 2, pp. 425-448.
Bull, S. and Britland, M. (2007), “Group interaction prompted by a simple assessed open learner model that can be optionally released to peers”, Proceedings of Workshop on Personalisation in E-Learning Environments at Individual and Group Level (PING), User Modeling, Vol. 2007.
Bull, S. and Mabbott, A. (2006), “20000 inspections of a domain-independent open learner model with individual and comparison views”, International Conference on Intelligent Tutoring Systems, Springer, Berlin, Heidelberg, pp. 422-432.
Bull, S., Mabbott, A. and Abu Issa, A.S. (2007), “UMPTEEN: named and anonymous learner model access for instructors and peers”, International Journal of Artificial Intelligence in Education, Vol. 17 No. 3, pp. 227-253.
Bull, S., Ginon, B., Boscolo, C. and Johnson, M.D. (2016), “Introduction of learning visualisations and metacognitive support in a persuadable open learner model”, in Gasevic, D. and Lynch, G. (Eds), Proceedings of LAK16 6th International Conference on Analytics and Knowledge, pp. 30-39.
Davis, F.D. (1993), “User acceptance of information technology: system characteristics, user perceptions and behavioral impacts”, International Journal of Man-Machine Studies, Vol. 38 No. 3, pp. 475-487.
Ferreira, H., Oliveira, G., Araújo, R., Dorça, F. and Cattelan, R. (2019), “An open model for student assessment visualization”, 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), Vol. 2161, IEEE, pp. 375-379.
Festinger, L. (1954), “A theory of social comparison processes”, Human Relations, Vol. 7 No. 2, pp. 117-140.
Fournier, H., Kop, R. and Sitlia, H. (2011), “The value of learning analytics to networked learning on a personal learning environment”, in Long, P., Siemens, G., Conole, G. and Gasevic, D. (Eds), Proceedings of the 1st International Conference on Learning Analytics and Knowledge, ACM, New York, NY, pp. 104-109.
Guerra, J., Hosseini, R., Somyurek, S. and Brusilovsky, P. (2016), “An intelligent interface for learning content: combining an open learner model and social comparison to support self-regulated learning and engagement”, Proceedings of the 21st International Conference on Intelligent User Interfaces, pp. 152-163.
Guyer, J.J. and Vaughan-Johnston, T.I. (2018), “Upward and downward social comparisons: a brief historical overview”, Encyclopedia of Personality and Individual Differences, Springer, Cham. doi: 10.1007/978-3-319-28099-8_1912-1.
Hooshyar, D., Kori, K., Pedaste, M. and Bardone, E. (2019), “The potential of open learner models to promote active thinking by enhancing self‐regulated learning in online higher education learning environments”, British Journal of Educational Technology, Vol. 50 No. 5, pp. 2365-2386.
Hooshyar, D., Pedaste, M., Saks, K., Leijen, Ä., Bardone, E. and Wang, M. (2020), “Open learner models in supporting self-regulated learning in higher education: a systematic literature review”, Computers and Education, 103878, doi: 10.1016/j.compedu.2020.103878.
Hsiao, I.H. and Brusilovsky, P. (2017), “Guiding and motivating students through open social student modeling: lessons learned”, Teachers College Record, Vol. 119 No. 3, 030308.
Jivet, I., Scheffel, M., Specht, M. and Drachsler, H. (2018), “License to evaluate: preparing learning analytics dashboards for educational practice”, Proceedings of the 8th International Conference on Learning Analytics and Knowledge, pp. 31-40.
Kerly, A. and Bull, S. (2008), “Children's interactions with inspectable and negotiated learner models”, International Conference on Intelligent Tutoring Systems, Springer, Berlin, Heidelberg, pp. 132-141.
Merriam, S. (2009), Qualitative Research: A Guide to Design and Implementation, Jossey-Bass, San Francisco.
Muller, D. and Fayant, M.P. (2010), “On being exposed to superior others: consequences of self‐threatening upward social comparisons”, Social and Personality Psychology Compass, Vol. 4 No. 8, pp. 621-634.
Rogers, T. and Feller, A. (2016), “Discouraged by peer excellence: exposure to exemplary peer performance causes quitting”, Psychological Science, Vol. 27 No. 3, pp. 365-374.
Somyürek, S. and Brusilovsky, P. (2015), “Impact of open social student modeling on self-assessment of performance”, E-learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Association for the Advancement of Computing in Education (AACE), pp. 1181-1188.
Somyürek, S., Brusilovsky, P. and Guerra, J. (2020), “Supporting knowledge monitoring ability: open learner modeling vs. open social learner modeling”, Research and Practice in Technology Enhanced Learning, Vol. 15 No. 1, pp. 1-24.
Suleman, R.M., Mizoguchi, R. and Ikeda, M. (2016), “A new perspective of negotiation-based dialog to enhance metacognitive skills in the context of open learner models”, International Journal of Artificial Intelligence in Education, Vol. 26 No. 4, pp. 1069-1115.
Corresponding author
About the authors
Sibel Somyürek received the bachelor's and master's degrees in the Computer Education and Instructional Technologies department, in 2002 and 2004, respectively, and the Ph.D. degree in Educational Technology in 2008. She is currently an associate professor at Gazi University. She worked on open social learner modeling as a visiting scholar in the School of Information Sciences, University of Pittsburgh. Some of her research interests are adaptive educational hypermedia, individual differences, instructional design and the measures of the hypermedia navigation.
Peter Brusilovsky is a Professor of Information Science and Intelligent Systems at the University of Pittsburgh, where he directs Personalized Adaptive Web Systems (PAWS) lab. He has been working in the field of adaptive educational systems, user modeling and intelligent user interfaces for more than 20 years. He published numerous papers and edited several books on adaptive hypermedia, adaptive educational systems, user modeling and the adaptive Web. Peter is a board member of several journals including User Modeling and User Adapted Interaction, ACM Transactions on Social Computing and International Journal of AI in Education.
Ayça Çebi received her Ph.D. degree in Computer Education and Instructional Technologies from Gazi University. She is currently a researcher at Trabzon University. She worked on adaptive educational hypermedia as a visiting scholar in the School of Information Sciences, University of Pittsburgh. Some of her research interests are adaptive educational hypermedia, educational data mining, distance education and digital competence.
Kamil Akhüseyinoğlu is a Ph.D. student in the School of Computing and Information at the University of Pittsburgh, USA. He is a member of the Personalized Adaptive Web System (PAWS) lab and working at NSF supported computer science education projects. His research interests include designing and developing personalized learning tools for computer science education -specifically for programming- in self-regulated learning environments, personalized educational recommender systems, educational data mining and user modeling. He received his M.S. degree in Information Systems from Middle East Technical University, Turkey. He received his B.S. degree in Computer Engineering from Bilkent University, Turkey.
Tolga Güyer is a researcher at Gazi University, Department of Computer Education and Instructional Technologies. He received his MSc and PhD degrees in Applied Mathematics from Hacettepe and Gazi Universities in 1996 and 2002, respectively. His research interests consist of learning analytics, instructional web design, web metrics and measures, adaptive hypermedia and mathematical modeling of the hyperdocuments.