Lifelong learner needs for human-centered self-regulated learning analytics

Andy Nguyen (Learning and Educational Technology (LET) Research Lab, University of Oulu, Oulu, Finland and Oulu Advanced Research on Service and Information Systems (OASIS) Research Group, University of Oulu, Oulu, Finland)
Joni Lämsä (Learning and Educational Technology (LET) Research Lab, University of Oulu, Oulu, Finland)
Adinda Dwiarie (Learning and Educational Technology (LET) Research Lab, University of Oulu, Oulu, Finland)
Sanna Järvelä (Learning and Educational Technology (LET) Research Lab, University of Oulu, Oulu, Finland)

Information and Learning Sciences

ISSN: 2398-5348

Article publication date: 11 December 2023

Issue publication date: 15 January 2024

1171

Abstract

Purpose

Self-regulated learning (SRL) is crucial for successful learning and lifelong learning in today’s rapidly changing world, yet research has shown that many learners need support for SRL. Recently, learning analytics has offered exciting opportunities for better understanding and supporting SRL. However, substantial endeavors are still needed not only to detect learners’ SRL processes but also to incorporate human values, individual needs and goals into the design and development of self-regulated learning analytics (SRLA). This paper aims to examine the challenges that lifelong learners faced in SRL, their needs and desirable features for SRLA.

Design/methodology/approach

This study triangulated data collected from three groups of educational stakeholders: focus group discussions with lifelong learners (n = 27); five teacher interviews and four expert evaluations. The groups of two or three learners discussed perceived challenges, support needs and willing-to-share data contextualized in each phase of SRL.

Findings

Lifelong learners in professional development programs face challenges in managing their learning time and motivation, and support for time management and motivation can improve their SRL. This paper proposed and evaluated a set of design principles for SRLA.

Originality/value

This paper presents a novel approach for theory-driven participatory design with multistakeholders that involves integrating learners, teachers and experts’ perspectives for designing SRLA. The results of the study will answer the questions of how learners’ voices can be integrated into the design process of SRLA and offer a set the design principles for the future development of SRLA.

Keywords

Citation

Nguyen, A., Lämsä, J., Dwiarie, A. and Järvelä, S. (2024), "Lifelong learner needs for human-centered self-regulated learning analytics", Information and Learning Sciences, Vol. 125 No. 1/2, pp. 68-108. https://doi.org/10.1108/ILS-07-2023-0091

Publisher

:

Emerald Publishing Limited

Copyright © 2023, Andy Nguyen, Joni Lämsä, Adinda Dwiarie and Sanna Järvelä.

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Self-regulated learning (SRL) refers to the cyclical process in which learners actively and reflectively plan and adapt their learning to achieve their goals (Järvelä et al., 2018; Zimmerman, 2000). Being able to regulate one’s own learning is a critical aspect of successful learning and can enhance academic performance and motivation (Theobald, 2021). Furthermore, SRL is essential for lifelong learning because learners need to be able to monitor and control their learning throughout their lives in today’s rapidly changing world (Taranto and Buchanan, 2020). Despite the importance of SRL, many learners struggle with SRL, particularly in the face of complex and demanding learning tasks (Koivuniemi et al., 2017). This can lead to frustration, disengagement and poor performance (Anthonysamy et al., 2020). Some students may need support to regulate their own learning effectively, and achieving self-regulation can be difficult for them.

However, due to the complexity of SRL processes, its measurement has been a major challenge (Järvelä and Bannert, 2021). To address this challenge, researchers have started to use various data streams and learning analytics (LA) to unobtrusively measure facets of the SRL to provide personalized support for the different phases of SRL, even in real-time (Azevedo and Gašević, 2019). At its best, personalized support promotes learners’ SRL and, as a result, helps them to become more successful learners (Lim et al., 2023). Although the advancement of LA has offered several insights and tools to better understand and promote SRL (Järvelä et al., 2023), a significant amount of effort remains to be put into designing LA systems that support learning with the consideration of individual goals, authentic educational needs and ethical principles (Sarmiento and Wise, 2022).

Human-centered LA has gained prominence as a distinct approach to the design and implementation of LA systems that prioritize the needs and perspectives of learners and teachers. Recent calls have asked for learners’ voices to be considered, along with the involvement of teachers in LA design, to increase LA’s transparency, acceptability and successful implementation across contexts (Buckingham Shum et al., 2019). Building on this foundation, participatory design methodologies have been recognized as an effective means for achieving human-centered LA (Bødker et al., 2022). These methodologies actively involve end-users, namely, learners, in the design process, thereby ensuring that the analytics solutions are tailored to their requirements and contextual realities. Nevertheless, as the majority of current human-centered LA studies focus on higher education, it is essential that human-centered LA studies also address other learning contexts, such as lifelong learning and workplace learning.

In this paper, we respond to these recent calls by examining lifelong learner voice for self-regulated learning analytics (SRLA) in lifelong learning. Moreover, the findings from lifelong learner voice are corroborated with their teacher’s voice for validation. In designing LA, it is important to define the stakeholders who can and should participate (Dollinger et al., 2019). Thus, balancing learner voice and teacher voice can be considered because each perspective offers unique insights. On the one hand, learner voice needs to be considered in the design of LA because students are the ultimate users of the system, and their perspectives on how it can benefit their SRL process are essential. On the other hand, teachers have a wealth of knowledge about teaching and learning and their perspectives can help ensure that the system is designed to meet the needs of both students and teachers. This paper examines learners’ challenges and desirable LA features in each SRL phase from both lifelong learners’ and teachers’ perspectives.

Nevertheless, when it comes to human–computer interaction, it is well-known that users do not always know or are certain about what they want, especially with advanced novel technologies that have the potential to change the way people work (Buckingham Shum et al., 2019; Dollinger et al., 2019; Martinez-Maldonado et al., 2021). This concern also applies to human-centered LA, that learners and teachers may not be fully aware of the effectiveness and undesirable consequences of the LA features. As a result, we argue that triangulating learner and teacher voices with domain expert opinions will ensure that the system meets their needs and provides a positive experience. Furthermore, by considering the pertinent learning theories, human-centered LA can be designed based on a solid understanding of the process of learning and is more likely to be effective in supporting learner success.

In this paper, we seek to address the recent calls by examining learners’ needs and aligning those with SRL theory and LA design principles to formulate design principles for SRLA. To address this aim, first, we propose a methodology that considers SRL theory and integrates both learners, teachers and LA experts’ perspectives for designing SRLA. Second, we apply the proposed methodology in a professional development context and illustrate how the methodology can be used to develop a set of design principles for SRLA. In particular, we seek to answer the following research questions (RQs):

RQ1.

How can learners’ and teachers’ voices be integrated with learning and design theories into the design process of SRLA?

RQ2.

What are the design principles for SRLA?

In the following section, we first establish the theoretical foundations for the study by reviewing the literature on SRL in lifelong learning for professional development and SRLA. Then, we introduce our theory-driven participatory design with multistakeholders (TPDM) method for integrating learner needs in establishing design principles for SRLA. We present and evaluate the voice of lifelong learners on their SRL challenges and desirable SRLA features. The design principles for SRLA were established as prescriptive statements that guide the future development of SRLA. After presenting and evaluating the set of design principles for SRLA, we discuss the implications of our study and we conclude by discussing its limitations and future research directions.

2. Theoretical foundations

2.1 Self-regulated learning in lifelong learning for professional development

SRL constitutes a dynamic framework that describes the processes by which learners personally orchestrate their cognitive, metacognitive and motivational strategies to attain their educational goals (Zimmerman, 2000, 2002; Järvelä et al., 2018). Learners who exhibit proficiency in SRL engage in a systematic sequence of planning, implementation and adaptation, which encompasses the setting of specific learning objectives, the continuous monitoring of their progress and the recalibration of their strategies when circumstances necessitate such changes. The significance of SRL transcends the immediate context of academic settings, holding implications for students’ long-term academic success, motivation and engagement (Theobald, 2021). SRL serves as a critical scaffold that not only augments students’ academic performance but also fuels their intrinsic motivation to learn, thereby offering a synergistic effect on their educational outcomes.

SRL extends its importance to the domain of lifelong learning, especially within the scope of professional development. Lifelong learning is an ongoing process extending beyond formal educational structures, often taking the form of informal or nonformal education embedded in daily activities and occupations (Milligan and Littlejohn, 2014). Within the framework of this study, professional development is defined as a type of lifelong learning aimed at enhancing career-specific skills. Contemporary society, characterized by an accelerated pace of technological advancements, mandates an ongoing learning process for individuals to remain adaptable and competent (Taranto and Buchanan, 2020).

In this regard, SRL emerges as an indispensable skill set for continuous growth and adaptability, empowering individuals to meet the challenges and opportunities presented by a digitized and dynamic global landscape. This can occur in various learning settings such as informal workplaces, nonformal educational platforms, as well as online and offline courses. Previous studies confirm the significance of SRL in empowering learners to actively acquire the requisite skills and competencies in a fast-paced, technology-driven society (Taranto and Buchanan, 2020). These findings align with well-established SRL strategies such as self-assessment, goal setting and self-reflection (Zimmerman, 2000). Accordingly, this study focuses on a nonformal online academy designed to teach digital skills over a period of three to six months to adult learners with diverse educational and professional backgrounds.

Research evidence suggests that the capability for SRL is especially pertinent for lifelong learners in professional development settings (Persico et al., 2015, 2020). Milligan and Littlejohn (2014) identify four key behaviors that are integral to SRL in these contexts as: consume, create, connect and contribute. Furthermore, the educational environment in professional development is often less structured, requiring a higher degree of self-regulation from learners (Milligan and Littlejohn, 2014). In such educational landscapes, students exercise greater agency over their learning paths, choosing subjects and pacing autonomously (Manganello et al., 2021). The design of digital skills boot camps, such as the one examined in this study, often incorporates pedagogical techniques that necessitate active participation from the learner (Moshirpour et al., 2019).

However, it should be acknowledged that not all individuals readily use SRL strategies in their learning processes. Challenges in practicing SRL can manifest in the form of poor task management or emotional regulation, ultimately leading to reduced academic performance or disengagement (Koivuniemi et al., 2017; Anthonysamy et al., 2020). Given these challenges, it is imperative that educational settings provide adequate support to facilitate learners’ SRL strategies.

In this study, Zimmerman and Moylan (2009) SRL model is adopted as the theoretical framework for investigating learners’ challenges in the different phases of SRL and the needs for SRLA. The model depicts the SRL process in three phases of SRL in which the output of the previous phase provides input to the next phase: (i) forethought including task analysis and self-motivation beliefs, (ii) performance including monitoring and controlling one’s learning and (iii) self-reflection including of self-judgment and self-reaction. In the context of professional development, learning is mostly informal and learners are required to take decisions not only about how and when to learn but also about what to learn (Manganello et al., 2021).

2.2 Self-regulated learning analytics

The advancement of the LA field has offered unique capabilities for better understanding and supporting SRL. LA refers to “the application of data analytic techniques and tools for the purposes of understanding and enhancing learning and teaching” (Nguyen et al., 2020). The literature has shown that LA delivers remarkable benefits to the learners, such as providing insight in a timely manner to improve outcomes (Knight and Buckingham Shum, 2017), identifying learning needs and providing personalized learning support (Nunn et al., 2016). For instance, Arnold and Pistilli (2012) demonstrated how real-time analytics could be used to identify at-risk students in the early stages of a course by integrating data from various sources, such as grades, course interactions and past academic performance, to generate “traffic light” indicators. Nevertheless, prior studies have called for better alignment between LA design and learning theories to maximize its impact on learning (Gašević et al., 2015). In particular, Heikkinen et al. (2022) noted in their systematic literature review that achieving the full potential of SRLA requires that the different phases of SRL are considered when designing and developing LA solutions.

When learners enact in forethought, performance or self-reflection phases, there are several affordances of LA that can promote SRL (Persico et al., 2015). First, SRLA can help learners to monitor their own learning and to make data-informed decisions on controlling their learning accordingly (Manganello et al., 2021). Second, SRLA can provide personalized scaffolds and guide learners to use specific learning materials or resources that promote SRL (Lim et al., 2023). Third, SRLA could be used to promote self-reflection by engaging learners in self-assessment of their own learning (see Domínguez et al., 2021). While alignment between LA design and learning theories is necessary, it alone is not sufficient to guarantee the design of good LA tools. Therefore, when designing such tools, it is critical to also consider the needs of nontechnical stakeholders, including students and teachers (Giacomin, 2014).

Furthermore, recent research has made significant contributions to the field of LA by offering various strategies that give students and teachers a voice (Alvarez et al., 2020; Buckingham Shum, 2022; Prieto-Alvarez et al., 2018). This has led to the development of human-centered LA in which critical stakeholders (students, teachers), learning and teaching environments and their relationships are identified and considered (Buckingham Shum et al., 2019). Human-centered LA is often achieved through participatory design that offers an innovative approach for ensuring that analytics tools are both effective and closely aligned with the specific needs of learners (Sarmiento and Wise, 2022). First, designing the LA tools with learners and teachers can inform them on the what, how and why the different kinds of data are collected, including how LA tools will acquire and interpret the data to promote SRL (Mangaroska et al., 2021). This increased transparency and acceptability of these tools can also foster their long-term use. Second, by using human-centered LA, both students and teachers may be better equipped to derive meaningful insights and make informed decisions based on SRLA. Namely, learners’ and teachers’ implementations of LA tools may differ from the purposes of designers (Wang and Hannafin, 2005), so designing the interventions with students and teachers may bridge the gap between the desired SRL processes and strategies and those actually realized (Schumacher and Ifenthaler, 2018).

Within the context of SRLA, central to SRL is the notion of learner agency, autonomy and self-reflection, elements that are naturally supported by participatory design methodologies (Zimmerman, 2000). Inviting learners to be active participants in the design of analytics systems ensures that the resultant tools better reflect learner requirements, preferences and contexts (Shum and Ferguson, 2012). Consequently, this participative and multidimensional approach not only yields SRLA tools that are more contextually apt but also reinforces the foundational principles of SRL by encouraging learner input and agency.

Even though the lack of students involvement in the design processes has been acknowledged (Buckingham Shum et al., 2019; Mangaroska et al., 2021), considering only their voice in designing SRLA is questionable because students understanding of effective SRL processes and strategies may be limited (Bjork et al., 2013). Even though the different stakeholders (students and teachers) can have different or even conflicting voices during design processes (Bødker et al., 2022), it is important to investigate both voices to validate design principles for SRLA. Accordingly, this design study seeks to formulate design principles for SRLA by synthesizing the voices of both students and teachers with solid theoretical foundations (Zimmerman and Moylan, 2009) and subsequently subjecting these principles to experts in SRL and LA.

3. Research methodology

To formulate a set of design principles for human-centered SRLA, this study used the well-known design science research methodology in information sciences research (Peffers et al., 2007) in conjunction with a participatory design approach (Bødker et al., 2022). This approach prioritizes the values and concerns of learners as end-users in the design of LA. However, it is important to note that the needs and preferences of learners may conflict with the instructional values held by teachers; such tensions must be addressed. Additionally, in extreme scenarios, the needs and preferences of learners could potentially have adverse effects on their learning outcomes. As such, integrating the perspectives of teachers is imperative for a balanced approach, and the application of learning theories is crucial for guiding the design of human-centered SRLA. Thus, drawing upon the design science research methodology and the participatory design approach, we present our TDPM approach as a methodological framework for formulating design principles for SRLA.

3.1 Design science research methodology

The design science research methodology by Peffers et al. (2007) serves as a comprehensive set of guidelines encompassing principles, practices and protocols for carrying out studies that focus on the design and development of artifacts for information systems, particularly educational technology (e.g. Nguyen et al., 2021). Its core aim is to develop pragmatic solutions to identified issues by emphasizing a multistage process that entails artifact creation, scientific contribution, design evaluation and result dissemination. The resultant artifacts can manifest in various forms, ranging from constructs and models to methods and instantiations, as well as design principles (Baskerville et al., 2018; De Leoz and Petter, 2018).

In the context of the present study, the research output took the form of design principles aimed at human-centered SRLA. The execution of the study followed the structured steps delineated by the design science research methodology:

  • identifying and substantiating the problem;

  • formulating solution objectives;

  • engaging in design and development;

  • demonstrating the artifact;

  • evaluating the artifact; and

  • disseminating findings (Peffers et al., 2007).

In this study, the initial step involved identifying the research problem, which centered on the necessity of supporting SRL for lifelong learners via human-centered SRLA. The study’s objectives, encapsulated in its aim and research questions, focused on investigating the specific needs of learners and crafting design principles tailored for human-centered SRLA. Following this, the stages of design, development, demonstration and evaluation were carried out using a participatory design method (Bødker et al., 2022), adapted into the proposed TPDM approach to fit the study’s objectives. Finally, the study’s results and findings served as the vehicle for communication in line with the design science research methodology.

3.2 Theory-driven participatory design with multistakeholders

In this study, we present an approach for TPDM. Figure 1 illustrates our TPDM approach for integrating learner needs in establishing design principles for SRLA.

Based on SRL theory, learners’ focus group discussions and teacher interviews are triangulated to determine learner needs. Together with design principles for generic learning analytics information systems (LAIS), the findings are incorporated into the conceptualization of SRLA design principles. Both the learner needs and conceptualized design principles are iteratively evaluated by experts based on the following three evaluation criteria (adopted from Venable et al., 2012): 1) evaluate the artefact to establish its utility and efficacy for achieving its stated purpose; 2) evaluate the artefact to identify weaknesses and areas of improvement; and 3) evaluate the artefact to identify side effects or undesirable consequences of its use.

4. Design requirements and inputs – lifelong learners’ self-regulated learning challenges and desirable learning analytics features

In this section, we focus on the essential design requirements and inputs that will inform the foundational principles of SRLA. These guiding elements are derived from the thematic analysis of Learner Focus Group Discussions and Teacher Interviews. By examining these rich narratives, we identify the specific challenges that lifelong learners face in their learning journeys. We also discern the LA features that both learners and teachers consider most desirable for enhancing lifelong learners’ SRL. This multistakeholder insight serves as the cornerstone for establishing empirically grounded and contextually relevant design principles for SRLA.

4.1 Participants and procedures – learner focus group discussion and teacher interviews

The study started with data collection from two groups of participants which are the learners and the teachers. The first group of participants was learners who joined three- to six-months digital skills courses (e.g. Full-Stack Web Development, User Interface and User Experience, Data Science, Digital Marketing) in a nonformal online professional development academy, also known as an online boot camp. Qualitative data collection through focus group discussion was applied to acquire the learners’ voices on the challenges in applying SRL during their learning process and the desirable SRLA features to support them. In total, there were 27 students (n = 27) who formed into ten focus groups consisting of two (three groups) to three learners (seven groups) in each group. All study participants are Indonesian; their demographic characteristics are summarized in Table 1. The topics and guiding questions for the focus group discussion are depicted in Appendix 1.

The second group of participants was the teachers (n = 5) who facilitated the courses and assisted the learners during their learning process in the mentioned online boot camp. The semistructured interview was conducted to gather the teachers’ voices on the observable learners’ challenges and the desirable SRLA features to support the learners in practicing SRL. Similar questions (Appendix 1) were asked to obtain teachers’ perspectives on learners’ challenges in SRL and features that may help the learners improve their SRL.

4.2 Thematic analysis

The data collected from the learners’ focus group discussions and teachers’ semistructured interviews were analyzed using a thematic analysis approach. Thematic analysis is a widely used qualitative method for identifying, analyzing and reporting patterns or themes within the data (Braun and Clarke, 2006). The aim of the thematic analysis in this study was to examine the voices of the learners and validate them with a balance with teacher voice. The integrated voices from both learners and teachers were then conceptualized with iterative expert evaluation for establishing design principles SRLA.

The process of data analysis included several stages, and the NVivo-12 software, a commonly used tool for qualitative analysis, was used in the analysis. Initially, audio recordings of focus group discussions and interviews were transcribed into written form and checked for accuracy. The transcripts were then read several times, guided by the research question and the findings from the literature review, to identify initial codes related to the challenges in SRL and associated needs for SRLA. Following the initial code grouping, the themes were refined, combined or divided in the next stage. A series of revisions and modifications were made to the themes as new information emerged during this stage. During data analysis, a conclusion is reached when it appears that no new themes emerged from the data, and the themes seem to have captured the essence of that data. In the final stage, the themes were elaborated on in detail, and evidence of the themes was provided by relevant quotes from the participants.

4.3 Lifelong learners’ self-regulated learning challenges and desirable learning analytics features

Understanding learners’ expectations of SRLA is essential for meeting their needs. This study examines the challenges of learners and their expected features and subfeatures of SRLA in the different phases of SRL, namely, forethought, performance and self-reflection phases. The results of the thematic analysis on lifelong learners’ focus group discussions and teachers’ semistructured interviews are presented in Table 2, which summarizes lifelong learners’ voices on SRL challenges and needs, and desirable features and subfeatures for SRLA. The frequency of occurrence and examples of statements for each theme are reported in Appendix 2.

Each SRL phase is reported with associated challenges and needs as well as desirable features and subfeatures. Forethought is an important part of SRL, and it involves planning and preparation of learning activities (Zimmerman, 2002; Zimmerman and Moylan, 2009). In this phase, learners anticipate features that will assist them in understanding tasks, setting goals and planning their learning journeys. It includes information and visualizations of the learning path, instructions for tasks and end goals, as well as personalized recommendations tailored to the individual’s motivations. As part of this phase, a comprehensive and engaging dashboard will be needed to set effective goals and plans:

Group 7 – Student 3: “I start making a plan and strategy regarding what I needed to learn day by day. Sometimes, I’m confused to plan the roadmap, like what to learn first. There are so many resources on the internet, but I need a guide on what I need to learn first and how far. Fortunately, after I joined the Bootcamp, I get guidance. I also need the roadmap to keep me motivated and pressured so I will not procrastinate.”

Teacher 1: “It can be applied in the learning dashboard. I imagine it can be like a timeline or milestone which can show two dots. The first dots can be the general milestone that they need to achieve. The other dots show where they are now. So, it can be exciting that we can see where we are currently. So, this is other than the reminder notifications. It can show the end-to-end.”

During the performance phase, learners need features that will help them manage their time, monitor their progress, seek information and receive help and support. This includes task and time management, real-time monitoring, help-seeking, stimulating interests and excitement and rehearsing and constructing knowledge. The challenges in this phase include managing time, tracking progress, efficiently accessing information and receiving the required support:

Group 5 – Student 2: “[…] So, they know their progress each day and accumulate to each week, each chapter, and so on. We can see the tracker, progress, and report. It can be similar to the apps that we usually use to exercise at the gym. We can see the graph for each week, each day, and month. We can see whether it decreases or increases. If it increases, we can set it to keep on that pace. If it decreases, we can reflect, maybe we need to adjust our learning method or why am I being like this, what the cause is, is it because of my condition during the study, or any other personal problem?”

Teacher 3: “[…] motivation also plays a role. For example, I have a student that didn’t come from a technology background. Her motivation is enough to join the course in the beginning, but not enough to drive her until graduation. So, the motivation needs to be updated frequently along the learning journey. For example, actions to provide encouragement and asking questions to check up on their conditions. Those are impactful for the students…So, I think motivational encouragement should not only available in the beginning but also be provided until the end of the learning journey.”

The self-reflection phase requires features that will help learners evaluate their performance and make adaptive improvements. This includes self-evaluation analytics based on various standards and recommendations for improvement. The challenges in this phase include objectively evaluating performance and making adaptive improvements:

Group 4 – Student 1: “A summary of score to show our performance progress, like what things we still need to improve and what we have been good at.”

Teacher 2: “[…] Usually, the challenge is in how they try to acquire data to help them evaluate themselves. Because to evaluate yourself, you need to be able to see from several perspectives […] […] They tend to also see only from one factor […]”

Finally, ethics and data preferences require features that ensure the information is trustworthy and protect the privacy of the learner’s data. This includes data privacy, transparency and security. The challenges in this phase include concerns about sharing personal data and the need for information on data privacy and security:

Group 5 – Student 3: “Personally, as long as, we were asked for consent in the beginning on what kind of data and for what purpose, I think I’m okay if it is for the purpose of the learning analytics.”

Teacher 2: “Based on my understanding, I think there are two keys which are permission and personal data. It is better to not include personal data and for the permission, we stated clearly what data that will be acquired and what will be used. So, this is to protect privacy. But, we need to be very clear on what will be acquired and what it will be used for.”

In conclusion, learners expect SRLA to provide a comprehensive and engaging dashboard with features that support each phase of the learning process, from understanding the task to evaluating performance and ensuring data privacy. These features should help learners manage their time, monitor progress, seek information, receive help and support, evaluate performance and make adaptive improvements while ensuring that the information they use is trustworthy and protects their privacy.

5. Conceptualization of design principles for self-regulated learning analytics for lifelong learners

The conceptualization of design principles for SRLA was informed by the evidence gathered for lifelong-learner and teacher voices (C1–C9 in Table 2) in alignment with SRL theory (Zimmerman, 2000), and guided by the established design principles for generic LAIS (Nguyen et al., 2021) (see Figure 1). The conceptualized design principles were also iteratively evaluated and refined through expert assessment. The conceptualization centers on the activities that the system should afford for addressing learners’ needs as well as promoting SRL to enhance their learning outcomes.

According to Nguyen et al.’s (2021) principle of actionable information, LAIS “should have features that allow for the reporting of actionable information about learners and their learning” (p. 555). Our findings reported that learners need features to keep track of their learning progress toward personal goals and objectives (C1–C4). Importantly, the lifelong learners in this study highlighted the need for personalized recommendations based on their own motivational factors (C2). Accordingly, we argue that SRLA should be able to provide learners with personalized information and visualizations to self-monitor their learning process and prepare for the planned outcomes; thus, the first initial design principle [Init. design principles (DP)] is as follows:

5.1 Init. DP1. Principle of personalized monitoring

SRLA should provide personalized features for learners to self-monitor, self-track and rehearse their learning process toward the planned goals.

With LA, learners can receive personalized recommendations based on their learning interests, capabilities and outcome expectations (Nguyen et al., 2018; Van Schoors et al., 2021). As a result, learners can set more meaningful and achievable learning goals, and make more informed decisions about their futures. Furthermore, aligning learners’ individual learning progress with their own learning goals could also motivate them to better engage in learning. Motivation plays a significant role in SRL as it drives individuals to take initiative and actively engage in the learning process (Zimmerman and Moylan, 2009).

As a result of our findings, learners need analytics to regularly engage them in learning with features that stimulate their interests and motivate them to maintain their learning progress (C3, C6, C8). Accordingly, our initial DP2 was defined as the principle of continuous engagement that recommends SRLA to provide personalized features to stimulate interests and excitement to boost their motivation and engagement in learning. Nevertheless, Nguyen et al. (2021)’s principle of information timeliness recommends LAIS to deliver analytics at the right time for its maximum impact, and learning sciences and decision sciences can provide insight into the best way to design the time latency between data collection and reporting.

5.2 Init. DP2. Principle of continuous engagement

SRLA should provide personalized features to stimulate interest and excitement to maintain learners’ engagement in learning.

A key component of SRL is reflection, which enables learners to assess their learning experiences and use that information to guide future learning (Zimmerman and Moylan, 2009). It is a process of introspection in which learners examine their own experiences, thoughts and emotions. Nevertheless, learners can also reflect through the peer reflection process in which learners reflect on their own experiences and then share their insights and perspectives with their peers for discussion and further reflection. In addition to self-reflection and peer reflection, reflection on feedback from other sources, including LA tools, can also be a form of reflection. In this study, lifelong learners reported challenges to self-evaluate objectively and comprehensively to remain adaptive (C8). It is suggested for SRLA to involve features for acquiring or generating feedback from several sources, including individual and collective feedback from peers and experts.

5.3 Init. DP3. Principle of critical reflection

SRLA should deliver formative and summative feedback based on different standards and from different stakeholders for adaptive improvements.

Research has shown that social interactions and support could significantly influence learners’ SRL (Kwon et al., 2014). Our findings from learners’ focus group discussions emphasized the need for SRLA to offer access to social and psychological support during the learning process (C5). This resulted in our proposed principle of social support as follows:

5.4 Init. DP4. Principle of social support

SRLA should recommend and provide access to social and psychological support.

The principle of social support posits that SRLA systems should extend beyond mere academic metrics to include recommendations and resources for social and psychological support. This extension appears well-founded, given the gaps in the current literature. Traditional SRL theories (e.g. Zimmerman, 2000) predominantly focus on cognitive elements such as planning, time management and meta-cognition. Likewise, existing SRLA systems, as described by Winne (2017), have largely been confined to these dimensions. However, this narrow focus overlooks the significance of affective and social elements, which have been emphasized in broader educational psychology literature. The inclusion of social support aligns with Vygotsky and Cole’s (1978) constructivist theories, Bandura’s (2001) social cognitive theory and Hadwin et al.s (2018) SRL model. Empirical evidence from Chen (2022) and Järvelä et al. (2015) supports this, showing improved student engagement and well-being. However, ethical challenges related to data privacy (Nguyen et al., 2023; Slade and Prinsloo, 2013) and measure accuracy (Ifenthaler and Widanapathirana, 2014) should be carefully considered in future implementations. Accordingly, DP5 was conceptualized to address the ethics and privacy aspects of SRLA as follows:

5.5 Init. DP5. Principle of ethics and privacy

SRLA should provide anonymity and transparency for personal data and protect data against accidental or unlawful destruction or accidental loss, alteration, unauthorized disclosure, or access.

The principle of ethics and privacy calls for SRLA to emphasize anonymity, transparency and robust data protection. This principle complements existing literature that critiques the lack of ethical considerations in traditional educational analytics systems (Slade and Prinsloo, 2013). The call for anonymity aligns with broader trends in information systems emphasizing user control over personal data (Nguyen et al., 2021), while transparency fosters trust, enhancing the effectiveness of SRLA (Buckingham Shum et al., 2019). Additionally, the explicit focus on data protection addresses the real-world risks of data breaches, aligning with regulations like the General Data Protection Regulation (GDPR) in the European Union. In sum, this principle provides a crucial ethical framework for SRLA but demands ongoing commitment to ethical vigilance to adapt to emerging challenges.

6. Evaluation of the design principles for self-regulated learning analytics for lifelong learners

Four experts in the domains of LA and SRL were invited for the evaluation of both learner needs and the conceptualized design principles for SRLA. Three of the experts are working in the field of learning sciences (Expert A: Female, 12 years of experience; Expert B: Female, six years of experience; and Expert C: Male, 15 years of experience), while the other one is in the field of information sciences (Expert D: Female, eight years of experience). All the experts have published scientific papers within the domains of LA and SRL. They are presented with the conceptualized design principles for SRLA and the findings from learner focus group discussions and teacher interviews. After their own reading time, the researchers first walked them through the learners’ needs and their desirable features for SRLA. The experts were asked to provide their assessment based on the provided evaluation criteria (adopted from Venable et al., 2012; see above). They were then requested to evaluate whether the set of conceptualized design principles for SRLA has captured all learner needs, and any improvements needed for design principles to serve as useful guidance for SRLA design and development. The expert evaluation sessions lasted between 40 min and 60 min and were videotaped for later examination in evaluating and refining the design principles for SRLA. All the presented learner needs and design principles were sent to the expert after the evaluation sessions for potentially additional examination and feedback via emails.

In the evaluation, all four experts agreed with the importance of this principle of personalized monitoring (DP1 evaluated). Regarding DP2, a learning sciences expert (Expert A) has raised concerns about “continuous engagement” because excessive engagement in learning can be harmful to learners. Studies have pointed to the phenomenon of “academic burnout,” where constant engagement with educational tasks may lead to stress, emotional exhaustion and reduced academic performance (Schaufeli et al., 2002). Furthermore, Kirschner and Karpinski (2010) found that higher engagement in online learning environments could also result in decreased time for other crucial activities, including physical exercise and social interactions, which are essential for holistic well-being. Therefore, while engagement is often considered a positive metric in SRLA, there is a need to balance it with considerations for the learner’s overall health and well-being. Appropriately, after carefully reviewing the issue, we revised the initial DP2 to focus on persuasive motivation for SRLA to stimulate learning interests:

6.1 Revised DP2. Principle of persuasive motivation

SRLA should provide personalized features to stimulate interest and excitement to boost their motivation and engagement in learning.

This revised DP2 finds grounding in self-determination theory, which posits that meeting individual needs for autonomy and competence can boost intrinsic motivation (Deci and Ryan, 1985). For instance, the use of “gamification” elements has demonstrated the potential to increase both motivation and engagement in learning contexts (Deterding et al., 2011). Therefore, incorporating persuasive elements into SRLA could serve as a powerful strategy to enhance learner motivation and engagement, albeit with a mindful approach to avoid potential pitfalls such as excessive engagement.

Interestingly, regarding DP3, evaluation with the experts raised a critical discourse about the role of SRLA in improving and/or promoting learners’ SRL. Expert B from learning sciences questioned about the undesirable consequences of SRLA in establishing learners’ reliance on technology for sustaining SRL activities. It is recommended that SRLA should consider a scaffolding approach in developing learners’ SRL skills. Recent research has highlighted the importance of adaptive scaffolding in supporting SRL (Song and Glazewski, 2023). On the other hand, although Expert D from information sciences agreed with these side effects of SRLA, Expert D argued for the nature of living with technology in the age of artificial intelligence (AI). The augmentation was provided for the fact that many learning activities and processes are now cooperating with the use of technology, such as Google or Microsoft 365, which extends beyond the classroom to include daily life tasks that are integral to SRL. For instance, planning and time management – key components of SRL as outlined by Zimmerman (2000) – are often facilitated through digital tools like Google Calendar or Microsoft Outlook. These platforms not only assist in organizing academic deadlines but also help manage personal commitments, thereby enabling a holistic approach to self-regulation (Koehler and Mishra, 2005). Thus, the fusion of SRL activities with ubiquitous technology platforms highlights the increasingly blurred boundaries between educational and everyday digital tools, reinforcing the need for SRLA to be adaptable and integrated across multiple contexts. Future studies on the current topic from different philosophical perspectives are, therefore, recommended.

All the experts reached a consensus on DP4 and also referred to recent literature that highlighted the importance of emotional regulation (e.g. Järvenoja et al., 2019) and its support. They also shared an agreement on the principle of ethics and privacy that advocates SRLA to anonymous and transparent handling of personal data, with protection against unauthorized access, destruction, alteration or disclosure. It is important to consider questions, such as whether the collection and use of learner data are necessary, whether the data is being used in a fair and transparent manner, and whether the benefits of the data use outweigh any potential detriment. The ethical and privacy aspects of LA remain challenging (Ladjal et al., 2022), yet we hope that the findings from this study can contribute to the discussion to push forward the design and development of trustworthy LA systems.

Table 3 summarizes the final five design principles for SRLA as the output from our TPDM. The design principles offer prescriptive knowledge about the design of SRLA systems.

7. Discussion and implications

The importance of lifelong learning along with professional development has never been greater than it is today in a world that is fast-changing. SRL has been a key skill for successful lifelong learning (Molenaar, 2022). By providing learners with insights and recommendations based on their learning activities and progress, LA tools and technologies aim to facilitate and enhance SRL (Winne, 2017). The integration of LA into SRL can help lifelong learners make informed decisions about their learning activities, monitor their progress and evaluate their performance. Although research has provided insightful understandings of SRL processes and LA support for SRL, previous studies have been mostly conducted in the context of K-12 and higher education (Heikkinen et al., 2022). Up to now, far too little attention has been paid to SRLA targeting lifelong learners. This study thus attempts to fill a noteworthy gap by examining the challenges and needs of lifelong learners in SRL (RQ1, Table 1) to inform the design and development of SRLA (RQ2, Table 2).

Human-centered LA has grown rapidly in recent years and has the potential to support learners in developing SRL skills and improving their academic performance. In spite of these benefits, it has been questioned about how to consider the voice of learners more into the design and development of human-centered LA (Buckingham Shum et al., 2019). In this study, we present an approach for TPDM (Figure 1) that addresses the well-recognized concerns about the alignment between the needs of the learners and the design of LA tools and systems. Learners’ learning preferences are not always aligned with teachers’ pedagogical intentions or successful learning theories. Our TPDM methodology offers guidance for integrating and evaluating learner needs into the design of theory-grounded human-centered LA.

TPDM approach offers a comprehensive framework that combines viewpoints from various key players in the educational process. approach distinguishes itself from existing methodologies primarily through its comprehensive integration of perspectives, ranging from frontline users like lifelong learners and teachers to theoretical frameworks and expert evaluations. Traditional approaches, such as user-centered design (Norman, 1986) or participatory design (Schuler and Namioka, 1993) often focus more narrowly on user involvement or the inclusion of specific stakeholder groups, respectively, but may lack a structured integration of theoretical constructs. TDPM not only incorporates experiential insights from these stakeholders but also synthesizes foundational concepts from learning theories and design theories to inform its methodology. The inclusion of these nascent theories provides a structured backbone, allowing for data-driven adjustments and refinements in the design process (Baskerville et al., 2018; Heinrich and Schwabe, 2014). Finally, the TDPM approach is subject to evaluation by experts in the field, adding a layer of rigor and validity to the outcomes. This multilayered integration serves to produce more contextually relevant, empirically supported and theoretically robust educational interventions or tools, therefore, enhancing its applicability and impact.

In terms of research impact, TDPM contributes to the empirical rigor by synthesizing insights across a diverse set of stakeholders and theories. It addresses gaps in existing frameworks that may be overly reliant on a single theory or stakeholder perspective. Thus, TDPM fosters a more nuanced and comprehensive understanding of learning contexts, providing fertile ground for interdisciplinary research. For practical applications, TDPM’s multifaceted approach ensures that educational interventions or tools are not only theoretically robust but also contextually relevant and practically effective. By incorporating real-world insights from lifelong learners and teachers and subjecting the design to expert evaluations, TDPM enhances the usability and effectiveness of the final product. This broadens the research artefact’s appeal and adaptability, making it more likely to be adopted in diverse learning environments.

Our findings also suggest some practical implications for promoting SRL in professional development programs. Taking control of an individual’s learning process is an essential element of SRL, which leads to improved outcomes and greater satisfaction (Theobald, 2021). However, it is challenging for lifelong learners in professional development programs to manage their learning time while learning is often not their sole or even main occupation. Accordingly, it is crucial to provide lifelong learners with support for time management as well as to keep up their learning motivation. The results of this study can also shed light on developing SRLA systems to address lifelong learner needs.

The results of this study indicate that it can be helpful to learners to stay motivated by real-time monitoring and tracking of their learning activities and progress toward their personal goals and objectives. The use of motivational notifications and gamification can also aid in stimulating interest and excitement about learning. Previous research suggests that gamified approach embedded with SRL support can leverage students’ intrinsic motivation and lead to better learning (Qiao et al., 2022). In line with prior studies, our findings imply the importance of the self-regulated gamified approach for lifelong learning.

Furthermore, LA has been the subject of controversy for years (Pargman and McGrath, 2021). LA design and use are hampered by a lack of evidence and guidelines for practical ethics from end-users’ perceptions and assessments. Triangulating data collected from both lifelong learners and teachers, this study informs learners’ ethical concerns, relevant data and the conditions in which they are willing to share for SRLA. Learners in this study reported concerns regarding the security, privacy and transparency of their data, which is consistent with the literature. Nevertheless, as long as the LA process is secure and transparent without using sensitive personal information (e.g. income, health, etc.), lifelong learners expressed a high willingness to share their data for SRLA that could support their learning effectively.

It is important to note that this study is limited to analyzing data from a single group or context. This can limit the generalizability of the findings that may not be applicable to other settings or cultural contexts. Further research may conduct multisite studies or comparative studies in different contexts to validate the generalizability of our findings. Furthermore, the limitation of thematic analysis lies in its susceptibility to researcher bias and its challenges in managing large or context-rich data sets, which can compromise the depth and validity of the study (Braun and Clarke, 2006; Guest et al., 2011). Despite these limitations, the method’s adaptability and ease of use make it an effective approach for eliciting and understanding the varied viewpoints of stakeholders. Another limitation of this study is related to the small sample size of teachers and experts involved in this study. Although teacher voices and expert opinions were only applied to evaluate the learner needs, the small sample sizes may not provide a comprehensive validity and generalizability of the findings. Notwithstanding the relatively limited sample, we gained great insights from the triangulation of results from learners, teachers and experts.

8. Conclusion and final remarks

In the age of AI, the ability to self-regulate learning for lifelong learning is increasingly becoming important for individuals who want to remain relevant and competitive in the workplace. As an example, the recent release of OpenAI’s Chat-GPT serves as a reminder of the importance of self-regulated lifelong learning as it demonstrates the potential of AI to automate job tasks previously performed by humans. Individuals must constantly learn new skills and adapt to new knowledge due to the rapid pace of technological advancement and the constantly changing job market. Lifelong learners can stay up to date with the latest developments in their field by taking responsibility for their own learning through SRL. Embracing human-centered LA can contribute significantly to supporting lifelong learners’ SRL. The proposed approach for TPDM will allow advanced integration of learning sciences and design science research into the development of effective and trustworthy human-centered SRLA systems for lifelong learners.

Figures

Theory-driven participatory design with multistakeholders (TPDM) for integrating learner needs in establishing design principles for self-regulated learning analytics (SRLA)

Figure 1.

Theory-driven participatory design with multistakeholders (TPDM) for integrating learner needs in establishing design principles for self-regulated learning analytics (SRLA)

Lifelong learner participant demographics

Variable f %
Gender
Male 22 81.48
Female 5 18.52
Age
20–25 15 55.56
25–30 10 37.04
>30 2 7.41
Highest educational degree
Bachelor 25 92.59
Master 2 7.41
Employment status
Employee 14 51.85
Unemployed 12 44.44
Student 1 3.70

Source: By authors

Lifelong learner voices on self-regulated learning challenges and needs

ID Challenges and needs Desirable features Desirable subfeatures
Forethought phase: task analysis and self-motivation beliefs
C1 Challenges in setting goals and planning. Need for having a comprehensive and engaging visualization dashboard Understanding the task, setting the goal and planning Information and visualization of the: 1) general learning path and end goal; 2) task instruction and criteria
C2 Challenges in exploring their motivational factors to set motivating goals and plans. Need for analytics and personalized recommendations based on own motivational factors Personalized recommendation based on the motivational factors analytics 1) Learning interest; 2) initial level of capabilities; 3) outcome expectancies and goal orientation for future career
Performance phase: self-control and self-observation
C3 Challenges in time management and task prioritization. Need for information dashboard, prompts and reminders Task and time management 1) Information for time estimation, schedule and prioritization; 2) prompts to break down the goals; 3) reminder notifications for the target, task and deadline
C4 Challenges to easily record and track the learning activities and progress to motivate performance. Need for various data to be recorded, tracked and feasible Real-time monitoring Recording and tracking: 1) learning activities; 2) progress of learning goals; 3) enable to observe peer’s learning activities progress
C5 Challenges in getting the required social support. Need for recommendations on whom and how to access them anytime they need, and support for psychological safety Help-seeking 1) Recommendation and access for connecting to resourceful experts or peers; 2) function to support psychological safety
C6 Challenges in regulating emotion and motivation. Need for analytics features for gamification and personalized motivational notifications Stimulating interests and excitement during learning 1) Gamification elements; 2) personalized notification for motivational messages
C7 Challenges in constructing new knowledge. Need for prompts and reminders to rehearse information and stimulate self-learning Strategy to rehearse knowledge 1) Prompt and direct feedback to rehearse information; 2) stimulate self-learning; 3) reminders to review and revisit materials
Self-reflection phase: self-judgment and self-reaction
C8 Challenges in self-evaluating objectively and comprehensively to keep being adaptive. Need for analytics dashboards and improvement recommendations based on multiple standards Self-evaluation analytics based on various standards and recommendations for adaptive improvements 1) Certain targeted output criteria; 2) group performance without competition aspects; 3) own standards or track record; 4) review from the experts; 5) review from peers or other references; 6) recommendation on future improvement
Ethics and data preferences
C9 Concern about sharing their personal data. Need for information on data privacy, transparency and security Trustworthy information 1) Data privacy; 2) data transparency; 3) data security

Source: By authors

Design principles for self-regulated learning analytics (SRLA) for lifelong learners

# Design principle Specifications of design principle for SRLA
DP1 Principle of personalized monitoring Should generate personalized information and visualization for learners to self-monitor and rehearse their learning process toward the planned goals
DP2 Principle of persuasive motivation Should provide personalized features to stimulate interest and excitement to boost their motivation and engagement in learning
DP3 Principle of critical reflection Should deliver formative and summative feedback based on different standards and from different stakeholders for adaptive improvements
DP4 Principle of social support Should recommend and provide access to social and psychological support
DP5 Principle of ethics and privacy Should provide anonymity and transparency for personal data and protect data against accidental or unlawful destruction or accidental loss, alteration, unauthorized disclosure or access

Source: By authors

Appendix 1

Appendix 2

References

Alvarez, C.P., Martinez-Maldonado, R. and Buckingham Shum, S. (2020), “LA-DECK: a card-based learning analytics co-design tool”, Proceedings of the Tenth International Conference on Learning Analytics and Knowledge, Association for Computing Machinery, New York, NY, pp. 63-72, doi: 10.1145/3375462.3375476.

Anthonysamy, L., Koo, A.-C. and Hew, S.-H. (2020), “Self-regulated learning strategies and non-academic outcomes in higher education blended learning environments: a one decade review”, Education and Information Technologies, Vol. 25 No. 5, pp. 3677-3704, doi: 10.1007/s10639-020-10134-2.

Arnold, K. and Pistilli, M.D. (2012), “Course signals: Using learning analytics to increase student success”, Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, ACM, Vancouver, Canada, pp. 267-270.

Azevedo, R. and Gašević, D. (2019), “Analyzing multimodal multichannel data about self-regulated learning with advanced learning technologies: Issues and challenges”, Computers in Human Behavior, Vol. 96, pp. 207-210, doi: 10.1016/j.chb.2019.03.025.

Bandura, A. (2001), “Social cognitive theory of mass communication”, Media Psychology, Vol. 3 No. 3, pp. 265-299, doi: 10.1207/S1532785XMEP0303_03.

Baskerville, R., Baiyere, A., Gregor, S., Hevner, A. and Rossi, M. (2018), “Design science research contributions: Finding a balance between artifact and theory”, Journal of the Association for Information Systems, Vol. 19 No. 5.

Bjork, R.A., Dunlosky, J. and Kornell, N. (2013), “Self-Regulated learning: Beliefs, techniques, and illusions”, Annual Review of Psychology, Vol. 64 No. 1, pp. 417-444, doi: 10.1146/annurev-psych-113011-143823.

Bødker, S., Dindler, C., Iversen, O.S. and Smith, R.C. (2022), “What is participatory design?”, in Bødker, S., Dindler, C., Iversen, O.S. and Smith, R.C. (Eds), Participatory Design, Springer International Publishing, Cham, pp. 5-13, doi: 10.1007/978-3-031-02235-7_2.

Braun, V. and Clarke, V. (2006), “Using thematic analysis in psychology”, Qualitative Research in Psychology, Vol. 3 No. 2, pp. 77-101, doi: 10.1191/1478088706qp063oa.

Buckingham Shum, S. (2022), “What could learning analytics learn from Human-Computer interaction theory?”, in Bartimote, K., Howard, S. and Gasevic, D. (Eds), Theory Informing and Arising from Learning Analytics, Springer Nature, Cham.

Buckingham Shum, S., Ferguson, R. and Martinez-Maldonado, R. (2019), “Human-Centred learning analytics”, Journal of Learning Analytics, Vol. 6 No. 2, pp. 1-9, doi: 10.18608/jla.2019.62.1.

Chen, J. (2022), “The effectiveness of self-regulated learning (SRL) interventions on L2 learning achievement, strategy employment and self-efficacy: a meta-analytic study”, Frontiers in Psychology, Vol. 13, p. 1021101.

De Leoz, G. and Petter, S. (2018), “Considering the social impacts of artefacts in information systems design science research, edited by Peffers, K., Tuunanen, T. and Niehaves, B”, European Journal of Information Systems, Vol. 27 No. 2, pp. 154-170, doi: 10.1080/0960085X.2018.1445462.

Deci, E.L. and Ryan, R.M. (1985), “The general causality orientations scale: self-determination in personality”, Journal of Research in Personality, Vol. 19 No. 2, pp. 109-134.

Deterding, S., Dixon, D., Khaled, R. and Nacke, L. (2011), “From game design elements to gamefulness: defining ‘gamification”, Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, presented at the MindTrek ‘11: Academic MindTrek 2011, ACM, Tampere Finland, pp. 9-15, doi: 10.1145/2181037.2181040.

Dollinger, M., Liu, D., Arthars, N. and Lodge, J. (2019), “Working together in learning analytics towards the Co-Creation of value”, Journal of Learning Analytics, Vol. 6 No. 2, doi: 10.18608/jla.2019.62.2.

Domínguez, C., García-Izquierdo, F.J., Jaime, A., Pérez, B., Rubio, Á.L. and Zapata, M.A. (2021), “Using process mining to analyze time distribution of Self-Assessment and formative assessment exercises on an online learning tool”, IEEE Transactions on Learning Technologies, Vol. 14 No. 5, pp. 709-722, doi: 10.1109/TLT.2021.3119224.

Gašević, D., Dawson, S. and Siemens, G. (2015), “Let’s not forget: Learning analytics are about learning”, TechTrends, Vol. 59 No. 1, pp. 64-71, doi: 10.1007/s11528-014-0822-x.

Giacomin, J. (2014), “What is human centred design?”, The Design Journal, Routledge, Vol. 17 No. 4, pp. 606-623, doi: 10.2752/175630614X14056185480186.

Guest, G., MacQueen, K.M. and Namey, E.E. (2011), Applied Thematic Analysis, Sage Publications.

Hadwin, A.F., Järvelä, S. and Miller, M. (2018), “Self-regulation, co-regulation, and shared regulation in collaborative learning environments”, in Schunk, D.H. and Greene, J.A. (Eds), Handbook of Self-Regulation of Learning and Performance, 2nd ed., Routledge, New York, NY, pp, pp. 83-106.

Heikkinen, S., Saqr, M., Malmberg, J. and Tedre, M. (2022), “Supporting self-regulated learning with learning analytics interventions – a systematic literature review”, Education and Information Technologies, Vol. 28 No. 3, doi: 10.1007/s10639-022-11281-4.

Heinrich, P. and Schwabe, G. (2014), “Communicating nascent design theories on innovative information systems through multi-grounded design principles”, in Tremblay, M.C., VanderMeer, D., Rothenberger, M., Gupta, A. and Yoon, V. (Eds), Advancing the Impact of Design Science: Moving from Theory to Practice, Vol. 8463 Springer International Publishing, Cham, pp. 148-163, doi: 10.1007/978-3-319-06701-8_10.

Ifenthaler, D. and Widanapathirana, C. (2014), “Development and validation of a learning analytics framework: Two case studies using support vector machines”, Technology, Knowledge and Learning, Vol. 19 Nos 1/2, pp. 221-240.

Järvelä, S. and Bannert, M. (2021), “Temporal and adaptive processes of regulated learning - What can multimodal data tell?”, Learning and Instruction, Vol. 72, pp. 101268, doi: 10.1016/j.learninstruc.2019.101268.

Järvelä, S., Hadwin, A., Malmberg, J. and Miller, M. (2018), “Contemporary perspectives of regulated learning in collaboration”, in Fischer, F., Hmelo-Silver, C.E., Goldman, S.R and Reimann, P. (Eds), International Handbook of the Learning Sciences, 1st ed., Routledge, New York, NY, pp. 127-136, doi: 10.4324/9781315617572-13.

Järvelä, S., Kirschner, P.A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., Koivuniemi, M. and Järvenoja, H. (2015), “Enhancing socially shared regulation in collaborative learning groups: designing for CSCL regulation tools”, Educational Technology Research and Development, Vol. 63 No. 1, pp. 125-142, doi: 10.1007/s11423-014-9358-1.

Järvelä, S., Nguyen, A. and Molenaar, I. (2023), “Advancing SRL research with artificial intelligence”, Computers in Human Behavior, Vol. 147, p. 107847, doi: 10.1016/j.chb.2023.107847.

Järvenoja, H., Näykki, P. and Törmänen, T. (2019), “Emotional regulation in collaborative learning: when do higher education students activate group level regulation in the face of challenges?”, Studies in Higher Education, Vol. 44 No. 10, pp. 1747-1757, doi: 10.1080/03075079.2019.1665318.

Kirschner, P.A. and Karpinski, A.C. (2010), “Facebook® and academic performance”, Computers in Human Behavior, Vol. 26 No. 6, pp. 1237-1245.

Knight, S. and Buckingham Shum, S. (2017), “Theory and learning analytics”, in Lang, C., Siemens, G., Wise, A.F. and Gasevic, D. (Eds), The Handbook of Learning Analytics, 1st ed., Society for Learning Analytics Research (SoLAR), Alberta, Canada, pp. 17-22.

Koehler, M.J. and Mishra, P. (2005), “Teachers learning technology by design”, Journal of Computing in Teacher Education, Vol. 21 No. 3, pp. 94-102.

Koivuniemi, M., Panadero, E., Malmberg, J. and Järvelä, S. (2017), “Higher education students’ learning challenges and regulatory skills in different learning situations/desafíos de aprendizaje y habilidades de regulación en distintas situaciones de aprendizaje en estudiantes de educación superior”, Infancia y Aprendizaje, Vol. 40 No. 1, pp. 19-55, doi: 10.1080/02103702.2016.1272874.

Kwon, K., Liu, Y.H. and Johnson, L.P. (2014), “Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators”, Computers and Education, Vol. 78, pp. 185-200, doi: 10.1016/j.compedu.2014.06.004.

Ladjal, D., Joksimović, S., Rakotoarivelo, T. and Zhan, C. (2022), “Technological frameworks on ethical and trustworthy learning analytics”, British Journal of Educational Technology, Vol. 53 No. 4, pp. 733-736, doi: 10.1111/bjet.13236.

Lim, L., Bannert, M., van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., Rakovic, M., Molenaar, I., Moore, J. and Gašević, D. (2023), “Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning”, Computers in Human Behavior, Vol. 139, p. 107547, doi: 10.1016/j.chb.2022.107547.

Manganello, F., Pozzi, F., Passarelli, M., Persico, D. and Dagnino, F.M. (2021), “A dashboard to monitor Self-Regulated learning behaviours in online professional development”, International Journal of Distance Education Technologies (IJDET), Vol. 19 No. 1, pp. 18-34, doi: 10.4018/IJDET.2021010102.

Mangaroska, K., Martinez-Maldonado, R., Vesin, B. and Gašević, D. (2021), “Challenges and opportunities of multimodal data in human learning: the computer science students’ perspective”, Journal of Computer Assisted Learning, Vol. 37 No. 4, pp. 1030-1047, doi: 10.1111/jcal.12542.

Martinez-Maldonado, R., Gašević, D., Echeverria, V., Nieto, G.F., Swiecki, Z. and Buckingham Shum, S. (2021), “What do you mean by collaboration analytics? A conceptual model”, Journal of Learning Analytics, Vol. 8 No. 1, pp. 126-153.

Milligan, C. and Littlejohn, A. (2014), “Supporting professional learning in a massive open online course”, International Review of Research in Open and Distributed Learning, Vol. 15 No. 5, pp. 197-213.

Molenaar, I. (2022), “The concept of hybrid human-AI regulation: Exemplifying how to support young learners’ self-regulated learning”, Computers and Education: Artificial Intelligence, Vol. 3, p. 100070, doi: 10.1016/j.caeai.2022.100070.

Moshirpour, M., Paul, R. and Hemmanti, H. (2019), Designing a programming bootcamp for non-software engineers”, Proceedings of the Canadian Engineering Education Association (CEEA).

Nguyen, A., Gardner, L. and Sheridan, D. (2018), “A framework for applying learning analytics in serious games for people with intellectual disabilities”, British Journal of Educational Technology, Vol. 49 No. 4, pp. 673-689, doi: 10.1111/bjet.12625.

Nguyen, A., Gardner, L. and Sheridan, D. (2020), “Data analytics in higher education: an integrated view”, Journal of Information Systems Education, Vol. 31 No. 1, pp. 61-71.

Nguyen, A., Tuunanen, T., Gardner, L. and Sheridan, D. (2021), “Design principles for learning analytics information systems in higher education”, European Journal of Information Systems, Vol. 30 No. 5, pp. 541-568, doi: 10.1080/0960085X.2020.1816144.

Nguyen, A., Ngo, H.N., Hong, Y., Dang, B. and Nguyen, B.-P.T. (2023), “Ethical principles for artificial intelligence in education”, Education and Information Technologies, Vol. 28 No. 4, doi: 10.1007/s10639-022-11316-w.

Norman, D. (1986), “Cognitive engineering”, User Centered System Design: New Perspectives on Human-Computer Interaction, in Norman, D. and Draper, S. (Eds), Lawrence Erlbaum Associates, Hillsdale, NJ.

Nunn, S., Avella, J.T., Kanai, T. and Kebritchi, M. (2016), “Learning analytics methods, benefits, and challenges in higher education: a systematic literature review”, Online Learning, Vol. 20 No. 2, doi: 10.24059/olj.v20i2.790.

Pargman, T.C. and McGrath, C. (2021), “Mapping the ethics of learning analytics in higher education: a systematic literature review of empirical research”, Journal of Learning Analytics, Vol. 8 No. 2, pp. 123-139, doi: 10.18608/jla.2021.1.

Peffers, K., Tuunanen, T., Rothenberger, M.A. and Chatterjee, S. (2007), “A design science research methodology for information systems research”, Journal of Management Information Systems, Routledge, Vol. 24 No. 3, pp. 45-77, doi: 10.2753/MIS0742-1222240302.

Persico, D., Milligan, C. and Littlejohn, A. (2015), “The interplay between Self-Regulated professional learning and teachers’ Work-Practice”, Procedia – Social and Behavioral Sciences, Vol. 191, pp. 2481-2486, doi: 10.1016/j.sbspro.2015.04.590.

Persico, D., Passarelli, M., Manganello, F., Pozzi, F., Dagnino, F.M., Ceregini, A. and Caruso, G. (2020), “Automatic feedback, self-regulated learning and social comparison: a case study”, Qwerty-Open and Interdisciplinary Journal of Technology, Culture and Education, Vol. 15 No. 2, pp. 27-44.

Prieto-Alvarez, C.G., Martinez-Maldonado, R. and Buckingham Shum, S. (2018), “Mapping learner-data journeys: evolution of a visual co-design tool”, Proceedings of the 30th Australian Conference on Computer-Human Interaction, Association for Computing Machinery, New York, NY, pp. 205-214, doi: 10.1145/3292147.3292168.

Qiao, S., Chu, S.K.W., Shen, X. and Yeung, S.S. (2022), “The impact of an online gamified approach embedded with self-regulated learning support on students’ reading performance and intrinsic motivation: a randomized controlled trial”, Journal of Computer Assisted Learning, Vol. 38 No. 5, pp. 1379-1393, doi: 10.1111/jcal.12684.

Sarmiento, J.P. and Wise, A.F. (2022), “Participatory and Co-Design of learning analytics: an initial review of the literature”, LAK22: 12th International Learning Analytics and Knowledge Conference, Association for Computing Machinery, New York, NY, pp. 535-541, doi: 10.1145/3506860.3506910.

Schaufeli, W.B., Salanova, M., González-Romá, V. and Bakker, A.B. (2002), “The measurement of engagement and burnout: a two sample confirmatory factor analytic approach”, Journal of Happiness Studies, Vol. 3 No. 1, pp. 71-92.

Schuler, D. and Namioka, A. (1993), Participatory Design: Principles and Practices, CRC Press, London.

Schumacher, C. and Ifenthaler, D. (2018), “Features students really expect from learning analytics”, Computers in Human Behavior, Vol. 78, pp. 397-407, doi: 10.1016/j.chb.2017.06.030.

Shum, S.B. and Ferguson, R. (2012), “Social learning analytics”, Journal of Educational Technology and Society, Vol. 15 No. 3, pp. 3-26.

Slade, S. and Prinsloo, P. (2013), “Learning analytics ethical issues and dilemmas”, American Behavioral Scientist, Vol. 57 No. 10, pp. 1510-1529.

Song, D. and Glazewski, K. (2023), “Scaffolding self-regulated learning in student-generated questioning using mobile phones”, Education and Information Technologies, Vol. 28 No. 8, doi: 10.1007/s10639-023-11627-6.

Taranto, D. and Buchanan, M.T. (2020), “Sustaining lifelong learning: a self-regulated learning (SRL) approach”, Discourse and Communication for Sustainable Education, Vol. 11 No. 1, pp. 5-15, doi: 10.2478/dcse-2020-0002.

Theobald, M. (2021), “Self-regulated learning training programs enhance university students’ academic performance, self-regulated learning strategies, and motivation: a meta-analysis”, Contemporary Educational Psychology, Vol. 66, p. 101976, doi: 10.1016/j.cedpsych.2021.101976.

Van Schoors, R., Elen, J., Raes, A. and Depaepe, F. (2021), “An overview of 25 years of research on digital personalised learning in primary and secondary education: a systematic review of conceptual and methodological trends”, British Journal of Educational Technology, Vol. 52 No. 5, pp. 1798-1822, doi: 10.1111/bjet.13148.

Venable, J., Pries-Heje, J. and Baskerville, R. (2012), “A comprehensive framework for evaluation in design science research”, in Peffers, K., Rothenberger, M. and Kuechler, B. (Eds), Design Science Research in Information Systems. Advances in Theory and Practice, Springer, Berlin, Heidelberg, pp. 423-438, doi: 10.1007/978-3-642-29863-9_31.

Vygotsky, L.S. and Cole, M. (1978), Mind in Society: Development of Higher Psychological Processes, Harvard University Press, Boston, MI.

Wang, F. and Hannafin, M.J. (2005), “Design-based research and technology-enhanced learning environments”, Educational Technology Research and Development, Vol. 53 No. 4, pp. 5-23, doi: 10.1007/BF02504682.

Winne, P.H. (2017), “Learning analytics for Self-Regulated learning”, Handbook of Learning Analytics, Society for Learning Analytics Research (SoLAR), pp. 241-249, doi: 10.18608/hla17.021.

Zimmerman, B.J. (2000), “Chapter 2 - attaining self-regulation: a social cognitive perspective”, in Boekaerts, M., Pintrich, P.R. and Zeidner, M. (Eds), Handbook of Self-Regulation, Academic Press, San Diego, pp. 13-39, doi: 10.1016/B978-012109890-2/50031-7.

Zimmerman, B.J. (2002), “Becoming a self-regulated learner: an overview”, Theory into Practice, Vol. 41 No. 2, pp. 64-70, doi: 10.1207/s15430421tip4102_2.

Zimmerman, B.J. and Moylan, A.R. (2009), “Self-Regulation: where metacognition and motivation intersect”, Handbook of Metacognition in Education, Taylor & Francis, New York, pp. 299-315.

Acknowledgements

This research has been funded by the Finnish Academy grant number 350249.

Statements on open data: The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Ethics: This research complied with the university research integrity and ethics guidelines by the Ethics Committee of Human Sciences of the University of Oulu.

Conflict of interest: The authors have no conflicts of interest regarding this study to declare.

Corresponding author

Andy Nguyen is the corresponding author and can be contacted at: andy.nguyen@oulu.fi

About the authors

Andy Nguyen is a Research Council of Finland (previously known as the Academy of Finland) postdoctoral researcher working at the Learning and Educational Technology Research Lab (LET), University of Oulu. He holds the title of Docent (Adjunct Professor) in Applied Artificial Intelligence at the Faculty of Information Technology and Electrical Engineering (ITEE) within the same university. Dr Nguyen obtained his PhD in Information Systems from the University of Auckland, New Zealand. His research interests lie in bridging learning sciences, artificial intelligence (AI), data analytics, information systems and technology and related educational policy and management.

Joni Lämsä is a postdoctoral researcher working at the Learning and Educational Technology Research Lab (LET), University of Oulu. His research interests include self-regulated learning and socially shared regulation of learning, and how those can be detected and supported by advanced AI-enhanced learning technologies.

Adinda Dwiarie is the head of learning experience design and research at Binar Academy, specializing in ed-tech and a master’s student at the Learning and Educational Technology Research Lab (LET), University of Oulu. Her research interests are related to self-regulated learning and its implementation on learning design and technology.

Sanna Järvelä is a professor in Learning Sciences and the head of the Learning and Educational Technology Research Lab (LET) at the University of Oulu. Her research interests deal with self-regulated learning, computer-supported collaborative learning and AI in education.

Related articles