Examining AI competence, chatbot use and perceived autonomy as drivers of students' engagement in informal digital learning

Abstract

Purpose

Digital technology's integration into education has transformed learning frameworks, necessitating the exploration of factors influencing students’ engagement in digital informal settings. This study, grounded in self-determination theory (SDT), proposes a model comprising artificial intelligence (AI) competence, chatbot usage, perceived autonomy (PA), digital informal learning (DIL) and students’ engagement.

Design/methodology/approach

The study collected survey data from 409 participants at Saudi Arabian universities, ultimately using 387 valid responses for analysis. This dataset was subjected to a thorough examination to confirm the validity of our proposed model. To decipher the complex interactions within our model, we utilized partial least squares structural equation modeling (PLS-SEM). The study adopted a disjoint two-stage method to formulate a reflective-formative higher-order construct (HOC).

Findings

The study's findings showed that cognitive learning (CL), metacognitive learning (MCL) and social and motivational learning (SML) are the essential components of DIL. Significantly, the study determined that AI competence, chatbot usage, PA and DIL markedly affect students’ engagement. Moreover, the R² value of 0.592 for student engagement indicates the model's robustness in explaining 59.2% of the variance, highlighting its effectiveness in identifying key drivers of student engagement in DIL contexts.

Originality/value

This research enhances understanding by detailing the intricate relationships among AI competence, chatbot usage, and students’ engagement in informal digital learning. It extends SDT to emphasize intrinsic motivations and AI capabilities, introducing reflective-formative HOCs for comprehending educational intricacies. It provides practical strategies for enhancing AI abilities and chatbot use in education, promoting personalized, engaging and autonomous digital learning spaces, thereby advancing educational theory and practice.

Citation

Hidayat-ur-Rehman, I. (2024), "Examining AI competence, chatbot use and perceived autonomy as drivers of students' engagement in informal digital learning", Journal of Research in Innovative Teaching & Learning, Vol. 17 No. 2, pp. 196-212. https://doi.org/10.1108/JRIT-05-2024-0136

1. Introduction

The advent of artificial intelligence (AI) in education has introduced advanced tools like chatbots, reshaping the educational domain by providing personalized and interactive learning experiences (Pillai et al., 2023; Yakin et al., 2023). Chatbots mimic human interactions, offer immediate feedback and create engaging learning environments (Gleason, 2022), enhancing information accessibility and introducing innovative learning methods.

The increasing incorporation of AI in education emphasizes the need to develop AI competence among students. These skills enhance their engagement in digital learning, enabling tailored learning and self-directed study (Hidayat-Ur-Rehman, 2024). AI competence fosters critical thinking, problem-solving and adaptability, preparing students for the future workforce (Mahmudi et al., 2023).

In Saudi Arabia, technological advancements significantly impact societal operations (Aljaloud et al., 2019). Exploring digital competence's influence on students’ engagement is crucial for effective educational methodologies, especially in university science, technology, engineering and math (STEM) fields and e-learning (Almalki and Pleasants, 2021; Hasan et al., 2023). The COVID-19 pandemic has exacerbated challenges, altering teacher–student interactions and perceptions of digital capabilities, necessitating enhanced approaches to maintain engagement in digital education (Aladsani, 2022).

Contemporary studies often explore artificial intelligence, digital competence, the use of chatbots and digital informal learning (DIL) in isolation, rarely addressing the combined effect of these factors on students’ engagement. Heidari et al. (2021) underscore the significance of DIL in boosting academic engagement, particularly amidst the challenges posed by the COVID-19 pandemic. Moral-Sánchez et al. (2023) advocate for creating and integrating chatbots in educational settings, noting their positive impact on students' satisfaction levels and digital competence. Antony and Ramnath (2023) explore AI chatbots' effect on students’ engagement and support, revealing their potential to facilitate effective communication and boost engagement, underscoring the need for further research to optimize and integrate AI chatbots in educational contexts.

This study presents a conceptual model exploring AI competence, chatbot usage, perceived autonomy (PA) and their impact on students’ engagement in informal digital learning. Grounded in self-determination theory (SDT) (Deci and Ryan, 1980), it highlights the role of autonomy, competence and relatedness in digital platforms (Chiu, 2022). The framework shows how AI competence and chatbots enhance engagement by fulfilling these psychological needs.

This research aims to enrich the field of AI in education by studying how AI technologies, especially chatbots, affect students’ engagement through the SDT lens. By examining the dynamics between AI competence, chatbot usage and PA, this study seeks to provide insights into optimizing educational strategies for deeper students’ involvement and success in technologically advanced learning settings.

This study examines AI competence's influence on students’ engagement, focusing on the intermediary roles of chatbot usage and DIL. It aims to provide educators with insights to develop engaging digital learning environments. By applying SDT, the research demonstrates how internal motivations and external AI skills, facilitated by chatbots, impact students’ engagement, highlighting the relevance of SDT in understanding autonomy and technology in education.

This study examines AI competence's influence on students’ engagement, focusing on chatbot usage and DIL. It provides insights for educators to develop engaging digital learning environments. Using SDT, the research demonstrates how internal motivations and external AI skills, facilitated by chatbots, impact students’ engagement, highlighting SDT's relevance in understanding autonomy and technology in education.

2. Review of related literature

2.1 Theoretical foundations

SDT, proposed by Deci and Ryan (1980), serves as a comprehensive psychological framework for examining human motivation. It identifies three primary psychological needs essential for fostering motivation: autonomy (control over one’s actions), competence (ability to effectively face challenges) and relatedness (desire to form connections with others). Meeting these needs enhances intrinsic motivation, boosting engagement, well-being and personal growth. SDT is widely applied in education, work and interpersonal relationships, emphasizing supportive social environments to nurture these needs, thereby affecting motivation and engagement levels.

In digital learning contexts, SDT's emphasis on competence, autonomy and relatedness is crucial. The digital era necessitates SDT to understand and improve learner engagement by addressing these psychological needs. The theory's applicability extends to AI and digital tools competence, making it a fitting model for research in these areas. SDT explains intrinsic and extrinsic motivation in digital learning environments, as seen in online students’ engagement during the COVID-19 crisis (Chiu, 2022). Digital approaches fulfilling SDT's psychological needs have effectively boosted online learning engagement (Chiu et al., 2022). Competence in AI can enhance intrinsic motivation and engagement, emphasizing SDT's relevance in digital learning dynamics.

The study selects SDT for its insights into students’ engagement in digital learning, focusing on AI competence, chatbot usage and PA. SDT's emphasis on competence, autonomy and relatedness is essential in digital environments. Its effectiveness in online and blended learning, highlighted by Chiu (2022), confirms its suitability for exploring AI competence and students’ engagement.

This study adopts SDT to investigate how AI competence and chatbot usage influence students’ engagement in DIL. It hypothesizes that chatbot integration enhances autonomy and motivation, AI competence boosts intrinsic motivation and engagement and chatbot usage fosters connectedness, positively impacting engagement, making SDT the ideal theoretical foundation.

2.2 Framework design and hypotheses formulation

Drawing on SDT, this research proposes a model examining the effects of AI competence on students’ engagement via AI competence, chatbot use, perceived autonomy, digital informal learning and student engagement. This model, rooted in SDT's principles, aims to dissect AI competence's impact in the evolving educational landscape. The interconnected constructs suggest AI competence influences chatbot use, which affects PA and participation in DIL, ultimately impacting student engagement levels.

Each component is chosen for its relevance to digital learning and potential to enhance engagement. AI competence is crucial for effective technology use, influencing chatbot utilization and informal online education. Chatbot's recognition and multi-functionality make it valuable, provided students harness its capabilities. Perceived autonomy, a core SDT element, highlights autonomy's role in fostering engagement and motivation. DIL represents self-directed knowledge acquisition through digital means, driven by personal interests. Students’ engagement, the research's central theme, is linked to academic achievement and is significantly influenced by digital technology.

Investigating these variables in the Saudi Arabian context addresses challenges and opportunities from the swift incorporation of AI technologies and the shift to online education, underscored by COVID-19's impact. The following sections will elaborate on the constructs, examine empirical evidence regarding their interactions and discuss hypothesis development.

2.2.1 Students’ engagement

Students’ engagement is defined by Riden et al. (2021) as the active participation and accomplishments of students within the educational environment. Skinner and Pitzer (2012) further clarify academic engagement as the active readiness to engage in classroom and scholarly activities. Redmond et al. (2018) support the notion that this facet of engagement serves as a dependable measure of the quality of educational experiences students receive and their subsequent academic achievement.

Information technology and AI tools enhance cognitive engagement in organized settings (Downs et al., 2015). Prior research connects students’ engagement with performance and involvement (Watson and Berry, 2022). This study examines AI-driven chatbot interactions and AI literacy in informal digital learning environments. Research shows AI proficiency and chatbot use increase engagement (Hidayat-Ur-Rehman, 2024; Pillai et al., 2023). In Saudi Arabia, understanding AI competence's impact on engagement is crucial for educational reforms under Vision 2030 (Alotaibi and Alshehri, 2023). Exploring these factors can optimize educational strategies in digitally advanced environments (Alghamdi, 2022).

According to Pillai et al. (2023), AI-driven chatbots in educational settings improve, monitor, assess and organize students' learning experiences in digital classrooms. These systems provide support, guidance and responses to learners' inquiries (Garcia Brustenga et al., 2018). Chatbots also enrich education by presenting materials like videos, images and quizzes (Al-Ghadhban and Al-Twairesh, 2020), increasing engagement and interaction. Additionally, chatbots offer timely, secure feedback on academic performance, enhancing the learning experience (Pillai et al., 2023). In Saudi Arabia, student engagement is shaped by cultural, technological and educational transformations, requiring further inquiry into the impact of digital skills (Hidayat-Ur-Rehman, 2024).

The synthesis of research underscores the need to explore students’ engagement in DIL through AI competence. As AI-driven tools like chatbots permeate education, understanding their impact is crucial, especially in Saudi Arabia's reforming educational landscape. Investigating AI competence and chatbot interactions can guide strategies to enhance engagement and outcomes.

2.2.2 Students’ artificial intelligence competence

AI competence is the ability to solve problems using an understanding of AI (Ahn and Oh, 2024; Yoo et al., 2022), including performing tasks successfully (Falloon, 2020). It requires the practical application of AI principles. AI literacy, the foundation for AI competence, involves critical evaluation, effective communication with AI and understanding AI's implications (Chiu et al., 2024).

AI competency is defined as the ability to solve problems using AI, involving knowledge representation, data learning, machine learning, deep learning and AI ethics (Ahn and Oh, 2024). This framework stresses the importance of both technical skills and ethical understanding. Kitcharoen et al. (2024) describe AI competency as essential for real-world problem-solving, integrating cognitive, behavioral and affective elements. It includes understanding AI concepts, practical skills application and societal impact awareness. AI competence in students refers to their ability to effectively use AI tools in DIL settings, influenced by their perceived proficiency (Hatlevik, 2017).

As AI integrates into various sectors, DIL becomes vital for digital literacy. Incorporating it into educational curricula equips students with essential skills for the digital world (Kim and Kwon, 2023). Researchers have explored its impact across educational contexts to understand its influence on learning and adaptation in digital environments (Ahn and Oh, 2024; Kitcharoen et al., 2024; Sanusi et al., 2022).

AI integration in education, particularly through personalized learning environments, enhances students’ engagement by tailoring content to individual learning styles and abilities (Mohd and Mohd Abrar, 2024). AI technologies, such as adaptive learning systems, significantly boost engagement by offering customized educational experiences. AI proficiency influences the adoption of ChatGPT, promoting deeper understanding and retention and positively impacting academic achievement (Hidayat-ur-Rehman and Ibrahim, 2023).

AI-powered chatbots like ChatGPT enhance education by providing instant feedback, clarifying doubts and supporting knowledge acquisition outside traditional classrooms. Yakin et al. (2023) highlight high students’ engagement with ChatGPT, emphasizing strategic integration for critical thinking. Moral-Sánchez et al. (2023) demonstrate that chatbots improve digital competence and student satisfaction, indicating AI competence enhances the effectiveness of chatbot technologies in education.

AI competence significantly influences how students interact with and utilize chatbots for learning. With a solid understanding of AI principles, students navigate chatbot interfaces effectively, interpret responses accurately and ask better questions to facilitate their learning (Chiu et al., 2023). This competence allows them to critically assess chatbot information, discerning its relevance and accuracy. AI-savvy students can customize their interactions with educational chatbots, tailoring the learning experience to their needs and preferences, leading to improved digital competence and satisfaction (Moral-Sánchez et al., 2023).

AI competence transforms chatbots into dynamic educational partners, enhancing interest and motivation (Han, 2020). Incorporating AI education into curricula prepares students to navigate and shape the future digital landscape ethically and effectively. Mastery of AI enables students to effectively use chatbots for educational tasks, enhancing their learning experience and engagement with academic content in DIL environments.

2.2.3 Chatbots usage by students

Chatbots, leveraging AI Natural Language Processing (NLP) models, have significantly transformed various sectors, including education. Techniques like neural networks enable chatbots to mimic human language nuances (Wu et al., 2023). The rapid progression of chatbot technology, exemplified by OpenAI's GPT series, highlights their advanced capabilities (Brown et al., 2020).

In education, chatbots are celebrated for reshaping teaching and learning, acting as digital mentors that assist with inquiries, essays and programming (Gleason, 2022). Studies explore their role, impact on engagement and satisfaction and perceptions by pre-service educators and students (Yang and Chen, 2023).

Chatbot use by students enhances engagement and autonomy by providing instant, tailored feedback, creating a dynamic learning environment. This fosters self-guided learning, empowering students to control their educational journey. However, challenges include potential over-reliance on chatbots, risks of undermining educational standards, plagiarism, misuse and biased or incorrect responses (Cano et al., 2023; DW, 2023). To address these concerns, educational institutions are encouraged to implement preventative strategies and regulate chatbot use to maintain academic integrity and ethical technology usage. Balancing the benefits and challenges is crucial to effectively integrate chatbots into education while preserving the quality and standards of learning experiences.

Yakin et al. (2023) highlight chatbots as personalized digital tutors, enhancing AI literacy, engagement and comprehension through tailored, interactive learning experiences. Hidayat-ur-Rehman and Ibrahim (2023) found that chatbots increase PA, boosting participation and engagement. Pillai et al. (2023) noted students' adoption of ChatGPT for its utility, interactivity and reliability, underscoring chatbots' role in improving students’ engagement and the overall educational experience.

Despite challenges, chatbots significantly enhance students’ engagement and autonomy, offering flexible and supportive resources that complement traditional education. Their potential to improve educational outcomes is substantial, given careful management and alignment with educational principles and ethical standards.

2.2.4 Perceived autonomy (PA)

PA, a core aspect of SDT, concerns an individual's perception of freedom and control, particularly in educational settings, indicating voluntary participation in learning activities. It aligns with our natural inclination towards beneficial actions and well-being. SDT posits that satisfying autonomy, competence and relatedness spurs intrinsic motivation (Deci and Ryan, 1980). Connell (1990) explains that autonomy involves perceiving choices in actions that align with personal goals and values.

Empirical evidence confirms the profound influence of PA on student satisfaction and persistence (Mobarhan and Abdul Rahman, 2015; Hidayat-ur-Rehman, 2024). Students who feel autonomous are more motivated, take initiative and engage actively, choosing when and how to access learning materials and collaborate, promoting ownership and involvement in digital learning settings.

Wood (2016) demonstrates that meeting the psychological needs of competence, autonomy and relatedness in classrooms indicates students' readiness to participate in educational activities. Yang et al. (2022) find that teaching styles supporting autonomy promote engagement and reduce burnout among primary school students. Tao et al. (2022) use meta-analysis to reveal a significant link between perceived teacher support and academic success, emphasizing emotional support over autonomy in higher-secondary education. These studies highlight the importance of autonomy in student engagement and well-being.

Collectively, the mentioned studies emphasize autonomy's crucial role, recognized by students and supported by educators, in enhancing engagement and performance. Fostering autonomy promotes motivation, involvement and academic success, suggesting that PA positively influences students’ engagement in DIL contexts.

2.2.5 Digital informal learning

DIL encompasses students' informal learning behaviors through digital technology, playing a vital role in their educational journey (Heidari et al., 2021). It transitions from structured educational frameworks to dynamic, self-guided learning enabled by digital technologies (Mehrvarz et al., 2021). Unlike formal learning's organized nature and certification outcomes (Czerkawski, 2016; Meyers et al., 2013), DIL is unstructured and interest-driven, integrating cognitive, metacognitive and social-motivational elements (Mehrvarz et al., 2021). This approach fosters learning beyond traditional settings, utilizing digital tools for interactive, self-directed learning (Ungerer, 2016).

He and Li (2019) outline that DIL includes cognitive learning (CL), metacognitive learning (MCL) and social and motivational learning (SML). CL involves engaging with digital media for learning (Mayer, 1998). MCL encompasses planning, organizing, storing knowledge and monitoring understanding (Mayer, 1998), influencing strategy choice for information retention (Taheri et al., 2020). SML highlights social interaction and motivation in digital settings for collaborative knowledge construction. Effective strategies and technologies enhance DIL opportunities.

The effects of DIL on academic achievement vary: Lee and Dressman (2018) report positive outcomes on learning and engagement, while Junco (2012) and Kirschner and Karpinski (2010) highlight negative consequences, especially linked to social media usage.

Digital literacy, information literacy, media literacy, information and communication technology (ICT) literacy, Internet literacy and e-skills are often used interchangeably with digital competence, indicating their interconnectedness (Ferrari, 2013). This shift to competence underscores its educational importance (Janssen et al., 2013). Digital competence includes using technology for accessing, processing, creating and sharing information (Hatlevik and Christophersen, 2013).

DIL enhances students’ engagement by promoting autonomy, interactive and collaborative learning and personalized learning paths. Evidence shows DIL increases motivation and domain-specific knowledge retention (Sommerauer and Müller, 2014). Heidari et al. (2021) found that DIL mediates the relationship between digital competence and academic engagement. He and Li (2019) confirmed digital competence's significant role in DIL among diverse university students.

DIL represents an evolution in learning, driven by digital technologies and learner-centered approaches. Its effectiveness relies on digital competence, enabling effective navigation in digital environments. While DIL's impact on academic performance varies, it clearly enhances students’ engagement through personalized and interactive learning experiences. This research suggests that students' AI competence significantly drives DIL, increasing engagement by shifting from traditional models to technology-enabled, self-initiated learning shaped by personal interests. DIL covers cognitive, social and motivational dimensions, equipping students with digital competencies for engaging with digital media. The synergy between AI competence and DIL is crucial for enhancing students’ engagement in informal digital learning settings.

Following a thorough review of the literature presented in Section 2.2, the research developed the following hypotheses:

H1.

AI competence has significant impacts on students’ engagement.

H2.

AI competence has significant impacts on digital information learning.

H3.

AI competence has significant impacts on chatbots use.

H4.

DIL has significant impacts on students’ engagement.

H5.

Chatbots use has significant impacts on students’ engagement.

H6.

Chatbots use has significant impacts on PA.

H7.

PA has significant impacts on students’ engagement.

The model proposed by the study is illustrated in Figure 1 below.

3. Research methodology

3.1 Development of the research instrument

This research employed a quantitative methodology to collect reliable, generalizable and valid data. The primary tool was a 28-item survey, using scales from prior studies to evaluate key constructs. The study focused on three aspects of DIL: CL, MCL and SML, each represented by four items from Mehrvarz et al. (2021). A single item, DIL_Global, assessed the overall concept of DIL. PA was measured with five items from Mobarhan and Abdul Rahman (2015) and students’ engagement with three items from Aljaloud et al. (2019). Newly developed items for AI competence and chatbots use, with four items each, were validated through exploratory factor analysis (EFA). Expert feedback was incorporated, and the final survey used a 1–5 Likert scale.

3.2 Sampling and data gathering

To test the proposed model, the researcher collected data from various universities in Saudi Arabia through surveys. The country hosts 29 state-run and 15 independently managed universities, with a combined enrollment of 86,000 students (MOE, SA, 2023). The study targeted university students from leading public universities due to their extensive geographic spread and representativeness, enhancing the study's external validity. Public universities, which account for 95% of the student population at Colliers Education Advisory and Valuation Services (2022), were prioritized for a comprehensive assessment. A stratified sampling method selected five public universities from different regions: the University of Tabuk, King Abdul Aziz University, King Saud University, King Khalid University and King Fahd University of Petroleum and Minerals. It was followed by convenience sampling to recruit individual student participants. Data collection occurred between October and November 2023 via an online survey. The researcher gathered 409 responses, exceeding Thompson's (2012) recommended sample size of 384, and after discarding 22 incomplete responses, 387 valid responses remained. The methodology emphasized transparency, bias reduction and participant confidentiality.

4. Analysis of data

In this research, the data were analyzed using the partial least squares structural equation modeling (PLS-SEM) method. The study focused on a particular construct, referred to as DIL, which was composed of three formative dimensions. Following the guidance of Hair et al. (2023), a disjoint dual-phase approach was adopted to effectively handle the complexity of DIL, involving the creation of “reflective-formative higher-order constructs”. The first phase of the analysis was dedicated to a thorough examination of both the lower-order constructs (LOCs) and the higher-order constructs (HOCs) as part of the measurement framework. After such a detailed evaluation, the researcher moved on to evaluate the structural model to gain a deeper understanding of the relationships being studied. Subsequent sections will explore the details of the data analysis procedure.

4.1 Assessment of the measurement model

In this study's framework, DIL is characterized by three formative dimensions: CL, MCL and SML (Section 2.2.5). DIL is conceptualized as a HOC, with CL, MCL and SML as LOCs. Consequently, DIL is identified as a reflective-formative HOC.

4.1.1 Formulating reflective-formative HOC

This study employs the sequential dual-phase approach by Hair et al. (2023) to develop the HOC DIL, characterized by a reflective-formative structure. Initially, Phase-1 uses the PLS-SEM algorithm to validate the LOCs independently, confirming their reliability and convergent validity. Phase-2 replaces the LOCs with the HOC using construct scores. Redundancy analysis establishes the convergent validity of DIL, with a path coefficient (β) of 0.731, exceeding the 0.7 benchmark. Variance inflation factor (VIF) values for CL, MCL and SML are 1.398, 1.306 and 1.161, respectively, indicating minimal collinearity concerns. The results confirmed that DIL is recognized as a construct with a reflective-formative higher-order structure.

4.1.2 Assessing reliability and validity of constructs based on Stage-1 analysis (excluding the formative construct of DIL)

Table 1 displays the results of Phase-1 reliability and validity tests. Cronbach's alpha and composite reliability scores exceed 0.7, confirming internal consistency. Outer loadings for all indicators are above 0.7, ensuring reliability. The AVE scores, used as a metric for convergent validity, are all above 0.5, thereby validating the scales.

To evaluate discriminant validity, the heterotrait-monotrait (HTMT) ratio method was used. The results of this HTMT analysis, shown in Table 2, reveal that the HTMT ratios for each pair of variables remained below 0.9. Henseler et al. (2015) findings support the presence of discriminant validity, in line with these results.

4.2 Structural model analysis

This research employed the bootstrapping method, comprising 10,000 subsamples and chose the standard settings. The results, outlined in Table 3, demonstrate that all the proposed connections hold statistical significance, with p-values of 0.001 or less. Additionally, the analysis of reflective-formative HOC aligns with the evaluation criteria for structural models suggested by Hair et al. (2023). The outcomes of the bootstrapping tests are visually represented in Figure 2.

The empirical evidence confirms a significant impact of AI competence on students' engagement (β = 0.203, t = 5.503, p: 0.000). This finding supports the hypothesis H1, which proposes that students' competence in AI significantly improves their engagement level. The analysis of the second hypothesis, H2, which suggested direct and significant effects of AI competence on DIL, is supported by empirical results (β = 0.446, t = 9.915, p: 0.000). This validates H2, emphasizing the vital role of AI competence in promoting DIL among students. Specifically, in the context of using chatbots for informal learning activities, this study underscores the importance of students’ AI competence in enhancing their informal learning experiences.

Hypothesis 3, which explored the effects of AI competence on students' use of chatbots, receives empirical support from the findings of bootstrapping (β = 0.303, t = 7.021, p: 0.000). This research highlights the importance of AI competence for effective and beneficial use of chatbots in learning. The empirical findings confirm the fourth hypothesis (H4), which posited significant effects of DIL on students’ engagement (β = 0.331, t = 8.142, p: 0.000). This result underscores the relationship between DIL and students’ engagement, indicating that leveraging digital resources for informal learning has a positive impact on student engagement.

Hypothesis 5, which examined the effects of chatbots use on students’ engagement, is also corroborated by empirical results (β = 0.305, t = 7.027, p: 0.000), indicating that using chatbots leads to enhanced students’ engagement. Hypothesis 6 suggested a notable correlation between chatbots use and PA, and it is validated by empirical results (β = 0.244, t = 4.870, p: 0.000). This study suggests that using chatbots in DIL enhances students' autonomy in their learning processes. The seventh hypothesis, which examined the relationship between PA and students' engagement, is reinforced by the outcomes (β = 0.238, t = 6.679, p: 0.000). This finding specifies that higher perceptions of autonomy in DIL environments enhance students’ engagement.

The R² score for the exogenous construct, students' engagement, is 0.592. It suggests that the model explains 59.2% of the variability in students’ engagement, considering factors such as AI competence and chatbots usage in a DIL setting. Following the criteria outlined by Hair et al. (2017), this level of explanatory ability falls into the moderate category.

Overall, this study confirms each hypothesis with strong empirical evidence and validates the model. Additionally, the findings of the study related to the reflective-formative HOCs comply with the criteria for assessing structural models, further strengthening the credibility of the findings.

5. Discussion

This study significantly advances the understanding of factors influencing students’ engagement in DIL environments. Utilizing SDT as its foundation, the research introduces a model encompassing AI competence, chatbot usage and PA. Employing PLS-SEM for analysis, the study confirms the model's validity, with an R² value of 0.592, explaining 59.2% of the variance in students’ engagement. This high explanatory power underscores the model's effectiveness in capturing key drivers of engagement in DIL contexts.

The affirmation of the first hypothesis highlights the pivotal role of AI competence in enhancing students’ engagement, aligning with Yakin et al. (2023) and Moral-Sánchez et al. (2023). Proficiency in AI technologies fosters confidence and capability in navigating digital environments, thereby enhancing engagement. This finding underscores the importance of incorporating AI literacy into educational curricula, recognizing AI skills as essential for navigating the 21st-century digital landscape.

Support for the second hypothesis underscores the crucial role of AI competence in fostering DIL, aligning with Hatlevik (2017) and Hidayat-ur-Rehman and Ibrahim (2023). Effective use of AI technologies, including chatbots, enhances informal learning experiences, enabling personalized and interactive learning. This suggests AI competence facilitates information access and enriches learning. The finding highlights the need for educational strategies promoting AI skills, empowering students to leverage digital tools for autonomous learning.

Empirical support for the third hypothesis underscores the significance of AI competence for effective chatbot usage in learning, aligning with Chiu et al. (2023). This suggests students with higher AI skills are more adept at using chatbots as learning tools due to a better understanding of AI interactions and information extraction. This finding highlights the potential of chatbots as personalized learning aids, provided students have the necessary AI competencies to fully utilize these technologies.

The validation of the fourth hypothesis strengthens the link between DIL and student engagement, reflecting findings by Heidari et al. (2021) and Lee and Dressman (2018). This suggests that engagement is significantly influenced by quality DIL experiences. Digital tools and resources, supporting informal learning, play a crucial role in engaging students, emphasizing the importance of creating content-rich and interactive digital learning environments.

Support for the fifth hypothesis indicates that chatbot usage enhances student engagement, echoing Yakin et al. (2023) and Pillai et al. (2023). This relationship underscores the value of integrating chatbots into digital learning to stimulate interest and participation. Effectively utilized, chatbots provide immediate feedback, personalized learning paths and an interactive presence, serving as engaging learning tools.

The confirmation of the sixth hypothesis suggests that chatbot usage in DIL enhances students' perceptions of autonomy in their learning processes, aligning with Hidayat-ur-Rehman and Ibrahim (2023) and Pillai et al. (2023). This finding highlights chatbots' role in creating a learning environment where students feel in control, exploring topics at their own pace. This PA fosters intrinsic motivation and engagement, aligning with SDT principles.

Finally, the support for the seventh hypothesis indicates that higher perceptions of autonomy in DIL environments enhance students’ engagement, confirming Hidayat-ur-Rehman (2024) and Yang et al. (2022). This finding highlights the importance of designing autonomy-supportive learning experiences facilitated by digital tools and AI technologies, which significantly contribute to higher students’ engagement. Empowering students with choice in their learning is crucial.

In summary, the study substantiates its proposed model with solid empirical evidence, enriching the literature by mapping interactions between AI competence, chatbot usage, PA and students’ engagement in DIL. The findings offer valuable insights for educators and policymakers, emphasizing AI competence, effective chatbot integration and cultivating autonomous learning environments to enhance digital learning outcomes and engagement.

6. Theoretical and practical implications

This research integrates AI competence, chatbot usage and PA within SDT, offering significant contributions to DIL and students’ engagement. It validates and extends SDT's applicability to the evolving educational technology landscape.

Firstly, the investigation highlights the critical importance of AI competence in enhancing students’ engagement, suggesting that AI literacy is essential for modern educational success. This insight calls for a re-evaluation of educational theories to include digital literacy as a core component. Secondly, the findings underscore the importance of AI competence in DIL, highlighting it as a critical skill for enhancing learning autonomy and personalization. This suggests a theoretical shift to recognize technological proficiency's role in informal learning settings. Furthermore, the analysis emphasizes the effectiveness of chatbot usage in learning environments, demonstrating AI's potential for personalized education. This extends traditional educational theories to incorporate AI-mediated learning interactions as significant factors in students’ engagement and learning outcomes.

The results validate the link between DIL and students’ engagement, highlighting the significant impact of digital tools on engagement. This supports updating engagement theories to include the digital dimension, reinforcing the need for a broader theoretical understanding of digital environments. Implementing a disjoint two-stage approach for reflective-formative HOCs marks a methodological breakthrough in educational research. This method allows detailed exploration of complex constructs like digital competence and smartphone usage, enhancing analysis accuracy. It deepens understanding of these dimensions and sets a new benchmark for methodological excellence in educational studies.

From a managerial perspective, this research provides actionable insights for educators, administrators and policymakers. It underscores the need to integrate AI competence into curricula, preparing students to effectively navigate digital tools. This calls for a strategic curriculum overhaul to include AI literacy as a fundamental component, enhancing digital learning engagement. Additionally, the study highlights chatbots' crucial role in enhancing DIL. Educational institutions should adopt chatbot technologies for personalized, interactive learning. Managerial strategies should invest in AI technologies supporting chatbot deployment, boosting students’ engagement and learning autonomy. Furthermore, the findings on PA and its impact on students’ engagement advocate for empowering learning environments. This implies a managerial shift toward educational experiences prioritizing student choice and self-directed learning, supported by digital tools and AI technologies, and enhancing students’ engagement through autonomy-supportive practices.

Overall, the managerial implications advocate for incorporating AI and chatbot technologies in education and fostering autonomy-promoting environments. Aligning educational practices with these insights can enhance learning outcomes and better prepare students for success in the digital age.

7. Limitations and future research avenues

This study explores the complex relationship between AI competence and students’ engagement, acknowledging the need for further research due to its specific context and participant limitations. To enhance the findings' applicability, future research should extend these inquiries across diverse educational contexts with a broader participant base to confirm generalizability. Investigating cultural differences in shaping digital competence and engagement promises a globally nuanced understanding of these interactions.

The cross-sectional design limits causal inference, suggesting the need for longitudinal studies to track digital competence and its influence on engagement over time. While this study identifies key mediators within the AI competence–engagement relationship, exploring additional mediating and moderating variables could provide deeper insights. Future research might consider technological self-efficacy, personalized learning experiences and collaborative learning opportunities as mediators and motivation type (intrinsic vs. extrinsic) and technology accessibility as moderators.

Examining the efficacy of interventions to enhance AI competence and their effects on engagement could offer practical guidance for educators and policymakers. Additionally, given the rapid technological evolution, the impact of emerging tools like virtual reality and novel AI applications on the AI competence-engagement dynamic warrants investigation.

Pursuing these recommended avenues can significantly deepen our understanding of AI competence's role in boosting student engagement, providing valuable implications for improving educational strategies and technology integration in the digital era.

Figures

Figure 1

Proposed framework of the study

Figure 2

Bootstrapping results (inner model: path coefficients and p-values)

Table 1

Summary of reliability and convergent validity assessments

Constructs	α > 0.7	Composite_ reliability	Meas. Items	Indicators’ reliability	AVE
		>0.7		>=0.7	>0.5
AIC	0.874	0.878	AIC1	0.865	0.725
			AIC2	0.807
			AIC3	0.856
			AIC4	0.877
CL	0.828	0.831	CL1	0.841	0.661
			CL2	0.812
			CL3	0.835
			CL4	0.761
CU	0.824	0.829	CU1	0.719	0.657
			CU2	0.862
			CU3	0.860
			CU4	0.794
MCL	0.815	0.815	MCL1	0.836	0.645
			MCL2	0.820
			MCL3	0.841
			MCL4	0.710
PA	0.883	0.890	PA1	0.773	0.681
			PA2	0.836
			PA3	0.849
			PA4	0.832
			PA5	0.836
SE	0.899	0.899	SE1	0.916	0.831
			SE2	0.912
			SE3	0.908
SML	0.886	0.886	SML1	0.877	0.746
			SML2	0.887
			SML3	0.889
			SML4	0.798

Source(s): Author’s own work

Table 2

HTMT ratios evaluation

	AIC	CL	CU	MCL	PA	SE	SML
AIC
CL	0.367
CU	0.355	0.611
MCL	0.363	0.576	0.642
PA	0.215	0.164	0.281	0.122
SE	0.550	0.605	0.712	0.511	0.450
SML	0.440	0.416	0.330	0.302	0.139	0.534

Source(s): Author’s own work

Table 3

Summary of hypotheses testing

Hyp. #	Relationship	Path coefficient	Standard deviation	t-values	p-values	Remarks
H1	AIC → SE	0.203	0.037	5.503	0.000	Supported
H2	AIC → DIL	0.446	0.045	9.915	0.000	Supported
H3	AIC → CU	0.303	0.043	7.021	0.000	Supported
H4	DIL → SE	0.331	0.041	8.142	0.000	Supported
H5	CU → SE	0.305	0.043	7.027	0.000	Supported
H6	CU → PA	0.244	0.050	4.870	0.000	Supported
H7	PA → SE	0.238	0.036	6.679	0.000	Supported

Source(s): Author’s own work

Funding: The author received no financial support for the research, authorship and/or publication of this article.

Declaration of competing interest: The author declares that he has no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability: Data will be made available on request.