Artificial intelligence-powered tools and academic writing: to use or not to use ChatGPT

Abstract

Purpose

This study investigates the potential effectiveness of ChatGPT in enhancing the academic writing skills of Saudi EFL undergraduate students. It also examines the challenges associated with its use and suggests effective ways to address them in the education sector.

Design/methodology/approach

The study employed a sequential mixed-methods approach, which involved distributing questionnaires to gather data from students, followed by conducting semi-structured interviews with a purposeful selection of eight students and six teachers.

Findings

The findings revealed that students were generally satisfied with the effectiveness of ChatGPT in enhancing their academic writing skills. However, they also pinpointed some challenges associated with using ChatGPT, including plagiarism, overreliance, inadequate documentation, threats to academic integrity, and inaccurate information. To alleviate these challenges, effective strategies include deploying detection tools, equipping students and educators with training sessions, and revisiting academic policies and assessment methods. It is recommended that ChatGPT be used responsibly as an assistant tool, in conjunction with students' ideas and teachers' feedback. This approach can significantly enhance students' writing skills and facilitate completing their research projects and assignments.

Practical implications

ChatGPT can be a valuable tool in the educational landscape, but it is essential to use it judiciously. Therefore, teachers' effective integration of ChatGPT into their classrooms can significantly enhance students' writing abilities and streamline their research process.

Originality/value

This study contributes to recent AI-based research and provides practical insights on the responsible integration of ChatGPT into education while addressing potential challenges.

Citation

Al-Sofi, B.B.M.A. (2024), "Artificial intelligence-powered tools and academic writing: to use or not to use ChatGPT", Saudi Journal of Language Studies, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/SJLS-06-2024-0029

Introduction

Acquiring academic writing skills is a key objective in various English programs. Recently, AI-powered tools and applications have emerged to reshape education. They help educators tailor learning paths to meet the diverse needs of students. These tools also enrich classroom discussions by providing real-time data for research and personalized feedback to students. Additionally, they impact how students approach writing and the writing process. OpenAI has developed ChatGPT (Chat Generative Pre-trained Transformer), an AI-powered chatbot that generates conversational-style texts, resembling human-like responses based on the users’ input (Cotton et al., 2023). This tool covers the entire writing process, from generating ideas to the final stages of proofreading and editing the generated texts (Altmäe et al., 2023; Imran and Almusharraf, 2023; Salvagno et al., 2023; Wardat et al., 2023).

ChatGPT can function as an always-available assistant for students, offering support in understanding difficult concepts and ensuring a seamless learning experience in and out of the classroom. It has gained recognition for its ability to simplify the research process by automating and enhancing the writing skills of students, teachers, and researchers. These individuals are excited to employ ChatGPT for high-quality academic writing (Cotton et al., 2023; Dergaa et al., 2023; Elbanna and Armstrong, 2023; Strzelecki, 2023).

Students can utilize ChatGPT to ask questions, generate and brainstorm ideas, summarize papers, or proofread their assignments (Altmäe et al., 2023; Dergaa et al., 2023). Similarly, it has transformed the way teachers impart knowledge and evaluate assignments. Teachers can integrate this tool into their teaching methods to revise and assess tasks more efficiently, create engaging exams, and free up time to focus on higher-level learning activities that benefit students and teachers alike. Moreover, it has revolutionized academic research by effortlessly generating texts, editing, translating, and proofreading. In this sense, Salvagno et al. (2023) stated that “ChatGPT is an AI software potentially able to assist in the writing process of a scientific paper and can help in the literature review, identify research questions, provide an overview of the current state of the field, and assist with tasks, such as formatting and language review” (p. 4).

However, the advent of ChatGPT has brought about certain challenges and ethical issues. It can lead to plagiarism and misconduct, inaccurate information and referencing, excessive dependence on the tool, and a decline in critical thinking and creativity. AlAfnan et al. (2023) highlighted the temptation for students to utilize ChatGPT to generate assignment submissions, an approach that offers no educational or professional growth.

Research problem

The emergence of ChatGPT has sparked students' interest in generating high-quality academic and research-based writing. However, some students may excessively depend on ChatGPT or use it irresponsibly to perform their academic writing tasks, which may not accurately showcase their true writing skills. Despite the growing use of ChatGPT, a critical gap persists in academic research concerning the possible effectiveness of this tool in enhancing English as a Foreign Language (EFL) students’ academic writing skills, particularly within the context of performing research projects and assignments.

Research objectives

The present study aims to scrutinize the potential effectiveness of ChatGPT (GPT-4 Version) in enhancing the academic writing skills of Saudi EFL students while writing their research projects. Furthermore, it explores the possible challenges of using ChatGPT for writing research papers and assignments and proposes actionable strategies to overcome them, ultimately ensuring high-quality work.

Literature review

This section examines the theoretical framework for the study and reviews relevant studies in the field.

Theoretical framework

ChatGPT and academic writing

A plethora of studies have acknowledged the valuable and significant role of ChatGPT in revolutionizing academic and scientific writing (AlAfnan et al., 2023; Altmäe et al., 2023; Barrot, 2023; Imran and Almusharraf, 2023; Mijwil et al., 2023; Perkins, 2023; Rahman and Watanobe, 2023; Sallam, 2023; Salvagno et al., 2023; Sok and Heng, 2024; Wardat et al., 2023; Yu, 2023). This consensus among researchers stems from recognizing the transformative impact that AI-driven language models, such as ChatGPT, have had on fostering academic writing skills. ChatGPT functions as an interactive and generative tool that spans various aspects of the writing process, from generating ideas to proofreading content, ultimately resulting in more effective and reviewed academic writing.

Wardat et al. (2023) demonstrated that the application of ChatGPT in education and academia has sparked significant discussions. One key concern surrounding ChatGPT lies in its ability to generate written content that closely resembles human writing. This raises questions about its potential impact on academic integrity and authenticity. Sallam (2023) highlighted that ChatGPT holds great promise as a tool for scientific research, both in academic writing and the research process. Mijwil et al. (2023) further supported this by stating that “ChatGPT was used in research writing by writing the introduction, abstract, codes, conclusions, arranging references, improving writing language, and other things” (p. 116). Perkins (2023) emphasized that ChatGPT can “generate original text and assist students in the writing process, but it is important to consider the potential ethical implications of its use” (p. 6). While ChatGPT offers clear advantages in developing academic writing skills, it also presents challenges that require ongoing discussion and resolution.

Limitations of ChatGPT

Although ChatGPT can potentially enhance students' academic writing skills, its irresponsible use poses challenges and ethical considerations for educators and educational institutions. These issues include overreliance, plagiarism, inaccurate or biased information, improper referencing, a decrease in creativity and critical thinking, difficulties in evaluating ChatGPT-generated content, and misleading assessment of students' assignments (AlAfnan et al., 2023; Cotton et al., 2023; Eke, 2023; Lingard, 2023; Lo, 2023; Meyer et al., 2023; Mijwil et al., 2023; Rahman and Watanobe, 2023; Salvagno et al., 2023; Wardat et al., 2023; Yu, 2023). ChatGPT acknowledges its susceptibility to errors and emphasizes the importance of verifying critical information. Morgan (2023) argued that ChatGPT may never be ideal for qualitative analysis, highlighting its limitations for researchers. Morgan’s (2023) emphasis suggests that successful practical components require significant human efforts, with individuals leveraging their intellectual abilities and possessing greater expertise and experience in preparing, coding, analyzing, and discussing data. These skills and abilities are indispensable for ensuring the credibility and success of research work.

Eke (2023) highlighted that using ChatGPT to generate essays and presenting them as one’s original work violated academic integrity. In a comprehensive analysis of 50 studies, Lo (2023) identified false information and plagiarism as obstacles linked to ChatGPT. Lingard (2023) proposed that while ChatGPT proves to be effective in organizing content, it should be used with caution, especially in unfamiliar topics, as careful verification may be necessary. Nonetheless, ChatGPT remains highly beneficial in tasks such as providing title suggestions and creating outlines.

These challenges pose a difficulty for educators and supervisors in differentiating between AI-generated and human-written texts, potentially impacting academic integrity. This can lead to inaccurate assessments of student work, failing to capture their true abilities and hindering the development of crucial writing skills. Yu (2023) concluded that maximizing the educational benefits of ChatGPT in education requires recognizing and addressing the potential risks and challenges associated with its use. This includes implementing appropriate regulation and management practices. By addressing these challenges, we can ensure the responsible use of ChatGPT in various contexts and make it a central focus in the ongoing development and deployment of AI-powered tools, including ChatGPT.

Ways to overcome the potential challenges

To circumvent the challenges associated with the use of ChatGPT, a variety of effective strategies have been proposed (AlAfnan et al., 2023; Cotton et al., 2023; Eke, 2023; Kumar, 2023; Lingard, 2023; Meyer et al., 2023; Naidu and Sevnarayan, 2023; Sallam, 2023). For instance, Kumar (2023) found that increasing students' awareness of the potential benefits and limitations of ChatGPT positively influences their academic characteristics and motivates them to engage more effectively in metacognitive learning. However, similar to previously developed AI-driven applications, scholars suggested that ChatGPT should be employed as a supplementary writing tool to improve structure, brainstorm ideas, and proofread human-generated texts, rather than relying solely on it to generate texts from scratch. Sallam (2023) urged stakeholders to implement appropriate guidelines and regulations to promote the safe and responsible use of ChatGPT. Likewise, Eke (2023) called for updating academic integrity policies and equipping educators with training, support, and technological resources to address ethical concerns surrounding ChatGPT use.

Oral presentations could also be incorporated as a significant part of the student assessments when AI-based tools are employed in their studies. Furthermore, Cotton et al. (2023) suggested that academic staff can educate students on plagiarism, use AI-detection software, review and revise their policies and procedures, establish clear guidelines for ChatGPT usage, and closely monitor student’s work. Teachers can also design open-ended assessments to foster students' originality, critical thinking, and creativity while reducing their dependence on ChatGPT. Moreover, Meyer et al. (2023) proposed that students can be trained to use ChatGPT positively while adhering to ethical standards set by educational institutions. Implementing these actionable strategies can alleviate the challenges associated with using ChatGPT and foster the development of students’ academic writing skills.

Previous studies

Many previous studies have investigated the effectiveness and potential challenges of ChatGPT in different fields. For example, Shoufan (2023) examined students' viewpoints on employing ChatGPT in education. The results revealed that students found ChatGPT engaging, inspiring, and beneficial for academic and professional purposes. However, they also acknowledged certain limitations associated with ChatGPT, such as concerns regarding the accuracy of responses, academic integrity, and the potential impacts on employment and daily life.

Tlili et al. (2023) explored the perspectives of policymakers, educators, and learners regarding the role of ChatGPT in education. The analysis of tweets, interviews, and user experiences revealed the potential of ChatGPT to revolutionize education. However, the study emphasized the need for careful usage and the development of comprehensive guidelines to ensure its safe application in educational settings. They recommended embracing ChatGPT instead of imposing a complete ban on its use in education.

Mijwil et al. (2023) examined the ability of ChatGPT' to generate a scientific paper that adheres to the conventions of academic writing. Consistent with Morgan’s (2023) viewpoint, the results showed that although ChatGPT does not fully meet the rigorous standards of scholarly journals, it does excel in promptly providing precise information and utilizing flawless language. Therefore, ChatGPT has emerged as a valuable tool for task completion and improved writing quality. However, it is crucial to emphasize that ChatGPT should not be seen as a substitute for human expertise, effort in the writing process, and critical thinking.

In 2023, Imran and Almusharraf reviewed 30 articles to examine the role of ChatGPT as a writing assistant in academia. The results indicated that utilizing ChatGPT as a writing assistant had benefits and drawbacks for students and educators. However, to maintain academic integrity and originality, academia should reevaluate and improve training programs for students and teachers, revisit current policies, and adjust assessment methods in writing courses. This entails addressing issues such as plagiarism, AI-generated assignments, online or home-based exams, and the difficulties associated with automated corrections.

Mahapatra (2024) investigated the impact of ChatGPT on the writing skills of Indian undergraduate ESL students. The results indicated that they had positive perceptions of the role of ChatGPT on their academic writing skills. Based on the findings of Bin-Hady et al. (2023) regarding the potential improvement of learners' language abilities through ChatGPT, Ali et al. (2023) found that ChatGPT played a positive role in stimulating students’ interest in developing their reading and writing skills. Additionally, teachers perceived ChatGPT-based instruction as a valuable and motivating learning tool, advocating for its integration into education instead of its prohibition.

AlAfnan et al. (2023) investigated the potential benefits and challenges of ChatGPT for students and instructors in communication, business writing, and composition courses. The study found that ChatGPT provided students with accurate and reliable information for addressing theoretical and practical questions. Likewise, instructors could incorporate ChatGPT into their classrooms to facilitate discussion and evaluation of the generated responses. However, the study also raised concerns about the unethical use of ChatGPT, which could potentially undermine human intelligence and impede learning. Furthermore, instructors encountered difficulties differentiating between human-written and ChatGPT-generated content, making it more challenging to assess learning outcomes.

Altmäe et al. (2023) and Dergaa et al. (2023) conducted separate studies in various contexts to identify the potential benefits, limitations, and practical implications of ChatGPT in academic research. The studies confirmed that ChatGPT can be valuable as a writing assistant, improving writing skills and research efficiency. However, the studies also highlighted the need to use ChatGPT cautiously to prioritize academic integrity and ethical considerations. Human intelligence and critical thinking should always be at the forefront of the research process. Moreover, it is essential to properly acknowledge and cite the contributions of ChatGPT to avoid plagiarism, maintain scholarly writing standards, and fully leverage its advantages.

Chan and Hu (2023) explored the perceptions of Chinese students regarding the potential benefits and challenges of ChatGPT. The respondents reported a generally positive view of ChatGPT, valuing its ability to enhance the entire writing process, from initial brainstorming to final proofreading and editing. However, concerns were raised regarding accuracy, privacy, ethical issues, and the potential impact on personal development, career prospects, and societal values. Similarly, Khalifa and Albadawy (2024) investigated the multifaceted role of AI in supporting academic writing and research. Their review of 24 studies, retrieved from various databases, revealed that AI can assist students and researchers throughout the writing process, from initial brainstorming to post-publication communication.

After examining the previously cited research, it is evident that there is a general agreement regarding the widespread use and effectiveness of ChatGPT in different fields, especially in education. It can assist in theoretical aspects by providing information, generating content, and proofreading human-generated work. Rather than replacing educators, ChatGPT enhances their role in developing students' academic writing skills. However, as ChatGPT is a relatively new development, ethical concerns have been raised. It is expected that stakeholders will increasingly embrace the use of ChatGPT to maximize its usefulness in different areas while adhering to academic principles.

Unlike previous studies that have examined the effectiveness and challenges of using ChatGPT in various disciplines, the current study specifically focuses on a particular group of students enrolled in two research courses within the English Department at the University of Bisha in Saudi Arabia. These students use ChatGPT to prepare their research projects and proposals. Notably, the prior studies were conducted shortly after the emergence of the tool, at a time when it was inaccessible in some countries, including Saudi Arabia. Furthermore, these studies may be limited by stakeholders’ unfamiliarity with ChatGPT’s capabilities. The present study was conducted after ChatGPT became accessible in Saudi Arabia and people gained sufficient knowledge and experience in its usage. Consequently, this study is expected to contribute significantly to the existing body of AI research in education.

Research questions

RQ1.

To what extent does ChatGPT effectively enhance Saudi EFL students' academic writing?

RQ2.

What challenges arise when employing ChatGPT in academic writing?

RQ3.

What strategies can be implemented to mitigate the challenges associated with ChatGPT use in academic writing?

Methodology

Research design

A sequential mixed-methods approach was used for data collection. This approach “involves a two-phase data collection project in which the researcher collects quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on to) the second, qualitative phase” (Creswell and Creswell, 2018, p. 304). Furthermore, the qualitative data enhance the validity of the claims and strengthen the study by offering a nuanced understanding of the topic under investigation (Mahapatra, 2024). In the case of the present study, this two-phase approach started with collecting quantitative data through an online questionnaire, analyzing them, and then using those results to expand to a subsequent qualitative phase using semi-structured interviews with a selected group of students and teachers. These interviews aimed to gain deeper insights into their experiences with ChatGPT (Creswell and Creswell, 2018).

Participants

The data for the study were collected from 102 out of 117 Saudi EFL undergraduate students who were enrolled in the Department of English at the University of Bisha in Saudi Arabia. The remaining 15 students did not respond to the questionnaire. The respondents took two compulsory courses: Basics of Scientific Research in English and Research Project, both taken in the final year. Following the research phase, students were required to defend their research project before a jury. The selection of these students was driven by their English language proficiency, enrollment in the English department, firsthand experience with the tool, and the need for assistance in writing their projects.

In terms of familiarity with ChatGPT, Table 1 demonstrates that most participants were generally familiar with using ChatGPT for writing assignments and projects. Other participants rarely (N = 13, 12.7%) or never (N = 10, 9.8%) used ChatGPT. Regarding the participants’ enrollment in courses, over half of the participants (N = 60, 58.8%) took the Basics of Scientific Research in English course, a prerequisite for the Research Project course (N = 42, 41.2%).

To gain deeper insights, semi-structured interviews were conducted with eight participants who had previously completed the questionnaire. They were chosen based on their critical thinking skills, active participation in the classroom, and willingness to be interviewed. In addition, six teachers were interviewed. They were selected since they teach these courses to the respondents. They also have cumulative experience in teaching English courses at the university level.

Research instruments

First, a four-section questionnaire was developed using Google Forms and based on previous research (Ali et al., 2023; Shoufan, 2023). The study objectives were clearly stated, and confidentiality and anonymity were ensured at the beginning of the questionnaire. The first section sought to gather information about respondents' familiarity with ChatGPT and their enrollment in the selected courses. The second section focused on the effectiveness of ChatGPT and its potential role in enhancing students' academic writing skills. The third section elicited potential challenges that could arise from using ChatGPT. The last section focused on effective ways to alleviate these challenges. A five-point Likert-type scale was employed, ranging from “strongly disagree” (1) to “strongly agree” (2). The questionnaire’s design underwent revision, benefiting from the expertise of three linguists in the department. Furthermore, Cronbach’s Alpha test revealed an extremely high level of reliability for the questionnaire (p = 0.965).

After the questionnaire closed, eight students were purposefully selected for in-depth interviews to delve deeper into their experiences with ChatGPT. Similarly, a purposive sampling strategy was employed to conduct informal interviews with six teaching staff members who had taught these courses at various times. These interviews aimed to obtain more in-depth insights into the potential effectiveness and limitations of ChatGPT in enhancing their students' academic writing.

Data collection and analysis procedures

The instructors of the courses facilitated sharing the questionnaire link with students via WhatsApp groups during the academic year 2023/2024. To ensure ethical considerations, the participants were explicitly informed that their participation was voluntary. Assurances were given that their information would remain confidential and anonymous and not be disclosed to a third party. Participants also indicated their informed consent to participate in the study voluntarily by clicking the “Next” icon and proceeding with the questionnaire. They were prompted to assess their experience with ChatGPT, including both the benefits they enjoyed and the challenges they encountered.

Descriptive statistics (frequencies and percentages) were used to analyze the questionnaire data, using SPSS (Version 26). Subsequently, eight students were interviewed to gather further information that would clarify and validate the quantitative data of the questionnaire. Second, semi-structured interviews were carried out with the instructors of these courses. The interviews began with three central questions to establish a foundation. Additional follow-up questions were used to explore how ChatGPT contributes to students' development of their writing skills. The interviews were transcribed with the interviewees' prior agreement. Content analysis was used to qualitatively analyze the interview data, identifying and coding similar themes. A coding scheme was created, with the effectiveness of ChatGPT coded as E-X, challenges as C-X, and suggestions for alleviating these challenges as S-X. The “X” code represents the frequency of these themes. The quantitative statistical results were followed by a discussion of the qualitative findings, using a side-by-side approach (Creswell and Creswell, 2018). This approach facilitated a holistic examination of both qualitative and quantitative data.

Results and discussion

This section presents a combined analysis and discussion of the questionnaire and interview data. It is divided into three subsections to address the research questions and fulfill the study objectives.

The effectiveness of ChatGPT on academic writing skills

RQ1: to what extent does ChatGPT effectively enhance Saudi EFL students' academic writing?

Table 2 reports that half of the participants (N = 51, 50%) agreed and many other students (N = 33, 32.4%) strongly agreed that ChatGPT generated helpful texts for their research work (M = 4.06). Similarly, less than half of the participants (N = 42, 41.2%) agreed and (N = 36, 35.3%) strongly agreed that ChatGPT assisted them in summarizing relevant literature in their chosen field (M = 4.06), indicating that the participants used ChatGPT for generating theoretical content and summarizing literature. However, it was noticed that ChatGPT scored the lowest mean in keeping students up to date with the latest research trends (M = 3.80). This can be attributed to the limited information stored in ChatGPT. Therefore, students are encouraged to explore other well-known search engines and databases to stay informed about the latest research trends. It is worth mentioning that the overall weighted mean (M = 3.93) for the entire scale falls within the “agree” category, and the weighted mean for each item on the scale also falls within the “agree” category. This suggests that the participants generally agreed with the effectiveness and benefits of ChatGPT in enhancing their academic writing skills. It highlights the need for students to be more aware of how to effectively and ethically use AI-powered tools to develop their academic writing skills.

The students interviewed emphasized that ChatGPT’s standout features are its time-saving and user-friendly nature. For instance, student B stated, “ChatGPT is beneficial for research projects. It saves students' time.” Student C pointed out that “ChatGPT excels at summarizing information, making it easier for users to obtain necessary information.” Student E elaborated that ChatGPT can be used to research any idea or topic and proofread the students' research and assignments.

Interestingly, student E noted the potential of ChatGPT extending beyond education to other fields. It can be used to “get information about education, administration, and even medicine, research and revise assignments, create exams, complement students' abilities, and develop language skills, especially writing skills.” Student F believed that “ChatGPT helps students get any information they need for their studies.”

Regarding the brainstorming theme, three of the interviewed students (G, H, and I) confirmed using ChatGPT to brainstorm ideas and choose a topic for their project. They were able to gather information about the project but selectively chose the information they found useful. These students highlighted ChatGPT’s utility as a time-saving, cost-effective, and effort-reducing assistant tool for their studies.

It is intriguing to note that some students alluded to the application of ChatGPT beyond the educational setting for various purposes such as “voice modification” (Student A), “image processing” (Student D), “website design” (Student D and F), and “receiving accurate and quick translations” (Students G, H, and I). These diverse uses and engagements align with the students' skill development, bridging the gap between academic writing proficiency and the evolving demands of the professional world.

Supporting the opinions of the students gathered from the questionnaire and interviews, the teachers were also asked about the impact of ChatGPT on their students' overall work quality, particularly in research projects and other text-based assignments. The interviews revealed unanimous agreement among the six teachers regarding the effective role of ChatGPT in enhancing their students' academic writing skills. For instance, teacher 4 affirmed that ChatGPT significantly enhances her students' academic writing skills “based on their individual levels.” Likewise, teacher 5 confirmed that ChatGPT “provides students with feedback on their essays by offering innovative ideas for assignments, helping them learn new skills and improve the quality of their academic essays.”

To summarize, it is apparent that the participants generally agreed on ChatGPT’s effectiveness in enhancing their academic writing skills throughout the entire writing process, starting from brainstorming, generating ideas, and outlining, all the way to proofreading, and editing their projects and assignments. This result aligned with some previous studies conducted in different contexts (AlAfnan et al., 2023; Ali et al., 2023; Bin-Hady et al., 2023; Chan and Hu, 2023; Mijwil et al., 2023; Shoufan, 2023; Tlili et al., 2023).

The interviews' unique aspects lie in highlighting students' exploration of additional applications of ChatGPT that extend beyond educational boundaries. They specifically mentioned voice modification, image processing, website design, and translation, emphasizing the relevance of these activities in meeting the demands of today’s digital-centric job market. Moreover, students affirmed ChatGPT’s applicability in the administration and medicine fields. They also highlighted the usefulness of ChatGPT for teachers in tasks such as revising assignments and creating exams. Students particularly highlighted the ease of use and time-saving features of ChatGPT as major benefits.

The potential challenges of using ChatGPT

RQ2: what challenges arise when employing ChatGPT in academic writing?

Table 3 displays that the overall weighted mean (M = 3.21) for the entire scale fell within the “neutral” category, indicating that the respondents neither agreed nor disagreed regarding the challenges associated with using ChatGPT to enhance their academic writing skills. Specifically, some participants strongly disagreed (N = 25, 24.5%) and (N = 35, 34.4%) that ChatGPT can replace teachers in providing feedback. This result confirms the vital role of teachers, highlighting that AI-based tools should be seen as complementary rather than substitutes for teachers in editing and proofreading students' generated texts. Additionally, the participants agreed (M = 3.23) that the use of ChatGPT compromised the integrity of academic writing. This underscores the importance of ethically utilizing ChatGPT in education.

Unexpectedly, item number 1, which pertains to generating inaccurate or misleading content by ChatGPT in research projects, received the lowest mean score (M = 2.81) on the scale. The neutrality of the overall scale suggests that the participants did not strongly agree or disagree regarding these challenges. Respondents may hold positive and negative opinions about using ChatGPT for academic writing, resulting in this neutral stance. They might not be fully aware of or unwilling to acknowledge these limitations. The overall results were inconsistent with the results of previous studies (Eke, 2023; Lingard, 2023; Lo, 2023; Meyer et al., 2023; Mijwil et al., 2023; Rahman and Watanobe, 2023; Salvagno et al., 2023; Wardat et al., 2023), which identified overreliance, plagiarism, inaccurate or biased information, improper referencing, and a decrease in creativity and critical thinking as the primary challenges resulting from the use of ChatGPT in education.

Interviews with the students revealed valuable insights and a nuanced understanding of the challenges they encountered while using ChatGPT for academic writing. For example, student F pointed out that “students cannot solely rely on it as they might become careless. There are no references, and we do not know whether such information is true or false.” Similarly, students (G, H, and I) mentioned the issue of plagiarism, stating that “the generated content lacked citations.” Interviewees' remarks about the tendency to overly depend on ChatGPT, resulting in carelessness, the absence of references, and unverified information confirm that ChatGPT has crucial limitations.

Likewise, students' instructors were asked about the potential challenges they might face when their students use ChatGPT for writing their projects and assignments. Most of their responses centered on ethical issues and a lack of students' skills. For instance, teacher 1 expressed that the students' use of ChatGPT “may increase plagiarism and spread academic dishonesty.” Similarly, teacher 2 alluded to “ethics and a degradation of writing skills” that may stem from the students’ reliance on ChatGPT for writing projects and assignments.

The teachers also expressed concerns about the lack of creativity and critical thinking skills, resulting from using ChatGPT. Teacher 3 thought that the students' use of ChatGPT leads to “less innovation and a lack of support for creative thinking.” Teacher 4 disclosed that the students “simply copy and paste. They don’t know how to paraphrase the generated ideas.” The teacher raised the concern that students may not fully understand the concepts they write about, as they may merely copy and paste information provided by ChatGPT without genuinely understanding the subject matter. This could impede the development of their analytical and research skills, which are crucial for academic growth.

Interestingly, teacher 5 cited “misinformation” as a challenge they faced. He emphasized that ChatGPT can be a “source of distraction if not used productively.” He further referred to the students' dependence on this tool, stating that “ChatGPT may create a sense of dependency or complacency among students who rely too heavily on its assistance or output.” This overreliance on ChatGPT might also contribute to a “lack of emotional intelligence” in writing, as students may not fully develop their ability to convey emotions effectively. According to the sixth teacher, “ChatGPT seems ineffective in AI-generated content. Students do not seem to understand the mistakes pointed out to them in the ChatGPT content, as it hasn’t been created by them.”

To sum up, these challenges align with the findings of the previous studies and questionnaire responses, despite the students’ neutrality. Although the students did not take a clear stance or express concerns, the challenges highlighted by interviewed teachers and students were still evident and reflected the findings from prior research (AlAfnan et al., 2023; Ali et al., 2023; Bin-Hady et al., 2023; Chan and Hu, 2023; Mijwil et al., 2023; Shoufan, 2023; Tlili et al., 2023) and questionnaire responses (see Table 2). Specifically, students and teachers raised primary concerns, such as the potential for plagiarism in ChatGPT-generated texts, excessive reliance on ChatGPT, and a lack of creativity and critical thinking skills. Students' irresponsible use of ChatGPT, particularly overreliance on it as the sole tool for writing projects and text-based assignments, can lead to academic dishonesty and hinder the development of essential writing skills. Therefore, based on the questionnaire results, since the entire scale fell within the neutral range, students need to be made more aware of the challenges posed by ChatGPT. Addressing these challenges effectively requires collaborative efforts and actions from the concerned stakeholders.

Mitigating the challenges of ChatGPT for effective academic writing

QR 3: what strategies can be implemented to mitigate the challenges associated with ChatGPT use in academic writing?

The results presented in Table 4 revealed that the overall weighted mean for the entire scale was (M = 3.87), placing it in the “agree” category. More precisely, the respondents expressed that training sessions focusing on the effective use of ChatGPT (M = 4.27) and employing ChatGPT for proofreading and editing already-written texts rather than generating texts from scratch (M = 4.16) were among the top viable methods that can be used to mitigate the challenges associated with ChatGPT. Similarly, students agreed that they should critically evaluate the information provided by ChatGPT (M = 4.12). This result was consistent with other previous studies which emphasized the need to reevaluate and enhance training programs for students and teachers to ensure academic integrity and originality (Imran and Almusharraf, 2023; Khalifa and Albadawy, 2024; Mahapatra, 2024). Students also perceived other methods as effective, with their mean scores falling within the “agree” range (See Table 4). Unexpectedly, the lowest mean score (M = 3.94) for the first item of the scale, which suggests that students should use ChatGPT as an assistant tool rather than relying on it exclusively, revealed that the students heavily relied on ChatGPT for their academic writing. This result supported the findings of the teachers' interviews, which indicated that students excessively relied on ChatGPT in their academic writing tasks.

Concerning the opinions of the interviewed students on the effective ways to minimize ChatGPT’s challenges, student E recommended asking “specific questions rather than general questions” when seeking help from ChatGPT to fulfill one’s needs. Student F proposed that ChatGPT “can be used to look for information, paraphrase it using your own words, and copy and paste it. It is recommended that references can be added to the tool.”

Similarly, student instructors were queried about effective strategies to address challenges arising from student use of ChatGPT in research projects and assignments. They offered a range of ways to mitigate the concerns mentioned above. For example, teacher 1 emphasized that “ChatGPT can provide students with individualized support and break down complex problems into smaller challenges or tasks, which increases the students' confidence level and reduces the likelihood of cheating.”

Aligned with the questionnaire findings, Teacher 2 recommended training students on effectively using ChatGPT before incorporating it into writing assignments. Teacher 3 suggested encouraging students to update their ideas by saying “Let them update their ideas more and more.” Inconsistent with the questionnaire results, where students seemed to over-rely on ChatGPT rather than use it as an assistant tool, teacher 4 underscored that students “should use it only for assistance. My suggestion is that they should write everything in their own words to promote critical thinking in their writing. Otherwise, this app will not truly help them learn.” Similarly, teacher 5 proposed that the students should “treat ChatGPT as a learning partner” and “use ChatGPT to boost confidence” while maintaining an active role in the learning process. Teacher 6 urged teachers to “provide [students] with an outline that they must strictly follow.” They should also “ask them to write the content in the class, whenever possible.” She continued to affirm that teachers can “ask them to explain whatever they have written.”

The participants generally reported positive experiences using ChatGPT to improve their academic writing skills. However, it was observed that they were somewhat cautious in their responses to the questionnaire. While they mostly agreed that ChatGPT was beneficial, their responses regarding its limitations leaned towards neutrality. Their cautious reporting may stem from the tool’s novelty and students' natural inclination towards objectivity. This could lead to under-reporting of challenges encountered while using ChatGPT. To address this, students may benefit from becoming more familiar with and gaining hands-on experience with AI tools like ChatGPT. Additionally, concerning the effective ways that might contribute to alleviating ChatGPT’s challenges, the study generated interesting ideas on how to address the challenges of ChatGPT, which can be responsibly implemented through collaborative efforts among the stakeholders.

Conclusion

This study investigated the potential of ChatGPT to enhance the academic writing skills of Saudi EFL undergraduate students. It also examined the challenges associated with its use in the educational setting, proposing strategies to alleviate these challenges. A sequential mixed-methods approach was adopted, utilizing online questionnaires filled out by 102 students enrolled in the Department of English at University of Bisha in Saudi Arabia. Additionally, semi-structured interviews were conducted with eight participating students and six teachers from the department.

The results revealed that the students generally agreed on the effectiveness and usefulness of ChatGPT for developing their academic writing skills. It proved helpful for generating relevant texts for research work, summarizing literature, enhancing assignments and research papers, and assisting with proofreading. However, they also acknowledge that using ChatGPT in academic writing presented several challenges. These included a potential decrease in the integrity of academic writing and critical thinking, an overreliance on the tool leading to a decline in creative abilities, the risk of plagiarism, and the generation of inaccurate or misleading content. Consequently, they suggested that ChatGPT should be used responsibly as a writing assistant to develop academic writing skills, rather than relying on it to generate ideas from scratch or replace the valuable guidance and feedback provided by teachers and supervisors. To address these limitations effectively, they recommended revisiting policies, implementing AI-detection programs, organizing training sessions and workshops for students, raising awareness about the importance of critical thinking and creativity, and ensuring adherence to academic principles.

Like other AI-driven editing and proofreading tools (such as Grammarly, Quillbot, Wordtune, and Paperpal), tools for analyzing data (such as SPSS, AMOS, MAXQDA, and NVivo), and tools for referencing (such as Mendeley, Endnote, and Zetero), the increasing use of ChatGPT makes it impractical to exclude it. It has the potential to revolutionize the way users approach academic writing. This necessitates their responsible use with clear guidelines. AI is becoming an increasingly integral part of the learning process, moving away from being perceived as a threat or a substitute for teachers.

Implications

The present study contributes to recent AI-based research in education by recognizing the effectiveness of ChatGPT in enhancing students' academic writing skills. It also provides practical insights for stakeholders to revisit educational policies and tackle the potential challenges of incorporating ChatGPT into education. The responsible use of ChatGPT is proposed, which involves leveraging teachers” and users' personal experiences to accomplish research and text-based assignments, ultimately developing students” skills and capabilities.

Limitations and recommendations for further research

The current study has some limitations that require further investigation. Firstly, it is limited to only 102 students who are enrolled in two courses in the Department of English at the University of Bisha. Therefore, future research should aim to include a larger and more diverse sample, encompassing students from different departments and universities, including postgraduates. This will provide a more comprehensive understanding of their experiences with ChatGPT. Additionally, it is important to note that ChatGPT is a relatively new tool and has not yet been widely adopted by educational institutions. Future studies with participants exhibiting greater exposure and familiarity with ChatGPT may reveal distinct perspectives and experiences compared to those observed in this study. Finally, continuous research and development of AI tools are crucial to circumvent emerging challenges and ethical concerns stemming from using AI tools in the educational landscape.

Practical recommendations

Given that ChatGPT is freely accessible in Saudi Arabia, it is recommended that students use it responsibly as a supplementary resource alongside other academic resources. This will help enhance academic integrity and ensure a well-rounded education. Students need to develop their research skills independently and critically evaluate the content generated by ChatGPT. Instead of relying heavily on ChatGPT, students should prioritize practicing and developing their writing skills inside and outside the classroom.

Stakeholders should take immediate action to raise students' awareness of the importance of upholding integrity and adhering to academic principles. Moreover, by encouraging responsible usage of ChatGPT among students, the educational advantages can be maximized and their participation in interactive learning experiences can be ensured. These actionable strategies go beyond unlocking the potential of AI technology. They empower students to analyze AI-generated content critically, promoting critical thinking, creativity, and research skills. Furthermore, familiarizing students with AI-powered tools during their academic years will equip them for future career opportunities in the ever-changing digital world.

Table 1

Participants' general information

Items of section 1 in the questionnaire	Categories	Frequency	Percentage (%)
Frequency of using ChatGPT	Always	33	32.4
	Often	27	26.5
	Sometimes	19	18.6
	Rarely	13	12.7
	Never	10	9.8
Courses in which the students were enrolled	Research Project	42	41.2
	Basics of Scientific Research in English	60	58.8

Source(s): The author’s own work

Table 2

Effectiveness of ChatGPT in enhancing academic writing skills

Items of section 2 in the questionnaire		SA	A	N	D	SD	Mean	Std. Dev.	Rank
ChatGPT assists me in brainstorming ideas for my research work	N	27	40	29	3	3	3.83	0.955	6
	%	26.5%	39.2%	28.4%	2.9%	2.9%
ChatGPT assists me in outlining my research work	N	22	53	21	3	3	3.86	0.890	5
	%	21.6%	52.0%	20.6%	2.9%	2.9%
ChatGPT generates helpful texts for my research work	N	33	51	12	3	3	4.06	0.910	1
	%	32.4%	50.0%	11.8%	2.9%	2.9%
ChatGPT enhances my writing style when writing assignments and research work	N	28	56	15	0	3	4.04	0.832	2
	%	27.5%	54.9%	14.7%	0.0%	2.9%
ChatGPT assists me in summarizing relevant literature in my selected field	N	36	42	21	0	3	4.06	0.910	1
	%	35.3%	41.2%	20.6%	0.0%	2.9%
ChatGPT assists me in paraphrasing ideas	N	22	57	17	3	3	3.90	0.873	4
	%	21.6%	55.9%	16.7%	2.9%	2.9%
ChatGPT helps me in the process of reviewing and editing my research work	N	28	50	18	3	3	3.95	0.916	3
	%	27.5%	49.0%	17.6%	2.9%	2.9%
ChatGPT enables me to remain up-to-date with the latest research trends	N	18	58	17	6	3	3.80	0.901	7
	%	17.6%	56.9%	16.7%	5.9%	2.9%
Weighted mean							3.93
Standard deviation								0.7606

Note(s): Strongly agree (SA), Agree (A), Neutral (N), Disagree (D), Strongly disagree (SD)

Source(s): The author’s own work

Table 3

Challenges of the use of ChatGPT for enhancing academic research writing

Items of section 3 of the questionnaire		SA	A	N	D	SD	Mean	Std. Dev.	Rank
ChatGPT generates inaccurate or misleading content when used in writing research projects	N	22	21	26	20	13	2.81	1.326	8
	%	21.6%	20.6%	25.5%	19.6%	12.6%
ChatGPT encourages plagiarism in my academic writing	N	19	26	24	18	15	2.84	1.326	7
	%	18.6%	25.5%	23.5%	17.6%	14.7%
ChatGPT prompts me to become overly reliant on its assistance when used in writing research projects	N	23	21	16	27	15	2.90	1.404	6
	%	22.5%	20.6%	15.7%	26.5%	14.7%
ChatGPT prompts ethical concerns when used in academic research writing	N	15	29	22	21	15	2.92	1.295	5
	%	14.7%	28.5%	21.6%	20.6%	14.7%
Using ChatGPT in academic research writing results in a decline in my creative abilities	N	6	32	25	19	20	3.15	1.230	4
	%	5.9%	31.4	24.5	18.6%	19.6%
Using ChatGPT in academic research writing prompts me to abandon my critical thinking	N	3	23	28	33	15	3.33	1.075	2
	%	2.9%	22.5%	27.5	32.4%	14.7%
Using ChatGPT in academic research writing reduces the integrity of academic writing	N	11	30	10	27	24	3.23	1.378	3
	%	10.8%	29.4%	9.8%	26.5%	23.5%
ChatGPT has the potential to substitute for a teacher in providing feedback on my research work	N	9	12	21	35	25	3.54	1.232	1
	%	8.8%	11.8%	20.6%	34.4%	24.5%
Weighted mean							3.21
Standard deviation								0.862

Note(s): Strongly agree (SA), Agree (A), Neutral (N), Disagree (D), Strongly disagree (SD)

Source(s): The author’s own work

Table 4

Effective ways to overcome the potential challenges

Items of section 4 of the questionnaire		SA	A	N	D	SD	Mean	Std. Dev.	Rank
Students should use ChatGPT as an assistant tool rather than relying on it exclusively	N	37	45	6	5	9	3.94	1.193	6
	%	36.3%	41.1%	5.9%	4.9%	8.8%
Students should use ChatGPT responsibly to enhance their academic writing skills	N	36	52	5	3	6	4.07	1.027	4
	%	35.3%	51.0%	4.9%	2.9%	5.9%
Students should critically evaluate the information provided by ChatGPT.	N	34	52	13	0	3	4.12	0.848	3
	%	33.3%	51.0%	12.7%	0.0%	2.9%
Students should be trained on how to use the ChatGPT effectively	N	52	35	9	3	3	4.27	0.956	1
	%	51.0%	34.3%	8.8%	2.9%	2.9%
Students should use ChatGPT for proofreading and editing already-written texts rather than generating texts	N	46	38	12	0	6	4.16	1.041	2
	%	45.1%	37.3%	11.8%	0.0%	5.9%
Teachers can use detective tools to revise the content generated	N	32	50	14	0	6	4.00	0.995	5
	%	31.4%	49.0%	13.7%	0.0%	5.9%
Teachers should reexamine their teaching methods and assessment techniques	N	27	39	30	3	3	3.82	0.959	7
	%	26.5%	38.2%	29.4%	2.9%	2.9%
Weighted mean							3.87
Standard deviation								0.8363

Note(s): Strongly agree (SA), Agree (A), Neutral (N), Disagree (D), Strongly disagree (SD)

Source(s): The author’s own work