Artificial intelligence (AI) and ChatGPT involvement in scientific and medical writing, a new concern for researchers. A scoping review

Purpose – The study aims to evaluate PubMed publications on ChatGPT or artificial intelligence (AI) involvement in scientific or medical writing and investigate whether ChatGPT or AI was used to create these articles or listed as authors. Design/methodology/approach – This scoping review was conducted according to Preferred Reporting ItemsforSystematicReviewsandMeta-AnalysesExtensionforScopingReviews(PRISMA-ScR)guidelines.APubMeddatabasesearchwasperformedforarticlespublishedbetweenJanuary1andNovember29,2023, usingappropriatesearchterms;bothauthorsperformedscreeningandselectionindependently. Findings – From the initial search results of 127 articles, 41 were eligible for final analysis. Articles were published in 34 journals. Editorials were the most common article type, with 15 (36.6%) articles. Authors originated from 27 countries, and authors from the USA contributed the most, with 14 (34.1%) articles. The most discussed topic was AI tools and writing capabilities in 19 (46.3%) articles. AI or ChatGPT was involved in manuscript preparation in 31 (75.6%) articles. None of the articles listed AI or ChatGPT as an author, and in 19 (46.3%) articles, the authors acknowledged utilizing AI or ChatGPT. Practical implications – Researchers worldwide are concerned with AI or ChatGPT involvement in scientific research, specifically the writing process. The authors believe that precise and mature regulations will be developed soon by journals, publishers and editors, which will pave the way for the best usage of these tools. Originality/value – This scoping review expressed data published on using AI or ChatGPT in various scientific research and writing aspects, besides alluding to the advantages, disadvantages and implications of their usage.


Introduction
The scientific research process, including data analysis, manuscript writing and publication, is essential for individual researchers and the scientific community throughout various disciplines and specialties, eventually leading to knowledge advancement and contributing to individual researchers' career development (Iskander, Wolicki, Leeb, & Siegel, 2018;Kennedy, 2018;Khalifa, 2022;Pittman, Stahre, Tomedi, & Wurster, 2017).However, this process could be exhausting, time-consuming and requiring teamwork, making researchers eager to find and develop research and writing assisting tools and applications (Pittman et al., 2017;Salvagno, Taccone, & Gerli, 2023).
Artificial intelligence (AI) has been involved in different aspects of our life for a considerable time, and scientific research is not an exception (Haenlein & Kaplan, 2019;Salvagno et al., 2023); however, starting from November 2022, the idea and usage of AI was transformed enormously after the introduction of Generative Pretrained Transformer "ChatGPT" (OpenAI, San Francisco, CA, USA), which is an open AI platform trained on a massive amount of online data (including media, books and articles), enabling this tool to perform sophisticated tasks (Gao et al., 2023;Sallam, 2023).
The primary objective of the current review was to evaluate the recent PubMed publications on ChatGPT or AI involvement in scientific or medical writing.The secondary objectives were to determine whether ChatGPT or AI was used in creating any of these articles, and, if yes, whether it was listed as an author or acknowledged by the human authors.Furthermore, the aim was to shed some light on the advantages, disadvantages and future perspectives reported in the included articles.

Methods
Design: We used the scoping review methodology according to Arksey and O'Malley (Arksey & O'Malley, 2005).The review was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines (Tricco et al., 2018) (Supplementary file, Table S1).Per Arksey and O'Malley (Arksey & O'Malley, 2005), we followed their recommended six steps for conducting the current scoping review; however, they reported that the last is optional.
Stage 1: identifying objectives This was mentioned earlier.
Stage 2: identifying relevant studies After performing a preliminary screening of the literature to obtain and formulate an idea about the search terms to be used, the two authors independently created search terms, which were discussed among the authors to agree on a final search strategy.The Title/Abstract filed of the PubMed database was searched from January 1, 2023 to November 29, 2023, using the following search strategy: ((((ChatGPT[Title/Abstract]) OR (AI[Title/Abstract])) OR (Artificial intelligence[Title/Abstract])) AND (medical writing[Title/Abstract])) OR (scientific writing[Title/Abstract]).The search was limited to articles on humans and published in the English language.
Stage 3: study selection We decided to include all articles (regardless of type) discussing AI or ChatGPT involvement in scientific writing.Furthermore, excluded articles reported on the role of AI or ChatGPT in medical education, disease diagnosis, radiograph assessment, writing medical reports and if the article was not centered on scientific writing and its related aspects.
All the results were downloaded to the Endnote program in reference form for evaluation against the predetermined inclusion and exclusion criteria.Both authors independently performed traditional screening (first by title and abstract, followed by full-text screening); any disagreement was resolved by discussion between the authors until a consensus was reached.
Stage 4: charting the data After agreeing on the finally included articles, each author independently extracted data into a predesigned Excel sheet.The following data were collected: first, study characteristics: Authors, country of origin, journal where the article was published and type of the article; second, evaluating the content of the articles and the central theme the authors discussed; and, finally, to document the role of AI or ChatGPT in creating the articles (if it was used or not?If used, was it listed as an author or not?If its use was mentioned in the acknowledgment section or not?).The data extracted by the authors were compared to resolve any discrepancies by discussion and agreement among the authors.

Stage 5: summarizing and reporting the results
As we aimed to formulate a scoping review, the collected findings were used to provide an overview of the subject in concern rather than an individual study quality assessment.The results were grouped and presented in a table.A brief qualitative assessment was performed, and a descriptive summary was reported.Both authors discussed the final results and agreed on the final presentation.
Stage 6: consultation Arksey and O'Malley suggested an optional final step, which is consultation with stakeholders and experts to provide ideas beyond what is collected from the literature; however, this step was not needed for the current review and was not performed.

Study context and design
Articles were published in 34 different journals.The Cureus journal had the highest contributions, with six (14.6%) articles, and editorials were the most common article type, with 15 (36.6%) (Figure 2).Authors originated from 27 countries, and authors from the USA contributed the most, in 14 (34.1%)articles (Figure 3).

Data synthesis
The included articles discussed various aspects of AI and ChatGPT involvement in scientific writing; however, the three main topics (themes) discussed were, first, AI tools and writing capabilities in 19 (46.3%) articles; second, the advantages and disadvantages in 12 (29.3%)articles; and, lastly, 11 (26.8%)articles discussed ethical issues and concerns.As reported by  1.

Discussion
Although various AI-assistive scientific writing tools are present and being increasingly used over the past few years (Chen, 2023;Katsnelson, 2022), it is evident that the introduction of

AI and ChatGPT: a concern for researchers
ChatGPT as a new tool involved in various aspects of scientific research and writing occurred rapidly and grasped the attention and concern of many researchers and scientists.The current review showed that articles discussing AI and ChatGPT involvement in scientific writing were published by authors from different parts of the world in various journals; furthermore, a lot of these authors themselves tried ChatGPT and its ability to formulate scientific text, besides reporting its advantages, disadvantages and the ethical concerns related to its use.
Machine learning (ML) is considered an advancement of AI, where a decision could be made using ML algorithms based on data collected from large data sets (Sallam, 2023).Furthermore, neural networks (NN) are considered a specific type of ML algorithm capable of recognizing even more complex data sets patterns, which was ultimately advanced to form the large language models (LLMs) trained on a vast amount of data enabling these models to generate content-defined text and predict the best following text elements, thus leading to the production of naturally read texts (Choi, Coyner, Kalpathy-Cramer, Chiang, & Campbell, 2020;Gao et al., 2023;Luitse & Denkena, 2021).
ChatGPT recently became one of the most popular LLMs used by personnel among different disciplines; soon, it found its way to being deeply involved in scientific research and academic writing, owing to the large amount of data it was trained on and its human-like writing capabilities (Buholayka et al., 2023;Sallam, 2023).However, there is still an ongoing debate and hassle among journals, publishers and even individual researchers regarding the optimum ChatGPT use in scientific research activities and ethical regulations controlling its involvement in the scientific writing process (Stokel-Walker, 2023;Thorp, 2023; Tools such as ChatGPT threaten transparent science; here are our ground rules for their use, 2023).
In a bibliometric analysis by Levin et al., the authors reported 42 articles published in PubMed after they used the search term "ChatGPT" on February 15, 2023 (Levin et al., 2023).At the time of preparing the current review, a new search was performed using the same search terms used by Levin et al., which revealed 1809 results, meaning that in about ten months, the publications increased by about 43 folds, indicating a plethora of increasing concerns of this topic.On the contrary, in the current review, the initial search revealed 127 articles, which is considerably few compared to the results obtained from using the search terms mentioned in the previous study; however, this could be attributed to the high selectivity of the articles using strict search terms.
Moreover, in the study by Levin et al., the most common publication type was editorials, constituting 52%, and authors from the UK contributed the most (33%).In the current review, we reported the same results regarding the article type where editorials represent the commonly published article type (36.6%); however, authors from the USA were the highest contributors (34.1%).The predominance of editorials, which are mostly written by eminent researchers, reflects the concern regarding this considerably new phenomenon; furthermore, contributing authors are from countries across the world, indicating the global interest in evaluating this issue.
The articles included in the current review handled various themes and topics, showing the authors' interest in discussing and evaluating various aspects of AI or ChatGPT involvement in scientific writing; furthermore, in some articles, more than one theme was discussed (Table 1).The most commonly evaluated issue was the role of AI or ChatGPT as an assistant during different aspects of manuscript writing (language corrections, especially for non-English speakers, abstract formulation, building a structured manuscript and obtaining related references).Then, various articles reported the advantages and disadvantages of these tools' involvement in the scientific writing process, and few provided some recommendations and future perspectives for their use.Lastly, the ethical issues and increasing concerns regarding the misuse of these tools were raised in some articles (Table 2).

I Advantages
1 Making a linguistically coherent text from bullet points or short notes collected from different references while performing a literature review 2 Sorting and managing references 3 Performing study design, analysis of the study results and drafting them into text, which is thought to help writers overcome "Blank-sheet syndrome." 4 ChatGPT, with the supervision of a human author, significantly shortened the manuscript writing time, reaching up to one hour for a complete manuscript draft 5 It could help with different manuscript preparation steps, including formatting, linguistic correction and translation for non-English-speaking authors 6 It could rapidly analyze and summarize data from many articles, which is helpful while performing a literature search and review 7 Overall, it helps improve the writing quality by detecting areas of inconsistencies and potential errors 8 Plagiarism detection could be performed using these tools II Disadvantages and worries 1 Text generated by ChatGPT could be a mix of accurate and fabricated data 2 Sometimes, the references suggested by ChatGPT are either wrong or non-existent 3 The text generated to answer a particular question is not necessarily accurate, and the answer could change if the question were asked multiple times 4 Controversy regarding the intellectual property rights if the text was generated using ChatGPT, do they belong to the human authors or the algorithm?5 ChatGPT does not necessarily follow the journal instructions if asked to formulate a manuscript according to specific journal authors' instructions 6 It is worth noting that OpenAI incorporated a disclaimer in all chat windows that AI or ChatGPT could offer incorrect and biased information 7 Possible unethical use of these tools (however, still not proven), such as its use to produce vast amounts of abstracts by organizations such as paper mills or even the production of fabricated research articles 8 AI assisting tools could help researchers to increase their research production; however, this comes without an actual increase in their real experience.This could make publication numbers less trustworthy as a parameter for promotion or hiring academics 9 If fabrication or plagiarism were detected within a text or manuscript generated with the assistance of AI tools, who would bear the responsibility and the consequences?
III Future perspectives 1 Changing the regulations and policies needed for evaluating scientific manuscripts submitted to journals and abstracts submitted for conferences, including using AIgenerated text detectors 2 AI could autonomously publish different manuscript types (commentaries, editorials and reviews) 3 Implementation of new ethical roles, including clear disclosure regarding using AI-assistive tools during preparing manuscripts and at which stage it was used.Furthermore, journals should also disclose their use of AI-generated text detection tools 4 Other ethical issues that are not yet resolved, such as whether ChatGPT or any other AI-based tools should be listed as an author or not; furthermore, these tools are currently free of charge, and these could soon be paid tools, which could further increase the gap between researchers from high-and low-income countries as well as between senior and junior researchers 5 AI-detector tools investment will flourish, the same as what happened with plagiarism checkers (continued )

AI and ChatGPT: a concern for researchers
IV Theoretical and practical implications (based on the current review results) 1 For researchers -Judicious utilization of AI-related tools in various aspects of scientific writing (such as translation and editing) could be beneficial -Preserving the research community integrity is the responsibility of each researcher -Honest reporting of using these tools during manuscript preparation is mandatory 2 For decision-makers in higher education institutions and medical research -Institution-specific regulations in the light of the reported advantages, disadvantages and ethical issues should be developed and implemented to regulate the use of these tools during various stages of scientific research -Educational curriculums could be introduced to inform researchers of the benefits and harms of using these tools.Furthermore, it advises on the best practices and the trustable tools to be used -Using these tools during critical thinking for proper decision-making regarding healthcare-related decisions -Instead of ignoring or prohibiting these tools altogether, there should be beneficial controlled introduction and involvement during various processes of education, training, managing patients (adjusting drug doses, remote monitoring, symptoms development checks) and improving medical research (help in designing studies, recruiting patients and documentation) Source(s):

AGJSR
The current review has some limitations.First, the search was performed in one database, which could lead to missing some articles published in other databases.Second, using restricted search terms and focusing on a specific research point led to excluding articles discussing other aspects of scientific research rather than manuscript writing and preparation.Third, a full bibliometric analysis (including the number of citations of each article) was not performed.Fourth, as this was a scoping review focusing on a few points, only qualitative data synthesis and descriptive presentation were performed.Further, more detailed systematic reviews, including quantitative analysis (metanalysis), could be performed to provide more insight into AI and ChatGPT's role in scientific research and overcome the current review's drawbacks.Last, we could not offer recommendations for the proper utilization of AI or ChatGPT; however, we believe the current review could work as a foundation for formulating solid recommendations and regulations based on the data we reported.

Conclusion
Interests and worries regarding using AI or ChatGPT in different aspects of scientific research, including specifically the writing process, pose an increasing concern among researchers in different specialties.It is too early to judge these new tools; however, it is apparent that these are here to stay, and the main burden is on the shoulders of individual researchers and their moral and ethical consciousness to control their judicious use of these tools and applications.We believe that precise and mature regulations will be developed soon by journals, publishers and editors, which will pave the way for the best usage of these tools.As Ridley Scott said, "Life is not black and white.It is a million gray areas, don't you find?" there is no absolute white (right) or black (wrong) regarding AI or ChatGPT involvement and usage in scientific research and writing; however, we are waiting for millions of gray areas, which possibly will appear shortly.

Fig. 1 .
Fig. 1.PRISMA chart of the search strategy

Fig
Fig. 2. Chart indicating the number of article types

Table 1 .
Details N: no, Y: yes, NA: not applicable Source(s): Table by authors Table 1.

Table 2 .
Table by authors