Thinking critically about scientific information generated by ChatGPT
Information and Learning Sciences
ISSN: 2398-5348
Article publication date: 27 August 2024
Issue publication date: 19 November 2024
Abstract
Purpose
In November 2022, the commercial company, OpenAI, launched ChatGPT. Since then, university students have rapidly become regular users of this artificial intelligence (AI) platform. One reason for this is the powerful capability of this generative AI tool to produce textual content, which in many cases, is almost indistinguishable from human-generated content. Another reason is that ChatGPT easily gives anyone access to knowledge. However, there is a problem as the vast majority of its users have no idea how this AI platform works and thus overlook the importance of thinking critically about information communicated in ChatGPT. While some call for banning this generative AI tool, this study aims to provide evidence that science classrooms can become scenarios where students find explicit, concrete, and realistic opportunities to critically evaluate scientific information generated by ChatGPT.
Design/methodology/approach
An intervention study was conducted with 55 students (26 females and 29 males, 17–24 years old) during a university Spanish-English bilingual science course taught within an active learning environment. The data consist of the written critiques of the students about Spanish-English bilingual scientific texts produced by ChatGPT.
Findings
Results indicate that the intervention had a positive effect on students’ abilities to construct sound arguments in Spanish and in English while judging the quality of scientific texts produced by this AI bot. Moreover, the findings suggest that the intervention enriched students’ skills to make improvements to texts produced by this generative AI tool.
Originality/value
The evidence provided in this study contributes to the exploration of possibilities to help students become critical users of ChatGPT.
Keywords
Acknowledgements
The authors wish to thank the undergraduate students for their time and participation in this study, as well as to acknowledge financial support from the Vice-Presidency of Research and Creation, Universidad de los Andes, Bogotá, Colombia.
Citation
Archila, P.A., Ortiz, B.T., Truscott de Mejía, A.-M. and Molina, J. (2024), "Thinking critically about scientific information generated by ChatGPT", Information and Learning Sciences, Vol. 125 No. 11/12, pp. 1074-1106. https://doi.org/10.1108/ILS-04-2024-0040
Publisher
:Emerald Publishing Limited
Copyright © 2024, Emerald Publishing Limited