Rethinking Reproducibility as a Criterion for Research Quality
Including a Symposium on Mary Morgan: Curiosity, Imagination, and Surprise
ISBN: 978-1-78756-424-4, eISBN: 978-1-78756-423-7
Publication date: 24 October 2018
A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a “crisis of reproducibility” to the rise of fraudulent, careless, and unreliable practices of knowledge production. Through the analysis of discourse and practices across research fields, I point out that reproducibility is not only interpreted in different ways, but also serves a variety of epistemic functions depending on the research at hand. Given such variation, I argue that the uncritical pursuit of reproducibility as an overarching epistemic value is misleading and potentially damaging to scientific advancement. Requirements for reproducibility, however they are interpreted, are one of many available means to secure reliable research outcomes. Furthermore, there are cases where the focus on enhancing reproducibility turns out not to foster high-quality research. Scientific communities and Open Science advocates should learn from inferential reasoning from irreproducible data, and promote incentives for all researchers to explicitly and publicly discuss (1) their methodological commitments, (2) the ways in which they learn from mistakes and problems in everyday practice, and (3) the strategies they use to choose which research components of any project need to be preserved in the long term, and how.
This chapter – like much of my work over the last 10 years – owes much to Mary Morgan’s generous advice, and her ability to identify and probe the core and significance of arguments and questions. I also gratefully acknowledge insightful comments by Hans Radder, Stephan Güttinger, Niccolo Tempini, two anonymous referees, and participants to the conference “Curiosity, Imagination and Surprise” (Utrecht, September 2017), particularly the host Marcel Boumans. This research was funded by the European Research Council grant award 335925 (“The Epistemology of Data-Intensive Science”), the Australian Research Council Discovery Project “Organisms and Us” and the UK Economic and Social Research Council award ES/P011489/1.
Leonelli, S. (2018), "Rethinking Reproducibility as a Criterion for Research Quality", Including a Symposium on Mary Morgan: Curiosity, Imagination, and Surprise (Research in the History of Economic Thought and Methodology, Vol. 36B), Emerald Publishing Limited, Bingley, pp. 129-146. https://doi.org/10.1108/S0743-41542018000036B009
Emerald Publishing Limited
Copyright © 2018 Emerald Publishing Limited