Utilizing a realist evaluative research approach to investigate complex technology implementations: An e-learning lecture capture exemplar
Journal of Systems and Information Technology
ISSN: 1328-7265
Article publication date: 13 March 2017
Abstract
Purpose
This study aims to investigate the effectiveness of a theory-driven realist evaluative research approach to better understand complex technology implementations in organizations.
Design/methodology/approach
An institution wide e-learning implementation of lecture capture (LC), within a UK University, was chosen, and a realist evaluation framework was used, tailored for educational technology. The research was conducted over four, increasingly focused, evaluation cycles combining engagement analytics, user interviews and theory to refine what works (or does not work), for whom, in which contexts and why.
Findings
Despite explicit demand and corresponding investment, overall student engagement is lower than expected. Increased student use appears linked to particular staff attitudes and behaviours and not to specific disciplines or course content. The main benefits of LC are providing reassurance to the majority, aiding revision and understanding for the many and enabling catch-up for the few. Recommendations for future research are based on some unexpected outcomes uncovered, including evolving detrimental student behaviours, policy development based on technological determinism and future learner-centred system development for next-generation LC technologies.
Practical implications
The realist approach taken, and evaluation framework used, can be adopted (and adapted) for future evaluative research. Domain specific reference models, categorizing people and technology, supported analysis across multiple contexts.
Originality/value
This study responds to a call for more theory-based research in the field of educational technology. The authors demonstrate that a theory-driven approach provides real and practical recommendations for institutions and allows for greater insight into the political, economic and social complexity of technology implementation.
Keywords
Citation
King, M.R.N., Dawson, R.J., Rothberg, S.J. and Batmaz, F. (2017), "Utilizing a realist evaluative research approach to investigate complex technology implementations: An e-learning lecture capture exemplar", Journal of Systems and Information Technology, Vol. 19 No. 1/2, pp. 22-41. https://doi.org/10.1108/JSIT-04-2017-0027
Publisher
:Emerald Publishing Limited
Copyright © 2017, Emerald Publishing Limited