Utilizing a realist evaluative research approach to investigate complex technology implementations: An e-learning lecture capture exemplar

Abstract

Purpose

This study aims to investigate the effectiveness of a theory-driven realist evaluative research approach to better understand complex technology implementations in organizations.

Design/methodology/approach

An institution wide e-learning implementation of lecture capture (LC), within a UK University, was chosen, and a realist evaluation framework was used, tailored for educational technology. The research was conducted over four, increasingly focused, evaluation cycles combining engagement analytics, user interviews and theory to refine what works (or does not work), for whom, in which contexts and why.

Findings

Despite explicit demand and corresponding investment, overall student engagement is lower than expected. Increased student use appears linked to particular staff attitudes and behaviours and not to specific disciplines or course content. The main benefits of LC are providing reassurance to the majority, aiding revision and understanding for the many and enabling catch-up for the few. Recommendations for future research are based on some unexpected outcomes uncovered, including evolving detrimental student behaviours, policy development based on technological determinism and future learner-centred system development for next-generation LC technologies.

Practical implications

The realist approach taken, and evaluation framework used, can be adopted (and adapted) for future evaluative research. Domain specific reference models, categorizing people and technology, supported analysis across multiple contexts.

Originality/value

This study responds to a call for more theory-based research in the field of educational technology. The authors demonstrate that a theory-driven approach provides real and practical recommendations for institutions and allows for greater insight into the political, economic and social complexity of technology implementation.

Citation

King, M.R.N., Dawson, R.J., Rothberg, S.J. and Batmaz, F. (2017), "Utilizing a realist evaluative research approach to investigate complex technology implementations: An e-learning lecture capture exemplar", Journal of Systems and Information Technology, Vol. 19 No. 1/2, pp. 22-41. https://doi.org/10.1108/JSIT-04-2017-0027