Multiple choice questions: answering correctly and knowing the answer

Peter McKenna (Department of Computing, Manchester Metropolitan University, Manchester, UK and Department of Computing and Digital Technology, Manchester Metropolitan University, Manchester, UK)

Interactive Technology and Smart Education

ISSN: 1741-5659

Publication date: 11 March 2019

Abstract

Purpose

This paper aims to examine whether multiple choice questions (MCQs) can be answered correctly without knowing the answer and whether constructed response questions (CRQs) offer more reliable assessment.

Design/methodology/approach

The paper presents a critical review of existing research on MCQs, then reports on an experimental study where two objective tests (using MCQs and CRQs) were set for an introductory undergraduate course. To maximise completion, tests were kept short; consequently, differences between individuals’ scores across both tests are examined rather than overall averages and pass rates.

Findings

Most students who excelled in the MCQ test did not do so in the CRQ test. Students could do well without necessarily understanding the principles being tested.

Research limitations/implications

Conclusions are limited by the small number of questions in each test and by delivery of the tests at different times. This meant that statistical average data would be too coarse to use, and that some students took one test but not the other. Conclusions concerning CRQs are limited to disciplines where numerical answers or short and constrained text answers are appropriate.

Practical implications

MCQs, while useful in formative assessment, are best avoided for summative assessments. Where appropriate, CRQs should be used instead.

Social implications

MCQs are commonplace as summative assessments in education and training. Increasing the use of CRQs in place of MCQs should increase the reliability of tests, including those administered in safety-critical areas.

Originality/value

While others have recommended that MCQs should not be used (Hinchliffe 2014, Srivastava et al., 2004) because they are vulnerable to guessing, this paper presents an experimental study designed to demonstrate whether this hypothesis is correct.

Keywords

Citation

McKenna, P. (2019), "Multiple choice questions: answering correctly and knowing the answer", Interactive Technology and Smart Education, Vol. 16 No. 1, pp. 59-73. https://doi.org/10.1108/ITSE-09-2018-0071

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Emerald Publishing Limited

Please note you might not have access to this content

You may be able to access this content by login via Shibboleth, Open Athens or with your Emerald account.
If you would like to contact us about accessing this content, click the button and fill out the form.
To rent this content from Deepdyve, please click the button.