The effect of insufficient effort responding on the validity of student evaluation of teaching

Mahmoud AlQuraan (Department of Counseling and Educational Psychology, Yarmouk University, Irbid, Jordan)

Journal of Applied Research in Higher Education

ISSN: 2050-7003

Publication date: 1 July 2019

Abstract

Purpose

The purpose of this paper is to investigate the effect of insufficient effort responding (IER) on construct validity of student evaluations of teaching (SET) in higher education.

Design/methodology/approach

A total of 13,340 SET surveys collected by a major Jordanian university to assess teaching effectiveness were analyzed in this study. The detection method was used to detect IER, and the construct (factorial) validity was assessed using confirmatory factor analysis (CFA) and principal component analysis (PCA) before and after removing detected IER.

Findings

The results of this study show that 2,160 SET surveys were flagged as insufficient effort responses out of 13,340 surveys. This figure represents 16.2 percent of the sample. Moreover, the results of CFA and PCA show that removing detected IER statistically enhanced the construct (factorial) validity of the SET survey.

Research limitations/implications

Since IER responses are often ignored by researchers and practitioners in industrial and organizational psychology (Liu et al., 2013), the results of this study strongly suggest that higher education administrations should give the necessary attention to IER responses, as SET results are used in making critical decisions

Practical implications

The results of the current study recommend universities to carefully design online SET surveys, and provide the students with clear instructions in order to minimize students’ engagement in IER. Moreover, since SET results are used in making critical decisions, higher education administrations should give the necessary attention to IER by examining the IERs rate in their data sets and its consequences on the data quality.

Originality/value

Reviewing the related literature shows that this is the first study that investigates the effect of IER on construct validity of SET in higher education using an IRT-based detection method.

Keywords

Citation

AlQuraan, M. (2019), "The effect of insufficient effort responding on the validity of student evaluation of teaching", Journal of Applied Research in Higher Education, Vol. 11 No. 3, pp. 604-615. https://doi.org/10.1108/JARHE-03-2018-0034

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Emerald Publishing Limited

To read the full version of this content please select one of the options below

You may be able to access this content by logging in via Shibboleth, Open Athens or with your Emerald account.
To rent this content from Deepdyve, please click the button.
If you think you should have access to this content, click the button to contact our support team.