Understanding validity issues in test-based models of school and teacher evaluation

Abstract

Purpose

This policy brief, the second AERI-NEPC eBrief in the series “Understanding validity issues around the world”, focuses on validity as it applies to test-based models of evaluation employed for schools, instructional programs, and teachers around the world. It discusses validity issues that could arise when data from student achievement test administrations and other sources are used for conducting personnel appraisals, program evaluations, or for external accountability purposes, suggesting solutions and recommendations for improving validity in such applications of test-based information.

Design/methodology/approach

This policy brief is based on a synthesis of conference proceedings and review of selected pieces of extant literature. It begins by summarizing perspectives of an invited expert panel on the topic. To that synthesis, the authors add their own analysis of key issues. They conclude by offering recommendations for test developers and test users.

Findings

The authors conclude that systematic improvement and transformation of schools depends on thoughtfully conceptualizing, implementing, and using data from testing and broad-based evaluation systems that incorporate multiple kinds of evidence. Evaluation systems that are valid and fair to students, teachers and education leaders need all three of the following: assessment resources and training for all participants and evaluation users; knowledgeable staff to continuously monitor processes and use assessment results appropriately to improve teaching and learning activities; and a strengths-based approach to make improvements to the education system based on relevant data and reports (as opposed to a deficits-based one in which blame or punishment is leveled at individuals or groups of workers when gaps in performance are observed).

Originality/value

To improve validity in interpretations of results from test-based teacher and school evaluation models, the authors provide recommendations for measurement and evaluation specialists as well as for educators, policy makers, and public users of data. Standardized test use in formative and more “high stakes” educational accountability contexts is rapidly spreading to various regions of the world. This eBrief shows that understandings of validity are still uneven among key stakeholders. By translating complex information pertinent to current validity issues, this policy brief attempts to address this need, and also bridge knowledge and communications gaps among different constituencies.

Citation

L. Bridglall, B., Caines, J. and Chatterji, M. (2014), "Understanding validity issues in test-based models of school and teacher evaluation", Quality Assurance in Education, Vol. 22 No. 1, pp. 19-30. https://doi.org/10.1108/QAE-12-2013-0053