The role of cognitive validity testing in the development of CALL, the comprehensive assessment of leadership for learning
Abstract
Purpose
The Comprehensive Assessment of Leadership for Learning (CALL) is an online task-based assessment of distributed instructional leadership. In developing CALL, researchers faced the challenge of structuring survey items that would measure leadership practice rather than individual traits. Critical in this work was developing items that accurately reflected current leadership practice. Therefore, the purpose of this paper is to pilot the CALL instrument and conducted cognitive validity testing on the instrument.
Design/methodology/approach
CALL researchers piloted the survey in six schools in Wisconsin in order to test and refine the survey instrument. Researchers conducted cognitive walk-through interviews with five participants from each school: principals, associate principals, teachers, department chairs, guidance counselors, and activities directors. The interviews focussed on specific items in order to observe the users’ thought processes and rationale in choosing a response for each item. The researchers focussed on relevancy, clarity, and accuracy of survey items in collecting and analyzing the resulting data.
Findings
Three specific survey items were identified that exemplify these challenges and opportunities such as: accessible language, extended leadership, socially desirable responding, 360-degree perspectives, applying appropriate terminology, and identifying appropriate practices. These findings provided insight into survey development work and implications of distributed leadership. The authors discuss the challenges of creating a task-based leadership assessment.
Originality/value
Developing a formative assessment of school leadership is valuable in supporting school leaders’ work. The process of utilizing a qualitative approach to develop a quantitative instrument has proven critical in measuring task-based distributed leadership.
Keywords
Acknowledgements
The research reported was supported by the US Department of Education Institute of Education Sciences (Award R305A090265) and by the Wisconsin Center for Education Research, School of Education, University of Wisconsin-Madison. Any opinions, findings, or conclusions expressed in this paper are those of the author and do not necessarily reflect the views of the funding agencies, WCER, or cooperating institutions.
Citation
H. Blitz, M., Salisbury, J. and Kelley, C. (2014), "The role of cognitive validity testing in the development of CALL, the comprehensive assessment of leadership for learning", Journal of Educational Administration, Vol. 52 No. 3, pp. 358-378. https://doi.org/10.1108/JEA-01-2013-0008
Publisher
:Emerald Group Publishing Limited
Copyright © 2014, Emerald Group Publishing Limited