Exploring cognitive bias: expert ratings of business schools
Abstract
Purpose
This study aims to explore the objectivity in third-party ratings. Third-party ratings are often based on some form of aggregation of various experts' opinions with the assumption that the potential judgment biases of the experts cancel each other out. While psychology research has suggested that experts can be unintentionally biased, management literature has not considered the effect of expert bias on the objectivity of third-party ratings. Thus, this study seeks to address this issue.
Design/methodology/approach
Ranking data from the US News and World Report between 1993 and 2008, institution-related variables and, to represent sports prominence, NCAA football and basketball performance variables are leveraged in testing our hypotheses. A mediating-model is tested using regression with panel-corrected standard errors.
Findings
This study finds that the judgments of academicians and recruiters, concerning the quality of universities, have been biased by the prominence of a university's sports teams and that the bias introduced to these experts mediates the aggregated bias in the resultant rankings of MBA programs. Moreover, it finds that experts may inflate rankings by up to two positions.
Practical implications
This study is particularly relevant for university officials as it uncovers how universities can tangibly manipulate the relative perception of quality through sports team prominence. For third-party rating systems, the reliability of ratings based on aggregated expert judgments is called into question.
Originality/value
This study addresses a significant gap in the literature by examining how a rating system may be unintentionally biased through the aggregation of experts' judgments. Given the heavy reliance on third-party rating systems by both academics and the general population, addressing the objectivity of such ratings is crucial.
Keywords
Acknowledgements
The authors thank Talya Bauer, Russell Coff, Gregory Dess, Berrin Erdogan, Jerry Goodstein, J. Richard Harrison, Michael Lounsbury, Madan Pillutla, Jing Martina Quan, Marvin Washington, editor Domingo Soriano, and two anonymous reviewers for their helpful comments and feedback on this research. Also thanks to John Mills for research assistance. An earlier version of this draft was presented at the 2010 Academy of Management Conference.
Citation
Pleggenkuhle-Miles, E., A. Khoury, T., L. Deeds, D. and Markoczy, L. (2013), "Exploring cognitive bias: expert ratings of business schools", Management Decision, Vol. 51 No. 9, pp. 1905-1927. https://doi.org/10.1108/MD-12-2012-0877
Publisher
:Emerald Group Publishing Limited
Copyright © 2013, Emerald Group Publishing Limited