Search results

University rankings and metrics have become an increasingly prominent basis of student decisions, generalized university reputation, and the resources university’s attract. We review the history of metrics in higher education and scholarship about the influence of ranking on the position and strategic behavior of universities and students. Most quantitative analyses on this topic estimate the influence of change in university rank on performance. These studies consistently identify a small, short-lived influence of rank shift on selectivity (e.g., one rank position corresponds to ≤1% more student applicants), comparable to ranking effects documented in other domains. This understates the larger system-level impact of metrification on universities, students, and the professions that surround them. We explore one system-level transformation likely influenced by the rise of rankings. Recent years have witnessed the rise of enrollment management and independent educational consultation. We illustrate a plausible pathway from ranking to this transformation: In an effort to improve rankings, universities solicit more applications from students to reduce their acceptance rate. Lower acceptance rates lead to more uncertainty for students about acceptance, leading them to apply to more schools, which decreases the probability that accepted students will attend. This leads to greater uncertainty about enrollment for students and universities and generates demand for new services to manage it. Because these and other system-level transformations are not as cleanly measured as rank position and performance, they have not received the same treatment or modeling attention in higher education scholarship, despite their importance for understanding and influencing education policy.

This chapter presents findings from a recently conducted process for obtaining Accounting Advisory Board (AAB) input related to Master of Accountancy curriculum of one university. Board members represent both large and small public accounting firms as well as corporate offices of Fortune 500 companies and non-profit organizations. AAB input includes perceptions of the relative importance of over 160 candidate topics for the courses making up the program’s infrastructure, as well as written comments noting other potential topics and pedagogical approaches to consider. Comparisons of topic rankings reveal a strong level of consistency among Board member types for the traditional accounting courses with structured content, as opposed to those courses involving more systems-related topics or having a wider range of specialized topics. Furthermore, the authors compare Board perceptions regarding topic necessity to those of faculty and note faculty reactions. Specifically, the authors find that faculty ranking consistency with the Board is weak, illustrating the importance of seeking curricular Board input on an ongoing basis. To “close the loop,” faculty incorporated many curriculum changes, involving both the topics to be covered and the overall approach to the course.

Through the example of a “regulatory ranking” – an index produced with the aim to regulate the pharmaceutical market by pushing companies in the direction of providing greater access to medicine in developing countries – this chapter focuses on indexing and ranking as infrastructural processes which inscribe global problem spaces as unfolding actionable territories for market intervention. It foregrounds the “Indexal thinking” which structures and informs regulatory rankings – their aspiration to align the interests of different stakeholders and to entice competition among the ranked companies. The authors detail the infrastructural work through which such ambitions are enacted, detailing processes of infrastructural layering/collage and patchwork through which analysts naturalize/denaturalize various contested categories in the ranking’s territory. They reflect on the consequences of such attempts at reconfiguring global topologies for the problems these governance initiatives seek to address.

Rankings have become a popular topic in the social sciences over the past two decades. Adding to these debates, the present volume assembles studies that explore a variety of empirical settings, emphasizing the importance of acknowledging that there are multiple “Worlds of Rankings.” To this end, the first part of the chapter addresses the implications of two modes of criticism that characterize much of the scholarly work on rankings and summarizes extant conceptual debates. Taking stock of what we know, the second part distinguishes three areas of empirical research. The first area concerns the activities of those who produce rankings, such as the collection of data or different business strategies. Studies in the second area focus on inter-organizational, field-level, or discursive phenomena, particularly how rankings are received, interpreted, and institutionalized. The third area covers the manifold effects that research has unveiled, ranging from the diffusion of practices and changes in organizational identities to emotional distress. Taken together, the contributions to this volume expand our knowledge in all three areas, inviting new debates and suggesting pathways forward.

Rankings are widely regarded as particularly well-suited for capturing the public eye, which is considered a reason why they have become ubiquitous. However, we know little about how rankings direct media attention, as well as how media in turn shape and help sustain careers of specific rankings in the public over longer periods of time. To advance our understanding of the discursive dynamics at the intersection of rankings and the press, this study examines the media career of the Global Slavery Index (GSI) by analyzing 361 newspaper and magazine articles, published between the release of index’s inaugural edition in 2013 and until the end of 2019. To interpret the media coverage, the study draws attention to GSI’s universality, highly rationalized character, and a pledge to spotlight violation of the global moral order. The examination of the media coverage points to the following properties of the index as having shaped and helped sustain its career in the public: (1) repeated publication; (2) broad conceptualization of modern slavery; and (3) the construction thereof as a measurable global burden. The study finds that, throughout the period, the media were remarkably consistent in amplifying the most dramatic elements of the index. Over time, however, the index was increasingly more invoked for other purposes, usually either to lend credibility to a story or as a way of embedding local and situational concerns into global narratives.

Extant research conceptualizes rankings as measures that fundamentally shape modern life by virtue of being publicly available. Yet, studies seldom explore the act of publishing when accounting for the attention rankings raise in larger stakeholder audiences. In short, we know a lot about the impact of rankings, but considerably less about the organizational practices devised by those who produce them – the rankers. Borrowing from Goffman, the paper conceptualizes modern rankings as public performances carefully prepared on backstages and unfolding on multiple frontstages. Using a qualitative data set, the paper traces the full spectrum of organizational practices that make rankings public performances: on the backstage, launch dates have to be set, numbers packaged, and “teams” prepared; on the frontstage, performances are held through face-to-face interactions (at launch events) as well as in a variety of mediated settings. Overall, the findings indicate that the more ranking organizations seek the attention of larger stakeholder audiences, the more the publication process is transformed into what one of the informants describes as “a big firework.”

The thesis that rankings do more than just make visible an organization’s position viz-á-viz a competitor, but stimulate new competitive rivalries, has provoked much interest. Yet, to date, scholars lack an understanding of how such competitive rivalries unfold at the level of organizational strategy. Put simply, if competition is played out in rankings, how does this change the way organizations strategize? We answer this question through an ethnographic study of how information technology organizations engage with rankings. The strategic responses we observed included “leapfrogging a rival,” “de-positioning a competitor,” “owning a market,” and “encouraging a breakout,” which together are theorized as “ranking strategy.” This novel conceptualization extends understanding of the organizational response to rankings by showing how common reactions like gaming are only the tip of the iceberg of a broader array of strategic responses. The study also throws light on the different ways a ranking can pattern competitive rivalries, including creating more episodic forms of rivalry.

Much has been analyzed regarding the origins and the impact of rankings and metrics on policies, behaviors, and missions of universities. Surprisingly, little attention has been allocated to describing and analyzing the emergence of metrics as a new action field. This industry, fueled by the “new public management” policy perspectives that operate at the backstage of the contemporary pervasive “regime of excellence,” still remains a black box worth exploring in depth. This paper intends to fill this loophole. It first sets the stage for this new action field by stressing the differences between the policy fields of higher education in the United States and Europe, as a way to understand the specificities of the use of metrics and rankings on both continents. The second part describes the actors of the field, which productive organizations they build, what skills they combine, which products they put on the market, and their shared norms and audiences.

Ranking groups (schools, regions, counties) according to the average score of their constituent parts – say, ranking schools by the academic achievements of students – is a common yardstick in evaluation and a cornerstone of any planning process. In this paper we show that under certain circumstances the ranking of groups, unlike the ranking of individuals, can be affected by the examiner. The fact that a ranking reversal is possible does not necessarily imply that the test itself is deficient, it merely reveals that the groups are non-homogeneous and therefore, the ranking of groups is meaningless. An investigation of the conditions under which such manipulation can occur leads us to suggest new statistical indicators as warning signals that can help one recognize such situations if and when they arise. It turns out that these indicators are related to the indicators for finding whether one distribution dominates another.

In 2009, Newsweek published a report in which they ranked the 500 largest US companies and the 100 largest global companies based on its environmental performance measures (http://greenrankings2009.newsweek.com/). This ranking is referred to as Newsweek’s Green Ranking. Included in this ranking is information about water and air pollution, solid waste disposal, toxic wastes, carbon emissions, and enforcement actions. The question we are addressing in this study is how well it measures pollution performance? The question is relevant to environmental accounting/reporting since it is part of a dilemma yet to be answered: Aggregated environmental indices/scores are easy for average information users to percept, while specific information may not be preserved when it is aggregated into the overall score(s).

Specifically, we examine whether Newsweek’s Green Ranking is correlated with pollution measures based on Toxics Release Inventory (TRI) in order to determine how valid or reliable Newsweek’s Green Ranking is – in other words, how much Newsweek’s Green Ranking can explain the pollution by the toxic releases. We find that there is no significant correlation between Newsweek’s Green Ranking and the TRI measures except for the firms in the utilities industry. Concluding that on one measure, which we consider a very important one, there is no justification for the overall Green Ranking Score presented by Newsweek. However, in Newsweek’s three-part score the element that is termed the Environmental Impact Score captures pollution performance measured based on TRI. The contrast between the overall ranking and performance ranking indicates that a composite index that incorporates hard performance and soft measures can dilute the information carried by performance data.