Search results

Rankings have become a popular topic in the social sciences over the past two decades. Adding to these debates, the present volume assembles studies that explore a variety of empirical settings, emphasizing the importance of acknowledging that there are multiple “Worlds of Rankings.” To this end, the first part of the chapter addresses the implications of two modes of criticism that characterize much of the scholarly work on rankings and summarizes extant conceptual debates. Taking stock of what we know, the second part distinguishes three areas of empirical research. The first area concerns the activities of those who produce rankings, such as the collection of data or different business strategies. Studies in the second area focus on inter-organizational, field-level, or discursive phenomena, particularly how rankings are received, interpreted, and institutionalized. The third area covers the manifold effects that research has unveiled, ranging from the diffusion of practices and changes in organizational identities to emotional distress. Taken together, the contributions to this volume expand our knowledge in all three areas, inviting new debates and suggesting pathways forward.

Rankings are widely regarded as particularly well-suited for capturing the public eye, which is considered a reason why they have become ubiquitous. However, we know little about how rankings direct media attention, as well as how media in turn shape and help sustain careers of specific rankings in the public over longer periods of time. To advance our understanding of the discursive dynamics at the intersection of rankings and the press, this study examines the media career of the Global Slavery Index (GSI) by analyzing 361 newspaper and magazine articles, published between the release of index’s inaugural edition in 2013 and until the end of 2019. To interpret the media coverage, the study draws attention to GSI’s universality, highly rationalized character, and a pledge to spotlight violation of the global moral order. The examination of the media coverage points to the following properties of the index as having shaped and helped sustain its career in the public: (1) repeated publication; (2) broad conceptualization of modern slavery; and (3) the construction thereof as a measurable global burden. The study finds that, throughout the period, the media were remarkably consistent in amplifying the most dramatic elements of the index. Over time, however, the index was increasingly more invoked for other purposes, usually either to lend credibility to a story or as a way of embedding local and situational concerns into global narratives.

Extant research conceptualizes rankings as measures that fundamentally shape modern life by virtue of being publicly available. Yet, studies seldom explore the act of publishing when accounting for the attention rankings raise in larger stakeholder audiences. In short, we know a lot about the impact of rankings, but considerably less about the organizational practices devised by those who produce them – the rankers. Borrowing from Goffman, the paper conceptualizes modern rankings as public performances carefully prepared on backstages and unfolding on multiple frontstages. Using a qualitative data set, the paper traces the full spectrum of organizational practices that make rankings public performances: on the backstage, launch dates have to be set, numbers packaged, and “teams” prepared; on the frontstage, performances are held through face-to-face interactions (at launch events) as well as in a variety of mediated settings. Overall, the findings indicate that the more ranking organizations seek the attention of larger stakeholder audiences, the more the publication process is transformed into what one of the informants describes as “a big firework.”

The thesis that rankings do more than just make visible an organization’s position viz-á-viz a competitor, but stimulate new competitive rivalries, has provoked much interest. Yet, to date, scholars lack an understanding of how such competitive rivalries unfold at the level of organizational strategy. Put simply, if competition is played out in rankings, how does this change the way organizations strategize? We answer this question through an ethnographic study of how information technology organizations engage with rankings. The strategic responses we observed included “leapfrogging a rival,” “de-positioning a competitor,” “owning a market,” and “encouraging a breakout,” which together are theorized as “ranking strategy.” This novel conceptualization extends understanding of the organizational response to rankings by showing how common reactions like gaming are only the tip of the iceberg of a broader array of strategic responses. The study also throws light on the different ways a ranking can pattern competitive rivalries, including creating more episodic forms of rivalry.

Much has been analyzed regarding the origins and the impact of rankings and metrics on policies, behaviors, and missions of universities. Surprisingly, little attention has been allocated to describing and analyzing the emergence of metrics as a new action field. This industry, fueled by the “new public management” policy perspectives that operate at the backstage of the contemporary pervasive “regime of excellence,” still remains a black box worth exploring in depth. This paper intends to fill this loophole. It first sets the stage for this new action field by stressing the differences between the policy fields of higher education in the United States and Europe, as a way to understand the specificities of the use of metrics and rankings on both continents. The second part describes the actors of the field, which productive organizations they build, what skills they combine, which products they put on the market, and their shared norms and audiences.

Ranking groups (schools, regions, counties) according to the average score of their constituent parts – say, ranking schools by the academic achievements of students – is a common yardstick in evaluation and a cornerstone of any planning process. In this paper we show that under certain circumstances the ranking of groups, unlike the ranking of individuals, can be affected by the examiner. The fact that a ranking reversal is possible does not necessarily imply that the test itself is deficient, it merely reveals that the groups are non-homogeneous and therefore, the ranking of groups is meaningless. An investigation of the conditions under which such manipulation can occur leads us to suggest new statistical indicators as warning signals that can help one recognize such situations if and when they arise. It turns out that these indicators are related to the indicators for finding whether one distribution dominates another.

In 2009, Newsweek published a report in which they ranked the 500 largest US companies and the 100 largest global companies based on its environmental performance measures (http://greenrankings2009.newsweek.com/). This ranking is referred to as Newsweek’s Green Ranking. Included in this ranking is information about water and air pollution, solid waste disposal, toxic wastes, carbon emissions, and enforcement actions. The question we are addressing in this study is how well it measures pollution performance? The question is relevant to environmental accounting/reporting since it is part of a dilemma yet to be answered: Aggregated environmental indices/scores are easy for average information users to percept, while specific information may not be preserved when it is aggregated into the overall score(s).

Specifically, we examine whether Newsweek’s Green Ranking is correlated with pollution measures based on Toxics Release Inventory (TRI) in order to determine how valid or reliable Newsweek’s Green Ranking is – in other words, how much Newsweek’s Green Ranking can explain the pollution by the toxic releases. We find that there is no significant correlation between Newsweek’s Green Ranking and the TRI measures except for the firms in the utilities industry. Concluding that on one measure, which we consider a very important one, there is no justification for the overall Green Ranking Score presented by Newsweek. However, in Newsweek’s three-part score the element that is termed the Environmental Impact Score captures pollution performance measured based on TRI. The contrast between the overall ranking and performance ranking indicates that a composite index that incorporates hard performance and soft measures can dilute the information carried by performance data.

Purpose — Ranking is a natural task for a search engine; a search engine result page is the most common example. This chapter aims at illustrating the motivations and the concepts of rank correlation in a practical way for the researchers active in the different domains of search engines.

Methodology/approach — To this end, this chapter provides a survey according to a topic-oriented basis of the search engine evaluation literature specifically devoted to or based on rank correlation; the chapter explains and illustrates how statistics is the only approach to rank correlation.

Findings/research limitations/implications — The chapter introduces the pros and cons of rank correlation measures through a light-weight formal description and a number of concrete examples to find the measure that better fit a context.

Practical implications — This chapter provides a blueprint for the application of rank correlation within scientific experimentation or item/service recommendation.

Social implications — Rank correlation analyses impact on the success or failure of a search engine in performing the tasks for which it has been designed and hence on the people's daily life activities.

Originality/value of paper — This chapter places rank correlation within a scientific research perspective and in particular connects to and complements documentation on search engine evaluation.

The visualization of ranking information in global public policy is moving away from traditional “league table” formats and toward dashboards and interactive data displays. This paper explores the rhetoric underpinning the visualization of ranking information in such interactive formats, the purpose of which is to encourage country participation in reporting on the Sustainable Development Goals. The paper unpacks the strategies that the visualization experts adopt in the measurement of global poverty and wellbeing, focusing on a variety of interactive ranking visualizations produced by the OECD, the World Bank, the Gates Foundation and the ‘Our World in Data’ group at the University of Oxford. Building on visual and discourse analysis, the study details how the politically and ethically sensitive nature of global public policy, coupled with the pressures for “decolonizing” development, influence how rankings are visualized. The study makes two contributions to the literature on rankings. First, it details the move away from league table formats toward multivocal interactive layouts that seek to mitigate the competitive and potentially dysfunctional pressures of the display of “winners and losers.” Second, it theorizes ranking visualizations in global public policy as “alignment devices” that entice country buy-in and seek to align actors around common global agendas.

When and how did modern rankings emerge? This paper aims to answer that question by taking a closer look at the history of American baseball. In the 1870s, baseball was the first team sport to introduce a competitive system, the league, that determined the champion based on teams’ overall number of wins and losses. The in-depth analysis of the baseball discourse from the 1850s to the 1870s shows that leagues were introduced as a solution to a specific problem: how to identify deserving champions that had proved their ability again and again over the course of a season. The rising awareness of this problem was due to a shift in the baseball discourse of the 1860s, which established a new, statistical understanding of athletic achievement that demanded consistency of performance together with an acceptance that even champions lose a game once in a while. Rankings and other statistics, based on constant scoring of individual plays and increasingly sophisticated methods, helped institutionalize this new understanding of achievement and, in so doing, made the introduction of the league system possible. Moreover, the league system proved to be dependent on rankings – in the form of league tables – that made it possible to observe and experience the championship race, making rankings an essential element of modern competitive sports. Given that today’s rankings apply similar ideas of achievement to other fields (e.g., the “excellence” of universities), the story draws attention to the history of a specific imaginary of achievement that transcends the field of sports and should be studied more widely to understand the institutionalization of rankings in other fields.