Beyond Winners and Losers: Ranking Visualizations as Alignment Devices in Global Public Policy*
ISBN: 978-1-80117-106-9, eISBN: 978-1-80117-105-2
ISSN: 0733-558X
Publication date: 22 July 2021
Abstract
The visualization of ranking information in global public policy is moving away from traditional “league table” formats and toward dashboards and interactive data displays. This paper explores the rhetoric underpinning the visualization of ranking information in such interactive formats, the purpose of which is to encourage country participation in reporting on the Sustainable Development Goals. The paper unpacks the strategies that the visualization experts adopt in the measurement of global poverty and wellbeing, focusing on a variety of interactive ranking visualizations produced by the OECD, the World Bank, the Gates Foundation and the ‘Our World in Data’ group at the University of Oxford. Building on visual and discourse analysis, the study details how the politically and ethically sensitive nature of global public policy, coupled with the pressures for “decolonizing” development, influence how rankings are visualized. The study makes two contributions to the literature on rankings. First, it details the move away from league table formats toward multivocal interactive layouts that seek to mitigate the competitive and potentially dysfunctional pressures of the display of “winners and losers.” Second, it theorizes ranking visualizations in global public policy as “alignment devices” that entice country buy-in and seek to align actors around common global agendas.
Keywords
Citation
Bandola-Gill, J., Grek, S. and Ronzani, M. (2021), "Beyond Winners and Losers: Ranking Visualizations as Alignment Devices in Global Public Policy
Publisher
:Emerald Publishing Limited
Copyright © 2021 Justyna Bandola-Gill, Sotiria Grek and Matteo Ronzani
License
This chapter is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
Rankings are ubiquitous devices for monitoring and assessing performance, as well as for supporting the implementation of social and environmental reforms around the world (e.g., the UN’s 2030 Agenda for Sustainable Development). Most notably, the introduction of the Sustainable Development Goals (SDGs) in 2015 has consolidated the “data-driven” nature of contemporary approaches to grand challenges in the global public policy space (Merry, 2016). The aspirational and transformative policy agenda behind the SDGs is grounded on a complex interplay of infrastructures of measurement (Merry, 2019) that concerns a variety of monitoring and steering processes via a range of calculative governance tools. Whilst the reliance on quantification has long been a defining feature of global governance (Grek, 2020; Rottenburg, 2009; Shore & Wright, 2015), the SDGs unlike their predecessors – the Millennium Development Goals – introduced innovative qualities to the development and monitoring of cooperation in global governance. The SDGs seek (at least in theory) to promote participatory and consensus-driven processes that foreground the country-led nature of this agenda as opposed to the more top-down set up of their predecessors (Fukuda-Parr & McNeil, 2019).
As the global governance space is seen to become more participatory (Barry, 2012; Biermann et al., 2009), the role of rankings is evolving to not only assess the performance of countries but also encourage collective action to promote global solutions to global problems. The emerging paradigm of global governance declares the turn toward country “ownership” of how performance information is produced, communicated, and acted upon (Fukuda-Parr, 2016) and, crucially, reflects the increasing sensitivity toward issues of data “democratization” and the pressures to “decolonize” global governance (Quijano, 2007; Rottenburg, 2009). These critiques highlight how the historical, cultural, and sociological underpinnings of eminently Western technologies of quantification such as rankings can contribute to the “data colonization” of the Global North upon the Global South (Arora, 2016). In this paper, we investigate how the discourse of participation embedded in the SDGs and the pressures to decolonize global performance measurement influences how rankings are visualized and communicated to heterogeneous global stakeholders. In so doing, we explore how rankings are visualized by key global actors to encourage country participation in transnational performance measurement initiatives.
Extant research has explored in depth the political work of rankings and the salient effects of their reactivity (Espeland & Sauder, 2007; Pollock, D’Adderio, Williams, & Leforestier, 2018; Sauder & Espeland, 2009; Slager & Gond, 2020) in different organizational contexts. However, the literature has privileged the study of rankings and the effects of their visualization in the consumer economy (Jeacle & Carter, 2011; Kornberger, 2017; Pollock & D’Adderio, 2012), in commodified higher education domains (Espeland & Sauder, 2007; Free, Salterio, & Shearer, 2009; Hazelkorn, 2011), and in public service delivery (e.g., Bevan & Wilson, 2013). Notably, the workings of rankings are particularly under-investigated in settings where the very notions of “winners and losers” and the hierarchical ordering of performance are politically and ethically sensitive (Bandola-Gill, 2020; Bhuta, Malito, & Umbach, 2018). In our study, we detail how and why the poverty and wellbeing rankings produced within the broader measurement infrastructure of the SDGs are visualized in interactive, indirect, and multivocal formats. The focus of the paper is not on the production of the quantitative measures that constitute a ranking or how such rankings are used; rather, we investigate how ranking information is visualized to communicate the political and rhetorical objectives of key actors involved in global performance measurement and monitoring initiatives.
Our effort to unpack the criteria that inform the visualization of rankings in global public policy builds on current research conducted as part of the European Research Council Grant funded project “International Organisations and the Rise of a Global Metrological Field”. We used a two-fold methodological approach that seeks to capture the “best practices” informing the visualization experts as well as how these are translated into material practice. First, we contextualize the making of ranking visualizations by analyzing the rhetoric of the white papers and guideline documents produced by the American visualization software company Tableau. Tableau is a key player in the interactive data visualization IT landscape and is emerging as the standard technology to produce visualizations by key global players such as the World Bank, the OECD, and UNESCO to mention but a few. Contextualizing our analysis from the prism of Tableau’s strategies and practices assists in entering the rhetoric of contemporary approaches to information visualization. Secondly, we conduct a visual analysis of a set of digital ranking visualizations concerned with poverty measurement and the measurement of wellbeing. These ranking visualizations were either developed with Tableau or produced according to its more general principles and guidelines that have been endorsed by the OECD, the Gates Foundation, and the World Bank. Our findings show that interactive formats allow ranking visualizations to become more malleable, customizable, and – more importantly – “softer” in their messaging. Unlike static league tables, ranking visualizations leave more room for users’ interpretative predispositions, which may mitigate the crude reactivity pressures resulting from the hierarchical order of performance on the ranked entities.
This paper makes two contributions to the study of rankings. First, the paper analyzes how interactive formats enable country rankings to move away from league table formats toward more “inclusive” forms of ranking visualizations. We theorize the role of ranking visualizations as “soft” tools for governing the global public policy space. The impellent pressures to decolonize global governance and the use of performance measures (Arora, 2016; Rottenburg, 2009) have profoundly affected how rankings are visualized in this setting and thus, how they convey more ambiguous meanings that seek to balance the clarity of the message and its political acceptability. The “soft” governance function of rankings is linked to how their visual configuration is designed to increase participation and seek collective problem identification and action. Hence, the visualization of rankings has become a crucial rhetorical locus for international organizations to showcase the horizontal relationships between countries as equal participants in the global sustainability agendas they are seeking to promote.
Our second contribution consists in the theorization of ranking visualizations as “alignment devices” in the fragmented and polycentric space of global public policy. As international organizations are guided by the paradigm of country-driven decision-making (Best, 2014), aligning actors along common sustainability agendas is becoming an increasingly delicate endeavor. The findings show that interactive data visualizations are primarily used to facilitate the active engagement of users in the production of their own multiple calculable and actionable problem spaces. Building on the literature on reactivity (Espeland & Sauder, 2007; Pollock et al., 2018; Sauder & Espeland, 2009; Slager & Gond, 2020), we show that ranking visualizations engage their users in global public policy by allowing for interactivity, through features such as customization, multivocality, and edutainment. We explain how the user-specific outcomes of these rankings are not a deterioration of the power of rankings as a technology of governance. To the contrary, rankings are so taken-for-granted (Brankovic, Ringel, & Werron, 2018) that rather than judging, they can “align” their users by allowing constant re-adjustment of parameters of evaluation. In so doing, ranking visualizations allow their users to create acceptable narratives around both the specific country performance and the common global sustainable development agenda. Ranking visualizations as alignment devices aim to retain the illusion of the political neutrality of their producers foregrounding an “issue-based” message (e.g., ending global poverty) rather than focusing on augmenting competition and peer pressure to achieve individual goals (e.g., benchmarking country-level performance). As such, ranking visualizations align actors with diverse interests and interpretations of performance by allowing for the co-existence of multiple, often contradictory interpretations of one ranking – a quality which is facilitated by the interactive visualizations.
2. Theoretical Approaches to Rankings and Their Visualization
2.1. The Ubiquity and Reactivity of Rankings
It is now commonplace that rankings affect more and more aspects of the economy and society. As Ringel, Espeland, Sauder, and Werron argue in this volume, the explosion in the quantity and kinds of these performance measures is reflected in the ever-growing interdisciplinary studies of rankings. Scholars have documented the impact of rankings on a plethora of settings, which include – but are not limited to – the management of schools (Wedlin, 2006), universities (Free et al., 2009), cities (Kornberger & Clegg, 2011), hotels and restaurants (Jeacle & Carter, 2011). These studies have shown that rankings can operate as exogenous drivers of organizational and social change and that their social, political, and practical appeal is tightly linked to their mechanical objectivity (Porter, 1995). The rapid emergence of rankings reflects the broader erosion of “local knowledge and professional autonomy […] [as] they insinuate and extend market logic” (Sauder & Espeland, 2009, p. 80). A powerful driver of these processes is data inertia (i.e., the acceptance of numbers that are difficult to verify), which is seen to increase the trust in rankings (Merry, 2016).
The vast scholarship on rankings explores the reactive behaviors they induce on the entities they orchestrate, foregrounding their almost innate capacity to entice a response (Espeland & Sauder, 2007; Desroisères, 2015; see Pollock, D’Adderio, & Kornberger, this volume). By problematizing how rankings have intended (e.g., “accountability” and “transparency”) and unintended consequences (e.g., “gaming” and “cherry-picking”), the literature details how rankings instigate self-fulfilling prophecies that push the ranked entities to adapt to, conform to, and buffer institutional pressures (Sauder & Espeland, 2009; Slager & Gond, 2020). These studies show that rankings force comparisons and oversimplify the phenomena they ostensibly represent (Espeland & Stevens, 2008), and are inherently opaque and potentially oppressive calculative practices (Shore & Wright, 2015). However, unlike the reaction to a ranking, the issue of users interpreting rankings is relatively under-developed in the literature. Comparatively few studies on university rankings have problematized how users’ interpretations are multiple and dependent on idiosyncratic processes of the narrative building that are predominantly self-serving (Elsbach & Kramer, 1996; Hazelkorn, 2011; see Jandrić & Loretto, 2020).
The extant literature on reactivity tends to portray the reaction to a ranking as “something like a reflex” (Pollock et al., 2018, p. 57). The connection between reactivity and the representational incompleteness of rankings that the literature foregrounds can prevent an appreciation of a ranking’s more implicit and far-reaching properties, such as how rankings are a form of expressive action with generative effects on the users’ interpretations. Importantly, when a ranking is successful, “its self-fulfilling prophecies become correct without being “true”’ (Esposito & Stark, 2019, p. 18). As such, rankings create social orders and are interesting “not because they inform us about how things are but because they provide an orientation about what others observe” (Esposito & Stark, 2019, p. 5).
2.2. Interpreting and Performing Ranking Visualizations
A key factor that props up the spread and generativity of rankings is their display in visual formats (Brankovic et al., 2018; Kornberger, 2017). As Minnetian and Werron suggest in this volume, rankings are inherently visual ordering technologies whose basic functioning is tightly linked to the device of the list, arranged in grid-like, tabular formats (Stark, 2020). The list itself is an ordering device whose rhetorical power has been deployed as early as in Homer’s Iliad and has profoundly shaped Western approaches to classification, enumeration, and the assignment of worth (Eco, 2009). As the semiotician Umberto Eco (2009) argues, lists are visual devices that – if characterized by an eminently practical purpose such as that of a ranking – can be understood as having referential purposes (i.e., are concerned with naming and listing), are finite (i.e., they seek to establish references between line items and what the items are seen to “stand for”), and do not seek to induce the reader look beyond what is visually represented. In other words, such “practical” lists seek to generate an organized cosmos within the impermeable boundary of their visual frame (Eco, 2009). Tables and grids are indeed the oldest, most established, and widespread form of ranking visualization and reflect rationalized visibility based on rigorous procedures for the rule-governed construction of images (see Chakhovich & McGoun, 2016).
Conventional league tables have long been shown to have – and criticized for having (Shore & Wright, 2015; Snyder, 2020; Van Erp, 2011) – powerful effects in enticing competitive behaviors (see Samiolo & Mehrpouya, this volume). This is especially salient in the context of public governance approaches inspired by “naming and shaming” where rankings guide the perception toward publicly identifying the poor performers and areas in which they lag behind (Barber, 2008; Bevan & Fasolo, 2013). The proponents of these approaches highlight how the visibility and alleged transparency of league tables is a key instrument in supporting reforms inspired by choice and competition ideologies (Le Grand, 2003, 2007), given how individuals and organizations commonly respond to threats to their reputation, which have stronger impact compared to the influence of praise and positive reinforcement (Bevan & Wilson, 2013). However, primarily thanks to the affordances enabled by digital media, ranking visualizations have become more sophisticated, engaging, inventive, and arguably less direct over the years.
There is an increasing realization that the visualization of performance measures may facilitate the understanding of complex information sets, support interpretation and sensemaking, and prompt engagement with calculative technologies (Gatzweiler & Ronzani, 2019; Quattrone, 2017). Importantly, the interactive and artefactual properties of digital data displays and ranking visualizations need not be confused with an unmediated presentation of underlying information as “all visualizations are artefacts in their own right, and knowing how to read them as graphical expressions is crucial” (Drucker, 2015, p. 2, emphasis in original). In particular, the properties of interactive ranking visualizations endow them with an aesthetic appeal that is irreducible to the hierarchical ordering of performance and affects how users interpret appropriate, and make meaning with them (see Espeland & Stevens, 2008; Kornberger, 2017). For these reasons, scholars called for a reconceptualization of rankings in the digital age, contending that the ranking of performance in digital platforms (e.g., Instagram) is softer and relies on aesthetic and palpable evaluations in absence of robust judgement devices (Begkos & Antonopoulou, 2020), such as league tables. Appealing ranking visualizations are seen to move beyond the crude reactivity mechanisms prompted by league tables. For example, recent research shows how higher education is influenced by the rankings’ visual formats, which “allow platforms to at once display cascades of inscriptions in a pleasant, aesthetic manner and further complicate the numerical-ordinal basis of traditional ranking systems” (Decuypere & Landri, 2020, p. 12).
The interpretative flexibility of ranking visualizations makes them prime sites to explore some of their generative effects (Pollock & D’Adderio, 2012). Visual elements are not only important because they support calculations but also because they offer interpretative clues that cognitively and esthetically engage the users of a ranking (Espeland & Stevens, 2008; Quattrone, 2017). For instance, ranking visualizations in two-by-two matrix format have been shown to shape the construction of markets thanks to “beautiful pictures” such as Gartner’s Magic Quadrant (Pollock & D’Adderio, 2012), which can allow the accustomed users to process complex performance information at a glance. Ranking visualizations are also critical to knowledge brokerage, as they enable the communication of research findings to different discourse communities and play important roles in the legitimation and dissemination of research contributions (Allen, 2018). The visualization of ranking information also imposes readings and creates opportunities for regulatory intervention on policy issues in the global governance space (Mehrpouya & Samiolo, 2016). However, the issue of how ranking visualizations operate in settings where the hierarchical ordering of performance – and its implied value judgments – are politically and ethically sensitive (e.g., in global poverty measurement) is under-investigated.
2.3. Ranking Visualizations in Global Public Policy: Decolonizing Development Whilst Monitoring Performance
In this study, we explore how interactive ranking visualizations operate in the policy field of global poverty and the measurement of wellbeing. In public policy, the hierarchical ordering of performance is frequently concealed behind alluring visual artefacts in a variety of formats, such as interactive data maps, dashboards, and playful graphs (see Lafortune, Gonzalez, & Lonti, 2018). However, what is the reason for the increased use of such interactive visuals in, for example, several of the SDGs? What are the characteristics of these visualizations that make them more attractive to the experts and brokers of such large transnational governing agendas? We suggest that answers to these questions are in the move toward the “decolonization” of global governance and “data democratization” (Fukuda-Parr & McNeil, 2019).
Indeed, the increased sensitivity toward “decolonizing” development and global performance measurement emerged in response to critiques of the “coloniality” of knowledge-making (Quijano, 2007) that is implied in quantitative approaches to knowledge production. Such approaches are frequently criticized for being based on neoliberal ideals and on a Western understanding of rationality that enforces and glorifies competition among nations and institutions (Best, 2014). Recent research has condemned the “data colonization” of the Global North upon the Global South, which can be seen as combining “the predatory extractive practices of historical colonialism with the abstract quantification methods of computing” (Couldry & Mejias, 2019, p. 337); this is of particular concern in relation to the expansion and commodification of big data in the Global South (Arora, 2016). As a result, initiatives aimed at systematizing the collection and use of country performance data could reinforce how the Global South remains at the bottom of the data pyramid (Arora, 2016).
In the case of large international learning assessments, rankings have been shown to create the conditions for “southering,” which suggests that “the presentation of the results as tables and world maps can result in exposing countries of the South to a pronounced deficit perspective” (Grotlüschen & Buddeberg, 2020, p. 1). These findings highlight how traditional, static rankings and league tables could systematically alienate the Global South, thereby exposing International Organisations to the risk of being seen as new colonial powers. Recent studies have started to document how global actors are prone to sacrificing the robustness of their data validation practices to avoid disenfranchizing specific countries (Grek, 2020) and that visualization practices are sensitive toward these pressures (Lafortune et al., 2018). However, this line of enquiry is still in its infancy and offers substantial potentialities for development for the study of rankings. In what follows, through the discourse and visual analysis of some of the rankings used in global poverty and the measurement of wellbeing, we investigate the visual and rhetorical strategies that influential global actors use to communicate the outcomes of performance measurement initiatives.
3. Methodological Considerations
3.1. A Visual and Discursive Analysis of Rankings
In this study, we analyze how rankings leverage on the visual semiotic mode to move beyond the crude reactivity mechanisms prompted by league tables and to convey meanings and opportunities for engagement to their users. A mode is “a socially shaped and culturally given semiotic resource for making meaning” (Kress, 2009, p. 79). The visual mode is in our case instantiated in the use of colors, shapes, lines, and a variety of interactive visual forms in rankings. As Meyer, Jancsary, Höllerer, and Boxenbaum (2018) argue, the visual mode has distinctive semiotic features (i.e., it enables particular forms of meaning construction), cognitive features (i.e., is processed differently than other modes, such as text), and reflects the cultural features of specific settings (i.e., the norms of a discourse community). The affordances of the visual mode not only make its perception more immediate than text or numbers but “the lack of a clear visual “syntax” makes visual meaning fluid and indeterminate and strongly dependent on the viewers interpretative predispositions” (Meyer et al., 2018, p. 396). The visual mode can play a variety of argumentative functions: for example, it can offer clues for narrative building; it can construct fluid relationships between its constitutive elements and allow space for different interpretations; it can captivate the users materialize complex ideas in a compelling manner (Meyer et al., 2018). In so doing, the visual mode can also play an implicit role in naturalizing and endowing value-laden ideas with matter-of-factual properties, thereby augmenting their rhetorical power and perceived authority.
The analysis of the interplay of different visual elements – such as colors, icons, headings, and graphics – allows us to unpack the rankings’ rhetorical functions (see Quattrone, 2017). Some of the rhetorical functions connected to the use of visual items include their roles in guiding the user through an interface, illustrate relationships between elements, provide context and tone, focus attention, and increase the impact of specific messages (Meyer et al., 2018). More specifically, visual analysis allows us to decode the rhetorical strategies deployed by the ranking makers and highlights how ranking visualizations are a form of expressive action that influences users’ interpretations. Whilst the commensuration and classificatory power of a ranking that relies on an orthodox league table format can be similar to that of a dynamic and interactive ranking visualization, in this study we foreground how the latter offers opportunities for engagement and interpretation that are not possible in grid-like, static formats.
From this perspective, visuals are value-laden materializations of specific visions of the world that make visible (or invisible) possible realities (Latour, 1986). Analyzing rankings visually and discursively foregrounds how visualization is not a soulless depiction but the outcome of a process of work:
And it is the site for the construction of and depiction of social difference. To understand a visualisation is thus to inquire into its provenance and the social work it does. It is to note its principles of exclusion and inclusion, to […] decode the hierarchies and differences that it naturalises. And it is also to analyse the ways in which authorship is constructed or concealed and the sense of audience is realized. (Fyfe & Law, 1992, p. 1)
Ranking visualizations are an ideal site to explore these issues as they are largely made quantitatively and denote arithmetic values whilst relying systematically on visual codes (through shapes, colors, and lines) that connote social, moral, and political values. The visual and discursive analysis of rankings is an attempt to decode the constitutive elements of those rankings to make sense of the ways visual elements work metaphorically and evocatively in the making of new ways of seeing and knowing the world. The underlying contention that inspires this approach is that “social change is […] a change in the regime of re-presentation” (Fyfe & Law, 1992, p. 2, ). The politically, ethically, and morally salient features of global performance measurement initiatives make this approach fitting to the study of ranking visualizations.
3.2. Data Sources and Analysis
To explore the formats and rhetoric of ranking visualizations, we relied on two data sources: the discourse analysis of guiding documents of a data visualization company (Tableau) and the visual analysis of the rankings published by the key global actors in poverty and wellbeing measurement. We use the case of Tableau to explore how visualization software companies promote the importance of the work they do at the global stage. Our visual analysis of rankings explores this set of strategic principles in practice. The combination of these two methods allowed for an exploration of both the intentionality behind the data visualizations (i.e., the strategies employed by the data visualization experts) as well as the products of the work (i.e., specific ranking visualizations).
Tableau is a producer of interactive dashboards, the main purpose of which is to “help people see and understand data” (Tableau, 2020b). Tableau makes “analysing data fast and easy, beautiful and useful” (Tableau, 2020b). Although one amongst many data visualization companies that emerged and grew during the last couple of decades, Tableau appears as a leading producer not only of visuals but also know-how in this area, as they published over 100 “Whitepapers” (Tableau, 2020c). These publications fulfill a double function: first, they present Tableau’s work in its different facets and second, they market these solutions to organizations that seek expertise on data visualization. For this paper, we examined more than 50 “Whitepapers.” We applied Critical Discourse Analysis (CDA) to 16 of these Whitepapers that were chosen based on their relevance to rankings and global governance. CDA is an apt method for the analysis of visualizations because it sees their discursive analysis as a key aspect of how certain understandings of the world are shaped and perpetuated (Fairclough, 1995; Wodak & Meyer, 2001). As such, it offers the necessary context for the analysis for the sources at the center of our study; that is, the analysis of ranking visualizations themselves.
Our case selection was guided by the objective of capturing the diversity of actors in global governance and their practices for visualizing ranking information. Global governance is a domain with an increasing number of actors and (fragmented) networks (Biermann et al., 2009) where dashboard and interactive data visualizations are becoming a paramount way of disseminating rankings into new domains (Lafortune et al., 2018). To capture this diversity, this paper draws on data across three types of institutions: International Organisations (World Bank, OECD), large philanthropic organizations (the Gates Foundation), and academia (Our World in Data hosted by the Oxford University). The World Bank was selected as it has been doing longstanding statistical analysis of global poverty, renowned through its “dollar per day” measure (Konkel, 2014). Secondly, we discuss the visual maps of global inequality produced by the Bill and Melinda Gates Foundation. They are one of the key funders of global governance initiatives, including through their role as assessor of the SDGs’ progress through their “Goalkeepers” reports (Gates, 2020). Thirdly, we focus on the visualization of poverty rankings produced by the “Our World in Data” group of the Oxford Martin Programme on Global Development. The Group has launched an “SDG Tracker,” which is an open-access “interactive hub where users can explore and track progress across all of the SDG indicators for which there is data available” (Ritchie et al., 2018). Our World in Data was selected as a paradigmatic example of a new knowledge-intensive actor that is not associated with the traditional international institutions. And yet, as illustrated by the ongoing COVID-19 crisis, the expertise in data visualization gives prominence to this group as an internationally recognized producer of visuals for understanding global challenges for media and various global health organizations (Our World in Data, 2020a).
We also examine one of the OECD’s initiatives to map country wellbeing data – the “Better Life Index” (OECD, 2020), launched in 2011. It is an interactive ranking that allows people to compare countries’ performance according to their own preferences in terms of what makes for a “better life.” It includes eleven performance dimensions of wellbeing, namely: housing, income, jobs, community, education, environment, governance, health, life satisfaction, safety, and work–life balance. The analysis of Better Life Index illustrates the evolution of the ranking visualizations as OECD in the past was one of the key producers of “naming and shaming” league tables, most notably their Programme for International Student Assessment (see Grek, 2009). With the Better Life Index, the OECD appears to have shifted its approach to the communication of performance assessments. This is a response to calls – such as the authoritative report by Stiglitz, Sen, and Fitoussi (2009, p.12) – that advocate for “measurement system[s] to shift emphasis from measuring economic production to measuring people’s wellbeing.” This softer take on the handling of country performance data emerges in the interactivity of their new ranking visualizations, which – as we will show – convey more multi-layered meanings compared to league tables.
4. Findings
In what follows, we present the findings of our exploration of ranking visualizations in global public policy. This section begins with a discourse analysis of the documents discussing the principles and guidelines employed by the experts that produce ranking visualizations for key actors in global policy. Specifically, we investigate the ethos that guides these visualization specialists and the values that inspire their work for clients in the global policy area. Second, we move to the visual analysis of rankings of poverty measurement and wellbeing to explore their central qualities as “soft” governing tools – namely, their interactivity, their multivocality, and their objective to move beyond the identification of “winners and losers” in global performance assessments.
4.1. The Visualizers’ Work: Interactive Data and Self-service Analytics
4.1.1. The Age of “Look and Touch”
Tableau describes its mission as delivering user-friendly data “for the people” (Tableau, 2020a) and this guiding principle is reflected in the values that inspire their design of interactive visualizations. According to Tableau’s white papers, the central quality of visualizations should be their interactivity (Krensky, 2014). Interactivity is placed in stark opposition with the “old worldly” static presentation of authoritative data visualizations targeted solely at experts. Tableau proclaims the end of that era: ‘The age of “look but don’t touch” is over’ (Krensky, 2014, p. 1). This principle is reflected in the interactive nature of data visualizations – whose benefits are “too manifest to ignore” (Krensky, 2014, p. 7). Interactive data visualizations are assumed by Tableau to allow for more collaboration and dissemination. They are seen to prompt questions and reflection, improve understanding of complex data sets, and reduce the risk of “gut-level decision-making” that is dictated by the lack of understanding of data (Krensky, 2014). Furthermore, interactive visualization, according to Tableau, is the “panacea” for the information overload that individuals are experiencing in a data-driven society, as it
Drives improvements in the analytical experience: […] adopters are more likely than static visualizers to have improved their speed of decision and trust in underlying data […] (it) fosters user development and engagement […] Adopters of interactive data visualization have a more satisfied user base: Happy users are more productive and more likely to explore data and uncover new insights. (Krensky, 2014, pp. 7–8)
This evangelical perspective assumes that the ease of use will offer greater satisfaction, allow for more inquisitive approaches, and even increase one’s intuition of possible new questions and solutions. At the same time, interactivity is also assumed to increase the trust in the data that sits behind the visual interfaces (Krensky, 2014).
4.1.2. Self-service Analytics
The second key quality of interactive visualizations, according to Tableau, is their ostensible capacity to create new audiences that go beyond the traditional technocratic experts:
There will always be a number of individuals who are power users. […] For most people, however, that would be counterproductive. Instead, they benefit from having data organised around specific topics, with an emphasis on the most meaningful metrics. This approach is especially critical when sharing data with the public, where little can be assumed about an end user’s technical or subject matter expertise. […] The concept of data-driven decision making assumes that decision-makers have access to the right data, not to every available data set. (Tableau, 2020b, p. 4, emphasis added)
Perhaps the key term used in this document is the vision of creating “self-service analytics” (Tableau, 2020b, p. 6). According to Tableau, a key design principle is that little expertise is needed to interpret their interactive visualizations. To make information accessible to non-experts, an interface needs to offer cues that will provide an intuitive way to interact with the data. Tableau’s interfaces aim to
Enable stakeholders to perform basic analytical tasks, such as filtering views, adjusting parameters, quick calculations and drilling down to examine underlying data – all through an intuitive user interface that requires no special expertise. (Tableau, 2020b, p. 9)
However, the diversity of uses and features does not mean the possibilities are unlimited, as even the most interactive visualizations are grounded on the same baseline of a common dataset:
Such discussions are much deeper and productive when everyone involved is looking at the same set of data – what is often called a single source of truth. (Tableau, 2020b, p. 9, emphasis added)
Hence, whilst the adaptable visual interface entices user involvement and encourages the manifestation of one’s preferences in the construction of their view of the data, the ultimate “source of truth” is still the data behind the visual. As such, the personalization of data interfaces is not indefinite – visualizations are only as flexible as the software infrastructure and the underpinning data.
Finally, the emphasis is placed on the creation of one’s own “data-worlds” through experimentation and enjoyment through interactive interfaces. This last point is a key: a certain level of edutainment is necessary to engage users in what has traditionally been seen to be a prerogative of technocratic experts. According to a Tableau Senior Executive:
We create a data culture that relies on language, is flexible and adaptive and is shared with others […] we promote governance through empowerment that relies on learning and fun. (Jewett, 2019)
Such an explorative way of working with data implies that the interactive performance monitoring tools are trying to disguise their capacity to “name and shame” the entities they assess; as we saw in the quotation above, they aim to “empower,” not to judge. They aim to be “flexible” and “adaptive,” allowing a seemingly reflexive, developmental, and multivocal understanding of the data by all those involved. However, the “single source of truth” – and thus a hierarchical ordering of performance – still lurks beneath these playful interfaces. Hence, the affordances of interactive data visualizations also play an implicit naturalization work as, by prescribing specific set of interaction possibilities, they also divert attention away from the value-laden message that these artefacts seek to convey and the complex and messy reality they seek to connote visually and numerically.
What emerges from Tableau’s documents is an effort to popularize the use of interactive visualizations. The users are encouraged to engage with the visualizations and explore the dimensions of data that are most compelling to them. As such, the visualizations become engines for production of multiple interpretations and lenses on data. In what follows, we analyze a series of ranking visualizations that were either produced with Tableau (e.g., in the case of the OECD) or were strongly inspired by its ethos.
4.2. Ranking Visualizations and the Power of Issue-based Messaging
The analysis of the design principles guiding visual experts discussed in the preceding section is central to understanding how rankings are visualized to operate as tools of global governance. The rankers translate these “industry standards” into classifications of countries and their performance in achieving societal goals – e.g., eradicating poverty or improving the wellbeing of the population. As we will show in the remaining section, the rankings produced in the highly political settings of global governance differ from their consumer economy counterparts. The focus here is not on identifying “winners” and “losers,” rather on balancing the clarity of message of the ranking and its political acceptability. In what follows, we explore the “soft” governance function of ranking visualizations by detailing how the design criteria of interactivity, discoverability, and personalization identified in Tableau’s best practices are employed to moderate the political risk of ranking countries in league table format.
4.2.1. “No-one Left Behind” – Ranking Without Critique
In global governance, maps are a ubiquitous way of visualizing country rankings. Turning a traditional league table into a map helps to identify the most affected regions and creates a more “issue-based” message about the problem at hand rather than emphasizing the performance of individual countries. This is achieved through the categorization of countries in accordance with ranges of outcomes rather than based on their individual position in a hierarchical ordering. An example of the translation of a league table ranking into an accompanying map is the World Bank’s map of extreme poverty published in their flagship report Piecing together the poverty puzzle – see Fig. 1 (World Bank, 2015, p. 27). The messaging here is shaped by the strategic use of color. Instead of the more conventional use of red, the map relies on blue. Deep blue is used for the countries that present the highest numbers of people in poverty, whereas the shading of blue changes from darker to lighter, to correspond to the different levels of poverty across the globe.
The map is void of any unnecessary information; what is there are only the absolute essentials, i.e., a title and the graph’s key. Particularly interesting is its lack of labeling – the map offers no geographical signification and the only visual cues come from the use of color. The large range of extreme poverty in countries in deep blue shows that nuance and numerical accuracy are not of the essence. The map has to convey a message, informing about where global poverty concentrates – this is the global challenge (the “issue”) that the visual ranking focuses upon. The use of blue as the sole color seeks to avoid evaluative claims regarding “winners” and “losers.” Instead, it connotes poverty as a universal problem, differing in scale (the gradient of the color) rather than core qualities. The goal is minimizing the stark divisions between the Global South and North. Akin to SDGs’ pledge of “leaving no one behind” (UNDP, 2018), this map highlights poverty as a global problem.
The softening of the political messaging embedded in the ranking visualization can be achieved by means other than minimal color use. An example of a different strategy is a map of the African subnational measles vaccination rate, produced by the Gates Foundation – see Fig. 2 (Gates Foundation, 2019, p. 51). Fig. 2 deploys an “explosion” of color and text to explain the dangers of the lack of measles’ vaccination for at least half of one-year-olds in the African continent. The coverage of data versus non-data is also very stark (an eruption of color in the South vs grey in the North). Country borders are depicted with the use of black lines, to allow the easier presentation of subnational vaccination rates.
This reflects the key role of visualizations in this context – which is softening the overall message of the ranking through a colorful map. Unlike the league table visual format, translating a ranking into a map avoids the clear judgment on “underperforming” countries. The league table is still presented in the report. However, the clarity of the message remains in terms of the areas that need support, rather than individual countries that are underperforming. The result is a spectrum of color separating and mixing, changing, and fading, almost dripping in one another. Thus, through the intricate use of color this visualization creates an evocative message: territories stained in deep reds and oranges appear as almost on fire, whereas other areas in blue tones give the impression of stability.
4.2.2. Comparisons and the Interactive Ranking
These political sensitivities of maps are further expanded by introducing elements of interactivity, corresponding to Tableau’s visual principles. The move toward “look and touch” is illustrated by the poverty maps drawing on World Bank’s PovcalNet Data (Fig. 1) but produced by Our World in Data (2020b). The following two visuals portray different ranking visualizations of extreme poverty according to the two dominant and parallel ways of measuring poverty. These are: (i) the proportion of the population below the international poverty line (below $1.90 a day) (Fig. 3, Our World in Data, 2020b); and (ii) the proportion of men, women and children living in poverty in all its dimensions according to national definitions (the Multidimensional Poverty Index – MPI, Fig. 4, Our World in Data, 2020b).
The central quality of the ranking visualizations in map format produced by Our World in Data is their interactivity. This is a significant evolution from the traditional table as the use of the interactive interface allows for an exploration of multiple visualizations at once. When one hovers over a specific country, its name and a specific poverty ratio appear; the text box with this information also allows linking with the country’s specific time chart, so that one can view increase or decrease of poverty over time. There is additional interactivity in the timeband below the map; one can move the blue pointer all the way back to 1977 and to the present time again to explore the historical dimension of the problem. The Chart command further below allows the user to select and compare multiple countries simultaneously and over time (Figs. 3 and 4). Visually, we notice an insistence on the world map floating on the vast white plane. The African continent is at the center here, highlighting the need to focus the policymakers and donors’ attention to efforts to eradicate poverty in this region; compare the map presented in Fig. 1 and the difference is stark. Although these interactive maps offer the potential of a single message, they also allow users a multiplicity of other actions: it is precisely the combination of the powerful messaging with the affordance of at least the appearance of endless space and time comparisons which render this ranking visualization exceptionally powerful in its communicative potential the opportunities for appropriation it offers to the user.
A comparison of Fig. 3 with the multidimensional poverty map (Fig. 4) reveals the extent to which poverty is seen as more widespread following the MPI measure. Whereas in the “dollar-per-day” map red and oranges are used to describe extreme poverty, a much wider spectrum of color is used for the MPI map. There is no data for the global North. On the contrary, a burst of color describes the Global South: blacks, purples and pastel cues portray more than half of the globe and the degree of multiple deprivations globally in 2014. No time dimension is given in this graph. It is also a telling manifestation of how the choice of the measure – the “single source of truth” – influences the way one views the world and its challenges, and which aspects are naturalized and which are flagged as requiring intervention.
The interactive maps allow the users to explore extreme poverty over geography and historical time, compare different countries and periods, explore the data, share the map constructed and download the data used to develop the visualization. These interactive visuals offer seemingly endless possibilities for exploring the “self-service analytics” discussed by Tableau and yet, the lack of data and existing filters do not render every comparison possible. The range of possibilities and interaction is delineated by software requirements and the limitations of the “league tables” and ordinal lists that underpin the data that underlie the visualizations. Nonetheless, the combination of the single powerful message with seemingly endless space and time comparisons illustrates the soft messaging of interactive maps. As the comparisons multiply, the clear “losers” become even more (but not completely) opaque.
4.3. The Multivocality of a Ranking Visualization
Lastly, we explore an index that most explicitly illustrates the key characteristic of ranking visualizations – their multivocality. 1 The Better Life Index (OECD, 2020) is a core part of the OECD’s Better Life Initiative. It is an interactive report based on statistical data released every two to three years that offers evidence on a range of wellbeing indicators and their variation over time, between population groups and across countries. The index is based on a multidimensional indicators framework that aggregates data provided by 37 OECD countries and 4 partner countries, reporting on more than 80 wellbeing indicators. The Better Life Index default visualization resembles a meadow, full of colorful flowers, with numerous petals of different colors and sizes (OECD, 2020). The country names serve as the “stems” of these flowers, as the default alphabetical order of the floral scene allows for the “meadow” to almost acquiring a dynamic character; the visual diversity of the petals and their positioning against the light background creates a sense of movement, proximity, and heterogeneity (Fig. 5).
It is important to note that this is the default image. At the bottom right, the table allows the user to move from the alphabetical to a ranked visualization. Here, the details of the flowers become impossible to distinguish immediately as the flowers (standing for countries) are visualized in an upward performance trend, with little explanation. If we zoom on any flower, we see a detailed country summary (Fig. 6).
Every color-coded petal stands for an individual wellbeing indicator, building on a 0–10 scale. In the ranked visual option, we can see how the aggregation of all the parameters creates a hierarchical ordering of countries by wellbeing scores. However, these “beautiful pictures” do not strive for contemplation as they seek to entice user participation. Whilst the flowery landscape is visually playful, there is an explicit invitation to the user: “Create Your Better Life Index.” One can adjust the significance that several aspects of social life carry according to their preferences regarding the importance they attribute to different elements of the ranking. The interactivity of the visual gives the impression that the user’s preferences are the engines behind the construction of their “ideal” flower (the meaning of wellbeing) that is to be “found” and “picked” in the field. Importantly, one can also decide to exclude one or more parameters of evaluation from the ranking, should they not be relevant to their evaluation processes. The room for personalization not only allows the OECD to gather user data but this ranking visualization is also marketed as a tool for non-experts to identify a country that meets their priorities and value systems, while learning potentially interesting information on their own country of origin or future destinations.
The OECD does not conceal that the Better Life Index is ultimately a wellbeing ranking. However, more emphasis is on how the index allows the (re-)construction of a combined perspective of country performance, users’ preferences, and multiple policy areas. As one begins to “play” with the tool, the interaction possibilities seem endless. The country-level statistical data are not lifeless; on the contrary, through interactive visual features, they encourage playful behavior and a spirit of discovery deliberately appealing to the edutainment function of the ranking. These affordances allow the user to orchestrate the hierarchical order of countries based on their personalized parameters of evaluation. Such an approach is unusual in the context of the OECD’s historical propensity to rely on league tables and naming and shaming as their key performance measurement and monitoring strategy (Grek, 2009).
The Better Life Index, therefore, illustrates the key quality of interactive ranking visualizations in global governance – it seeks to come across as seemingly apolitical, yet it conveys a political message. This is the most significant evolution from the tabular formats as this type of representation of the ranking completely abandons the idea of a single visualization. By offering the users an ability to create their own ranking – according to their values, interests, and priorities – the OECD ranks countries without providing a set (and more politically sensitive) list of winners and losers in the policy area of wellbeing. Instead, it seeks to provide a unity of experience and reflects the move toward more equitable relationships between countries as equal participants in the global sustainability agendas.
5. Discussion and Concluding Remarks
The rankings of poverty and wellbeing discussed in this paper operate in an institutional and political context that is shaped by ever-expanding infrastructures of measurement (Merry, 2019) and pressures to decolonize global governance (Rottenburg, 2009). This context is not only highly fragmented by the growing number of actors but it is also participatory, as it requires constant mediation and brokerage (Bandola-Gill, 2020; Grek, 2020). The extant literature has privileged the analysis of rankings in the consumer economy (e.g., Jeacle & Carter, 2011; Pollock et al., 2018; Pollock & D’Adderio, 2012) or in commodified higher education domains such as business schools and law schools (e.g., Elsbach & Kramer, 1996; Espeland & Sauder, 2007; Free et al., 2009; Hazelkorn, 2011). However, in the context of global governance, rankings aim to produce different organizational and political effects. Focusing the “hearts and minds” of those who participate in global performance measurement initiatives requires intense transnational coordination. Furthermore, an increasingly pressing issue is the development of measurement approaches that avoid alienating low-performing countries by continuing to pressurize them to conform with “best practices” from elsewhere. International organizations and other key global players purport to design their measurement programs following equity paradigms, where all the countries – and especially the developing ones – are seen as leading on tackling the global challenges (Best, 2014).
Against this backdrop, rankings occupy a liminal space between the rigid assessment of a league table and participatory paradigms of global governance. They have to be produced in a way that navigates political pressures while communicating the urgency of the global problems as truly global – affecting the entire international community. A challenge is to leverage the rankings’ power to entice participation in measurement programs (Desroisères, 2015; Le Grand, 2003, 2007) whilst keeping at bay their documented capacity to trigger competitive behaviors that would be dysfunctional in the global policy space (e.g., a sense of zero-sum competition, gaming, cherry-picking, or the manipulation of data – see Espeland & Sauder, 2007; Merry, 2016; Slager & Gond, 2020). Arguably, the global “need” for quantification and performance measurement has never been as perceivably legitimate as it is since the introduction of the SDGs framework. Simultaneously, there has never been as much attention paid to how global performance measurement may be a form of “southering” (Grotlüschen & Buddeberg, 2020) that presents developing countries as regions of persistent deficit, under the surveillance of Western institutions through different forms of quantification (Arora, 2016).
This leads to the key point and the key contribution of this paper; that is, their role as soft communication tools and their positioning as alignment devices.
5.1. Beyond Winners and Losers: Softening the Communicative Power of Ranking Visualizations
Rankings in this setting have to reflect multiple orders of worth that co-exist, sometimes overlap, and potentially clash (Stark, 2009, 2020; Clune & O’Dwyer, 2020; see Ferraro, Etzion, & Gehman, 2015). The “old” format of ranking visualizations – the league table – does not seem to be suitable to capture this multiplicity, as its main rhetorical focus is on the clarity of communication (at least at face value) of the ranking and in the immediate visibility of performance. The rankings’ “new” visual formats seek to address this tension through their technical features and visual configuration in interactive dashboard templates. The qualities of interactivity, engagement, and trust are essential in heterarchical and polycentric settings such as the global governance space, where different “hierarchies and orderings intertwine and reproduce, none of which can claim to be dominant or even to be fixed” (Esposito & Stark, 2019, p. 15). Since no single order shared by all exists in such a space, the designers of rankings do not necessarily have to order performance hierarchically in an explicit manner anymore. Our findings highlighted how the maps of poverty (Fig. 1) or the interactive visuals of poverty over time (Fig. 3) indirectly rank performance and background specific countries from their visual representation. These visuals seek to achieve more “equitable” and politically acceptable messaging by “softening” the rankings’ communicative power and appeal. Instead of ordering according to achievements, the visualizations increase the visibility of areas of concern and potential intervention without seeking to “shame” explicitly any country.
Through interactivity, the message of the ranking is not simply “fed” to the user; rather, although the messaging remains focused and clear, the user is also given the tools to engage with the digital interface. By playing with the interactive formats, the users can see the multiplicity of rankings, for example by choosing different comparisons or even – as it was the case with the Better Life Index – by choosing the importance of the specific value dimension of a ranking. What emerges at the end of this ostensibly playful interaction is a message that does not seek to identify out a clear “loser” because – rhetorically – there is none. By offering multiple views of the rankings, the ranked entities become de-individualized and move toward being aligned. The interactive formats we explored in our study are a clear departure from the traditional visibility that is perpetuated by more conventional rankings whose argumentative power is tightly linked to their capacity to communicate winners and losers almost at a glance (Bevan & Wilson, 2013; Pollock & D’Adderio, 2012; Wedlin, 2006). The interactive ranking visualizations we detailed in the findings capitalize on more subtle qualities: they invite engagement, they afford personalization, and seek to mold to individual preferences and priorities.
The interactive rankings of poverty and wellbeing are grounded in the logic of contextual comparisons, interactivity, and personalization of the message, rather than top-down hierarchical assessments. At the same time, this does not mean that hierarchical performance ordering disappears in the “softening” of ranking visualizations. These interactive visualizations are still accompanied by ranked data tables which, using Tableau’s jargon, could be referred to as the “single source of truth” underpinning these visual devices: the interpretations are multiple, but not endless – they are limited by the underpinning data table and software requirements. These considerations are indicative of the extent to which the ranking as a social form is taken-for-granted even in fields that seem to be inherently at odds with the rhetoric of performance improvement through competition (Brankovic et al., 2018). The “soft” way of ranking influence that we documented through the analysis interactive visualizations ostensibly leaves more room for the interpretative predispositions of the users and, as such, can be seen to drive engagement with the data that underpin the ranking.
5.2. Rankings as “Alignment Devices” in Global Public Policy
Extant literature has highlighted that ranking visualizations are interpretatively flexible (Pollock & D’Adderio, 2012) and that conflicting value dimensions in the design of visualizations can be a resource for meaning making (Gatzweiler & Ronzani, 2019; Quattrone, 2017). Our case study expands on these arguments and theorizes the ways that ranking visualizations are designed to convey malleable and multi-layered meanings to different audiences. Rankings are no longer just “interpretatively flexible” (e.g., Pollock & D’Adderio, 2012) – they become multivocal by design.
In this context, the multivocality of ranking visualizations transforms them into alignment devices that can help secure country buy-in in global performance measurement exercises. All the visual strategies we explored – from using a map to illustrate the severity of poverty without naming and shaming specific countries, to multiple comparisons embedded in the interactive ranking visualizations of poverty and wellbeing – conveyed the meaning of the ranking as allowing its adjustment to fit the specificity of the user. Rankings transform from “judgement devices” such league tables into alignment devices as they are oriented toward aligning countries toward the common goal without antagonizing them (see Ferraro et al., 2015). The set of visual cues and interactive features that the paper discussed is oriented toward the political goal of focusing policy priorities toward specific global challenges, many of which might look similar, yet contextual and regional specificities and trajectories render them different.
This is the central role of multivocality in this context: rankings have to be clear enough to point to problems and inspire collective action, without being specific enough to shame lower-performing countries. We do not posit that the multivocality of rankings is simply an idiosyncratic matter of interpretation (Elsbach & Kramer, 1996; Hazelkorn, 2011). The ranking visualizations are not only interpreted in multiple ways but also their interactive features actively facilitate this multiplicity of interpretations and fuel the messaging emerging from the ranking. The inherent multiplicity does not take away from the authoritative nature of the ranking and the data it carries. On the contrary, it reinforces the rankings’ credibility by making them relevant to many without antagonizing lower-performing countries. In other words, in global governance, the ranking visualizations afford re-adjustment of results and narrative building around a country’s performance, while offering a steer toward a very specific and discursively “universal” set of goals.
The inherent multiplicity of ranking visualizations brings us to a final comment: one on their “world-making” component. As argued by Latour (1986), visualizations stabilize specific versions of reality; they can make impossible things realistic and make possible objects more probable than others. In our findings on the OECD Better Life Index, we explored how this interactive ranking visualization not only allows for exploration of multiple aspects of the data but also enables customization of the ranking by allowing the user to choose different value dimensions in accordance to their own preferences. These visualizations are conceived and designed (as evident in our discourse analysis of Tableau’s white papers) to allow the user to create their own rankings. Consequently, there are multiple different rankings, assessing different versions of reality as shaped (as least to a degree) by the users. Hence interactive ranking visualizations position the user in the role of the creator whose own version of the world is being assessed.
Overall, this study has addressed a specific empirical and theoretical gap by exploring the notion of interpretability of the visual formats of rankings in the cases of global poverty and wellbeing. We argued that the visualizations of rankings are not merely add-ons, complementary to the work of tabular rankings, but rather they are central to their construction and relevance. We paid attention to the rankings’ multivocality by arguing that specific social settings (such as the global governance of public policy) require different versions and attributes of rankings. The shift toward the use of interactive ranking visualizations further enhances their dominant place in the politics of measurement by adjusting them to act as alignment tools for a complex and contradictory world.
Further research in ranking visualizations needs to examine their qualities and effects in other policy arenas and contexts, explore the actors behind them (both those who produce them, but crucially those that commission and use them) and study their effects. Ultimately, if the governance of global, complex and intertwined challenges – such as the current COVID-19 pandemic – unites us in needing to find common solutions, then more consensual approaches that go beyond divisions between North and South, or winners and losers will have to be secured. What is the future of rankings in this fluid, risky and interdependent world? The events of 2020 suggest that any kind of forecasting is futile; only further research will tell.
Notes
All authors contributed equally and are listed in alphabetical order
In what follows, we adopt the definition of multivocality as developed by Ferraro et al. (2015). In the case, we seek to emphasize how ranking visualizations are the outcome of a “discursive and material activity that sustains different interpretations among various audiences with different evaluative criteria, in a manner that promotes coordination without requiring explicit consensus” (Ferraro et al., 2015, p. 373).
References
Allen, 2018Allen, W. L. (2018). Visual brokerage: Communicating data and research through visualisation. Public Understanding of Science, 27(8), 906–922.
Arora, 2016Arora, P. (2016). The bottom of the data pyramid: Big Data and the global south. International Journal of Communication, 10, 1681–1699.
Bandola-Gill, 2020Bandola-Gill, J. (2020). The legitimacy of experts in policy: Navigating technocratic and political accountability in the case of global poverty governance. Evidence and Policy. Article in press.
Barber, 2008Barber, M. (2008). Instruction to deliver. Fighting to Transform Britain’s Public Services. London: Methuen.
Barry, 2012Barry, A. (2012). Political situations: Knowledge controversies in transnational governance. Critical Policy Studies, 6(3), 324–336.
Begkos, & Antonopoulou, 2020Begkos, C., & Antonopoulou, K. (2020). Measuring the unknown: Evaluative practices and performance indicators for digital platforms. Accounting, Auditing & Accountability Journal, 33(3), 588–619.
Bevan, & Fasolo, 2013Bevan, G., & Fasolo, B. (2013). Models of governance of public services: Empirical and behavioural analysis of econs and humans. In A. Oliver (Ed.), Behavioural Public Policy (pp. 38–62). Cambridge University Press.
Bevan, & Wilson, 2013Bevan, G., & Wilson, D. (2013). Does ‘naming and shaming’ work for schools and hospitals? Lessons from natural experiments following devolution in England and Wales. Public Money & Management, 33(4), 245–252.
Best, 2014Best, J. (2014). Governing failure: Provisional expertise and the transformation of global development finance. Cambridge: Cambridge University Press
Bhuta, Malito, & Umbach, 2018Bhuta, N., Malito, D. V., & Umbach, G. (2018). Introduction: Of numbers and narratives – Indicators in global governance and the rise of a reflexive indicator culture. In N. Bhuta, D. V. Malito, & G. Umbach (Eds.), The Palgrave handbook of indicators in global governance. London: Palgrave MacMillan.
Biermann, Pattberg, Van Asselt, & Zelli, 2009Biermann, F., Pattberg, P., Van Asselt, H., & Zelli, F. (2009). The fragmentation of global governance architectures: A framework for analysis. Global environmental politics, 9(4), 14–40.
Brankovic, Ringel, & Werron, 2018Brankovic, J., Ringel, L., & Werron, T. (2018). How rankings produce competition: The case of global university rankings. Zeitschrift für Soziologie, 47(4), 270–288.
Chakhovich, & McGoun, 2016Chakhovich, T., & McGoun, E. G. (2016). Why grids in accounting? Critical Perspectives on Accounting, 34(1), 36–59.
Clune, & O’Dwyer, 2020Clune, C., & O’Dwyer, B. (2020). Organizing dissonance through institutional work: The embedding of social and environmental accountability in an investment field. Accounting, Organizations and Society, 85, article in press.
Couldry, & Mejias, 2019Couldry, N., & Mejias, U. (2019). Data colonialism: Rethinking Big Data’s relation to the contemporary subject. Television and New Media, 20(3), 336–349.
Decuypere, & Landri, 2020Decuypere, M., & Landri, P. (2020). Governing by visual shapes: University rankings, digital education platforms and cosmologies of higher education. Critical Studies in Education. Article in press.
Desroisères, 2015Desroisères, A. (2015). Retroaction: How indicators feed back onto quantified actors. In R. Rottenburg, S. E. Merry, S. Park, & J. Mugler (Eds.), The world of indicators. The making of governmental knowledge through quantification (pp. 329–353). Cambridge: Cambridge University Press.
Drucker, 2015Drucker, J. (2015). Humanities approaches to graphical display. Digital Humanities Quarterly, 5(1), 1–15.
Eco, 2009Eco, U. (2009). The infinity of lists. London: MacLehose.
Elsbach, & Kramer, 1996Elsbach, K. D., & Kramer, R. (1996). Members’ responses to organizational identity threats: Encountering and countering the Business Week rankings. Administrative Science Quarterly, 442–476.
Espeland, & Sauder, 2007Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40.
Espeland, & Stevens, 2008Espeland, W. N., & Stevens, M. L. (2008). A sociology of quantification. European Journal of Sociology, 49(3), 401–436.
Esposito, & Stark, 2019Esposito, E., & Stark, D. (2019). What’s observed in a rating? Rankings as orientation in the face of uncertainty. Theory, Culture & Society, 36(4), 3–26.
Fairclough, 1995Fairclough, N. (1995). Critical discourse analysis: Theoretical study of language. Longman.
Ferraro, Etzion, & Gehman, 2015Ferraro, F., Etzion, D., & Gehman, J. (2015). Tackling grand challenges pragmatically: Robust action revisited. Organization Studies, 36(3), 263–390.
Free, Salterio, & Shearer, 2009Free, C., Salterio, S. E., & Shearer, T. (2009). The construction of auditability: MBA rankings and assurance in practice. Accounting, Organizations and Society, 34(1), 119–140.
Fukuda-Parr, 2016Fukuda-Parr, S. (2016). From the millennium development goals to the sustainable development goals: Shifts in purpose, concept, and politics of global goal setting for development. Gender and Development, 24(1), 43–52.
Fukuda-Parr, & McNeil, 2019Fukuda-Parr, S., & McNeil, D. (2019). Knowledge and politics in setting and measuring the SDGs: Introduction to special issue. Global Policy, 10(1), 5–15.
Fyfe, & Law, 1992Fyfe, G., & Law, J. (1992). Introduction: On the invisibility of the visual. In G. Fyfe & J. Law (Eds.), Picturing power: Visual depiction and social relations (pp. 1–14). Routledge.
Gates Foundation, 2019Gates Foundation. (2019). Goalkeepers Report: Examining Inequality 2019. Retrieved from https://www.gatesfoundation.org/goalkeepers/. Accessed on March 1, 2020.
Gatzweiler, & Ronzani, 2019Gatzweiler, M. K., & Ronzani, M. (2019). Prospective sensemaking and thinking infrastructures in a large-scale humanitarian crisis. Research in the Sociology of Organizations, 62, 85–112.
Grek, 2009Grek, S. (2009). Governing by numbers: The PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37.
Grek, 2020Grek, S. (2020). Prophets, saviours and saints: Symbolic governance and the rise of a transnational metrological field. International Review of Education, 66, 139–166.
Grotlüschen, & Buddeberg, 2020Grotlüschen, A., & Buddeberg, K. (2020). PIAAC and the south – Is southering the new othering? Global expansion of dominant discourses on adult literacy. European Journal for Research on the Education and Learning of Adults. Article in press.
Hazelkorn, 2011Hazelkorn, E. (2011). Rankings and the reshaping of higher education: The battle for worldwide excellence. London: Palgrave.
Jandrić, & Loretto, 2020Jandrić, J., & Loretto, W. (2020). Business school space, the hidden curriculum, and the construction of student experience. Management Learning. Article in press.
Jeacle, & Carter, 2011Jeacle, I., & Carter, C. (2011). In TripAdvisor we trust: Rankings, calculative regimes and abstract systems. Accounting, Organizations and Society, 36, 293–309.
Jewett, 2019Jewett, M. (2019). Tableau data day out. Retrieved from www.tableau.com. Accessed on March 1, 2020.
Konkel, 2014Konkel, R. (2014). The monetization of global poverty: The concept of poverty in World Bank history, 1944–90. Journal of Global History, 9(2), 276–300.
Kornberger, 2017Kornberger, M. (2017). The values of strategy: Valuation practices, rivalry and strategic agency. Organization Studies, 38(12), 1753–1773.
Kornberger, & Clegg, 2011Kornberger, M., & Clegg, S. (2011). Strategy as performative practice: The case of Sydney 2030. Strategic Organization, 9(2), 136–162.
Krensky, 2014Krensky, P. (2014). Interactive data visualization: The IT Perspective. Aberdeen Group. Retrieved from http://www.sift-ag.com/wp-content/uploads/AR-Aberdeen-Interactive-Data-Visualization-EN.pdf. Accessed on May 20, 2020.
Kress, 2009Kress, G. (2009). Multimodality: Exploring contemporary methods of communication. London: Routledge.
Lafortune, Gonzalez, & Lonti, 2018Lafortune, G., Gonzalez, S., & Lonti, Z. (2018). Government at a glance: A dashboard approach to indicators. In D. Malito, G. Umbach, & N. Bhuta (Eds.), The Palgrave handbook of indicators in global governance (pp. 207–238). Palgrave Macmillan.
Latour, 1986Latour, B. (1986). Visualization and cognition. Knowledge and Society, 6(6), 1–40.
Le Grand, 2003Le Grand, J. (2003). Motivation, agency, and public policy. Of knights, knaves, pawns & queens. New York, NY: Oxford University Press.
Le Grand, 2007Le Grand, J. (2007). The other invisible hand. Delivering public services through choice and competition. Princeton: Princeton University Press.
Mehrpouya, & Samiolo, 2016Mehrpouya, A., & Samiolo, R. (2016). Performance measurement in global governance: Ranking and the politics of variability. Accounting, Organizations and Society, 55, 12–31.
Merry, 2016Merry, S. E. (2016). The seductions of quantification: Measuring human rights, Gender violence and sex trafficking. Chicago: University of Chicago Press.
Merry, 2019Merry, S. E. (2019). The sustainable development goals confront the infrastructure of measurement. Global Policy, 10, 146–148.
Meyer, Jancsary, Höllerer, & Boxenbaum, 2018Meyer, R. E., Jancsary, D., Höllerer, M. A., & Boxenbaum, E. (2018). The role of verbal and visual text in the process of institutionalization. Academy of Management Review, 43(3), 489–555.
OECD, 2020OECD. (2020). OECD better life index. Retrieved from http://www.oecdbetterlifeindex.org/#/11111111111. Accessed on May 20, 2020.
Our World in Data, 2020aOur World in Data. (2020a). Our audience and coverage. Retrieved from https://ourworldindata.org/coverage. Accessed on October 23, 2020.
Our World in Data, 2020bOur World in Data. (2020b). Global Extreme Poverty. Retrieved from https://ourworldindata.org/extreme-poverty. Accessed on May 23, 2020.
Pollock, & D’Adderio, 2012Pollock, N., & D’Adderio, L. (2012). Give me a two-by-two matrix and I will create the market: Rankings, graphic visualisations and sociomateriality. Accounting, Organizations and Society, 37(8), 565–586.
Pollock, D’Adderio, Williams, & Leforestier, 2018Pollock, N., D’Adderio, L., Williams, R., & Leforestier, L. (2018). Conforming or transforming? How organizations respond to multiple rankings. Accounting, Organizations and Society, 64, 55–68.
Porter, 1995Porter, T. M. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Cambridge: Princeton: Princeton University Press.
Quattrone, 2017Quattrone, P. (2017). Embracing ambiguity in management controls and decision-making processes: On how to design data visualizations to prompt wise judgement. Accounting and Business Research, 47(5), 588–612.
Quijano, 2007Quijano, A. (2007). Coloniality and modernity/rationality. Cultural Studies, 21(2), 168–178.
Ritchie & Rosner, 2018Ritchie & Rosner. (2018). Introducing the Our World in Data ‘Sustainable Development Goal (SDG) Tracker’. Retrieved from https://ourworldindata.org/sdg-tracker. Accessed on March 9, 2021.
Rottenburg, 2009Rottenburg, R. (2009). Far-fetched facts: A parable of development aid. Cambridge: The MIT Press.
Sauder, & Espeland, 2009Sauder, M., & Espeland, W. (2009). The discipline of rankings: Tight coupling and organizational change. American Sociological Review, 74(1), 63–82.
Shore, & Wright, 2015Shore, C., & Wright, S. (2015). Audit culture revisited: Rankings, ratings, and the reassembling of society. Current Anthropology, 56(3), 421–444.
Slager, & Gond, 2020Slager, R., & Gond, J.-P. (2020). The politics of reactivity: Ambivalence in corporate responses to corporate social responsibility ratings. Organization Studies. Article in press.
Snyder, 2020Snyder, J. (2020). Backlash against naming and shaming: The politics of status and emotion. The British Journal of Politics and International Relations. Article in press.
Stark, 2009Stark, D. (2009). The sense of dissonance: Accounts of worth in economic life. Princeton: Princeton University Press.
Stark, 2020Stark, D. (2020). The performance complex. In D. Stark (Ed.), The performance complex: Competition and competitions in social life (pp. 1–28). Oxford: Oxford University Press.
Stiglitz, Sen, & Fitoussi, 2009Stiglitz, J. E., Sen, A., & Fitoussi, J. P. (2009). Report of the Commission on the Measurement of Economic Performance and Social Progress (CMEPSP).
Tableau, 2020aTableau. (2020a). Business intelligence and analytics software – Tableau. Retrieved from https://www.tableau.com/en-gb. Accessed on February 1, 2020.
Tableau, 2020bTableau. (2020b). About tableau. Retrieved from https://www.tableau.com/en-gb/about/mission. Accessed on February 1, 2020.
Tableau, 2020cTableau. (2020c). Whitepapers. Retrieved from https://www.tableau.com/learn/whitepapers, Accessed on February 1, 2020.
UNDP, 2018UNDP. (2018). What Does It Mean To Leave No One Behind? A UNDP Discussion Paper And Framework For Implementation.
Van Erp, 2011Van Erp, J. (2011). Naming without shaming: The publication of sanctions in the Dutch financial market. Regulation & Governance, 5, 287–308.
Wedlin, 2006Wedlin, L. (2006). Ranking business schools: Forming fields, identities and boundaries in international management education. Northampton: Edward Elgar Publishing.
Wodak, & Meyer, 2001Wodak, R., & Meyer, M. (Eds.) (2001). Methods of critical discourse analysis. London: Sage.
World Bank, 2015World Bank. (2015). Piecing together the poverty puzzle. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/30418/9781464813306.pdf. Accessed on February 1, 2020.
Acknowledgments
This paper is part of a project that has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program, under grant agreement No. 715125 METRO (ERC-2016-StG) (“International Organisations and the Rise of a Global Metrological Field,” 2017–2022, PI: Sotiria Grek). In addition, the authors are grateful for the help and advice of Tobias Werron, Leopold Ringel, and two anonymous reviewers who provided helpful comments on earlier drafts. We would also like to express our gratitude to Richard Freeman, Christopher Humphrey, Jakov Jandrić, Brendan O’Dwyer, Paolo Quattrone, Leonid Sokolovskiy, and Marlee Tichenor who provided valuable suggestions and ideas during the writing process. We are also grateful to the participants of various departmental seminars where this study has been presented (Bielefeld University; The University of Edinburgh School of Social and Political Science – SKAPE: Centre for Science, Knowledge and Policy; Alliance Manchester Business School).
- Prelims
- Worlds of Rankings
- The Producers of Rankings
- Beyond Winners and Losers: Ranking Visualizations as Alignment Devices in Global Public Policy
- Stepping into the Spotlight: How Rankings Become Public Performances
- Between Stakeholders and Third Parties: Regulatory Rankings and the Organization of Competition
- The Negotiation, Interpretation and Institutionalization of Rankings
- “Measure of Shame”: Media Career of the Global Slavery Index
- Redefining Achievement: The Emergence of Rankings in American Baseball
- Caring For Numbers: Performing Healthcare Practices Through Performance Metrics In Sweden and The Netherlands
- The Effects of Rankings
- The Garbage Can Model and Organizational Metrics
- Formality versus Flexibility: The Effects of Evaluation Practices on Judging and Performance
- Ranking Strategy: How Organizations Respond to The New Competitive Battlefields
- Index