Contesting commensuration: Public response tactics to performance evaluation of academia

Jonas Gerdin (School of Business, Örebro Universitet, Örebro, Sweden)
Hans Englund (School of Business, Örebro Universitet, Örebro, Sweden)

Accounting, Auditing & Accountability Journal

ISSN: 0951-3574

Publication date: 24 May 2019

Abstract

Purpose

The purpose of this paper is to explore how actors subjected to public performance evaluations may “contest commensuration,” i.e. may seek to influence how such ratings and rankings will be construed among important stakeholders.

Design/methodology/approach

A qualitative study of press releases, and interviews with department heads, is used as a basis for the analysis.

Findings

The empirically derived taxonomy of public responses to a state-initiated performance evaluation of educational programs shows that actors may mobilize an array of commensuration management tactics so as to maintain or improve one’s relative positional status. Such tactics may have at least three different foci, namely, on the comparison object (i.e. on the new grouping of actors), the comparison dimension (i.e. the standardized format for comparison) and the comparison rate (i.e. the rate received), respectively. The authors also find that not only are threats to positional status likely to spur commensuration management tactics, but also the opportunity to exploit a good rate.

Originality/value

The paper augments recent research that has problematized the so-called “reactive conformance thesis” by focusing on how evaluated organizations may directly try to influence external stakeholders through public responses. The study is also one of the first that analytically disentangles how they may skillfully exploit different forms of “plasticity” that are inherent in any type of commensuration.

Keywords

Citation

Gerdin, J. and Englund, H. (2019), "Contesting commensuration", Accounting, Auditing & Accountability Journal, Vol. 32 No. 4, pp. 1098-1116. https://doi.org/10.1108/AAAJ-09-2016-2717

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Jonas Gerdin and Hans Englund

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Publically announced league tables, quality audits, accreditations and performance measurements are now extensively used to assess and rank as diverse organizations as universities (Guthrie et al., 2015; Mingers and Willmott, 2013; Modell, 2003; Northcott and Linacre, 2010), software producers (Pollock and D’Adderio, 2012; Pollock et al., 2018) and leisure and tourist companies (Jeacle and Carter, 2011). A common belief in the research literature is that such public performance evaluations create organizational reactivity (Espeland and Sauder, 2007; Espeland and Stevens, 2008) – a type of response through which evaluated organizations conform to expectations that are embedded in the performance measure (Davies and Petersen, 2005; Gendron, 2008; Gioia and Corley, 2002; Mingers and Willmott, 2013; Parker, 2011; Sauder and Espeland, 2009).

As argued by Espeland and Stevens (1998, 2008) and Espeland and Sauder (2007), an important mechanism generating such reactive responses to public performance evaluations is commensuration. For through transforming qualities into quantities, commensuration “shapes what we pay attention to, which things are connected to other things, and how we express sameness and difference” (Espeland and Sauder, 2007, p. 16), thereby contributing to processes of conformity (Gendron, 2008; Sauder and Espeland, 2009; Willmott, 1995).

However, this “reactive conformance thesis” (Pollock et al., 2018) builds on the assumption that public and externally imposed ratings and rankings are perceived as essentially given (see e.g. Davies and Petersen, 2005). That is, it is assumed that the meanings and implications of performance evaluations cannot be influenced by the actors subjected to the evaluations in question. In this paper, we build on and add to a recent stream of research that has begun to problematize this basic assumption (see e.g. Martins, 2005; Pollock et al., 2018) through showing that there is typically at least some space for “rhetoric maneuvering” as subjects to externally imposed performance evaluations publically comment on the evaluation outcomes. More specifically, we draw upon a study of press releases published in conjunction with a state-initiated evaluation of educational university programs in business administration, to propose a set of response tactics through which representatives from the responsible organizations sought to influence how the ratings per se should be publically construed.

In brief, the taxonomy of so-called “commensuration management tactics” developed in this study (see Table I in Section 5 below) suggests that they may both problematize and/or strategically exploit one or more aspects of the rating device. For example, they could focus on the comparison object (i.e. on the grouping of educational programs in terms of the grade received) in order to signal group (non)membership through inclusion and exclusion rhetorics. They could also seek to (de)value the legitimacy of the comparison dimension as such, and (de)emphasize and selectively justify the significance of a weak/strong comparison rate. Below, we will refer to this type of proactive (rather than reactive) behaviors as “contesting commensurability,” and argue that it was influenced by threats to, or opportunities to improve, the positional status of the subjects of the performance evaluation (i.e. whether organizations responsible for the educational programs came out as a low or high performer).

Through bringing together insights from accounting (Gendron, 2008, 2015; Parker, 2011; Pollock and D’Adderio, 2012; Pollock et al., 2018) with those of organizational studies (Elsbach and Kramer, 1996; Martins, 2005; Wedlin, 2007; Mingers and Willmott, 2013) and sociology of commensuration (Espeland and Sauder, 2007; Espeland and Stevens, 1998, 2008), our study suggests two important contributions. First, it addresses recent calls for more research on “how organizations seek to influence stakeholder audiences directly in an effort to improve their positions in rankings” (Rindova et al., 2017, p. 12, see also Pollock et al., 2018) through identifying a set of commensuration management tactics. That is, rather than considering externally imposed ratings and rankings as essentially given thereby generating behavioral conformity (see e.g. Espeland and Sauder, 2007; Gendron, 2008; Sauder and Espeland, 2009; Willmott, 1995), organizational representatives typically take the opportunity to publically affect the interpretation of both the categorization of, and relations between, the evaluated entities that commensuration engenders.

Second and related, our study adds to the sociology of commensuration (Espeland and Stevens, 1998, 2008; Espeland and Sauder, 2007) through identifying and analytically disentangling different forms of “plasticity” (Briers and Chua, 2001) inherent in any type of commensuration. That is, while commensuration creates the social worlds it represents (Espeland and Sauder, 2007; Espeland and Stevens, 1998, 2008; Miller and Power, 2013), our taxonomy of public responses identifies a number of ways through which actors subjected to performance evaluations may seek to publically (re)define these social worlds.

The remainder of the paper is organized as follows. The next section briefly reviews the extant literature on commensuration, after which we develop the idea that commensuration conveys a certain degree of “plasticity” which can be skillfully exploited by actors subjected to external ratings and rankings. Next, we account for our research context, and methods for data collection and data analysis. After that, we turn to our empirical material and show how the taxonomy of commensuration management tactics emerged during the analyses of data. In a concluding section, we outline our results and contributions.

2. Literature review: commensuration and its effects

Characteristics of commensuration

Various forms of ratings, rankings and performance evaluations constitute a prominent, almost taken-for-granted part of today’s society (Martins, 2005; Rindova et al., 2017; Wedlin, 2007). For example, it would be almost impossible to imagine a school system without the grading of pupils. Likewise, it is difficult to see how governments can control local authorities without different types of summary reports on their financial status.

As suggested in the seminal work by Espeland and Stevens (1998) (see also Espeland and Sauder, 2007; Stevens and Espeland, 2005), such ratings, rankings and performance evaluations are very powerful as they involve commensuration, i.e. they imply “the expression of measurement of characteristics normally represented by different units according to a common metric” (Espeland and Stevens, 1998, p. 315). According to this stream of research, commensuration thus has two important characteristics.

First, it involves a chain of translation (Latour, 1999) where the multiplicity and particularity of organizational reality is reduced into a standardized and single rate or rank that works as a proxy (Rindova et al., 2017), or stand-in (Dambrin and Robson, 2011), for that reality. Publically announced ratings and rankings dealt with in this study are prominent examples, but commensuration comprises all types of efforts to express value quantitatively (Stevens and Espeland, 2005). However, commensuration not only makes particular qualities of the evaluated entities visible, but also makes many, if not most, of their qualities invisible. For through describing entities using a particular format – e.g. by means of a shared metric – commensuration makes everything that does not fit that format irrelevant. Or as argued by Espeland and Stevens (2008, p. 432), commensuration “facilitates a peculiarly modern ontology, in which the real easily becomes coextensive with what is measurable.”

Second, commensuration is inherently relative in character (Espeland and Stevens, 1998; Stevens and Espeland, 2005). For commensuration not only simplifies and unites entities by describing them in a standardized way, it also constructs particular relationships between these entities, i.e. “[i]t creates relations between attributes or dimensions where value is revealed in comparison” (Espeland and Stevens, 1998, p. 317). For example, by ordering entities in terms of hierarchies on a single, universal scale, some of them are constructed as generally better than others. And this is so irrespective of whether such differences may be negligible (Espeland and Sauder, 2007), or whether lower ranked entities may outperform higher ranked ones on other criteria than the one expressed by the chosen scale (see e.g. Elsbach and Kramer, 1996). Accordingly, while commensuration transforms all types of differences into a standardized format, thereby uniting them, it also creates differences between entities. Again, however, these differences are expressed in terms of magnitude on a single scale, i.e. as matters of more/less or better/worse, rather than as matters of kind (Espeland and Stevens, 2008; Stevens and Espeland, 2005).

Commensuration and the “reactive conformity thesis”

As argued by Espeland and Stevens (1998, 2008), Espeland and Sauder (2007) and Sauder and Espeland (2009), commensuration in general, and public ratings and rankings in particular, influence the subjects of such evaluations in many different ways. Although some do not explicitly refer to the notion of commensuration, there seems to be a general consensus among most researchers that public ratings and rankings tend to lead to conformity in organizational behavior. For example, Gioia and Corley (2002, p. 110) observed how rankings pushed organizations to “conform and perform to the rankings criteria,” while Martins (2005, p. 701) noted that “rankings constitute sources of institutional isomorphic pressure on organizations.” Along these lines, Wedlin’s (2007) study suggested that while a ranking of business schools left some room for differences in organizational practices, it nevertheless codified a template for a good business school of high international standard. A template that created isomorphic pressures on organizations to adapt to this role model (see also Mingers and Willmott, 2013; Parker, 2011; Willmott, 1995).

The literature on ratings and rankings has offered a number of explanations as to why commensuration should lead to organizational conformity. For example, Gioia and Corley (2002) suggested that rankings push organizations to adapt to the rankings criteria because such compliance provides access to important resources from key constituents (see also Gendron’s, 2015 discussion about the implications of the paying-off mentality in academia). And along these lines, Espeland and Sauder (2007, p. 13) found that even small differences in rank made law schools adapt to the evaluation criteria “not only to [influence] the reactions of prospective students, but also to other constituents such as trustees, boards of visitors, and alumni, all of whom provide financial and administrative support to the schools.”

It has also been argued that commensuration is generally associated with highly held values of objectivity and rational decision-making (Espeland and Stevens, 2008). And when such depersonalized and public form of knowledge is deemed superior to personalized and local knowledge by different stakeholders (Espeland and Stevens, 1998), it is difficult to be against such “proxies” of organizational performance, at least in the long run. For as Jeacle and Carter (2011) found when exploring the role of the ranking device TripAdvisor, this type of commensuration constitutes an important generator of trust in general, and of system trust in particular (i.e. trust in the ranking devices per se, see also Giddens, 1990).

Another stream of research has stressed the explanatory value of perceived discrepancies between the position assigned to an organization in rankings and the position implied by the organization’s identity in the minds of its top managers (Martins, 2005) for understanding behavioral conformity. Along these lines, for example, Elsbach and Kramer (1996) found that business schools used a number of response tactics so as to address perceived identity threats, i.e. when public rankings called into question the merit of core, enduring and distinctive features associated with their organizations. Similarly, Wedlin (2007) argued that rankings contribute to shaping the international business school field by codifying an organizational template that is used in identity-formation processes of organizations – a template that signifies identities and is formed around perceptions of what is real, good and legitimate (see also Mingers and Willmott, 2013; Parker, 2011).

Finally, Espeland and Sauder (2007) (see also Sauder and Espeland, 2009) suggested two key mechanisms that explain what they refer to as organizational reactivity. Apart from commensuration elaborated on in this study, they show how rankings evoke self-fulfilling prophecies. The premise is, they suggest, that when “subjects change their behavior in response to prediction; prediction then, becomes a new and dynamic factor that changes the conditions under which it comes true” (p. 11, emphasis in original). And along these lines, Espeland and Sauder (2007) found that law schools used previous rankings as the basis for internal resource allocation, or even adopted improved rankings as an explicit organizational goal, thereby reinforcing the legitimacy of and conformance to the ranking criteria.

Hence, as Rindova et al. (2017, p. 14) conclude in a recent comprehensive literature review of the ranking literature, an overall key insight from this research “is that quantification removes and disincentivizes individuation and increases conformity” (see also Gendron, 2008; Mingers and Willmott, 2013; Parker, 2011; Sauder and Espeland, 2009). In fact, even critical management scholars, who are perhaps best equipped to “see through” and resist the conforming effect of commensuration such as journal rankings and citations counting, generally have ambivalent attitudes to it (see e.g. Annisette et al., 2015; Butler and Spoelstra, 2012, 2014).

3. Theory development: contesting commensurability

As noted above, there has also emerged a stream of research that has begun to problematize the “reactive conformity thesis” described in the previous section. For example, Pollock et al. (2018) investigated organizations exposed to multiple, even conflicting rankings and concluded that such a heterogeneous context provides greater room for strategic maneuvering where evaluated organizations deployed a number of response tactics. As they note, “simply conforming would be implausible. An entity reacting equally and indiscriminately to increased and diverse pressures would risk being pulled in different directions” (p. 56).

Other studies have suggested that it is primarily “poor performers” that take measures to conform to the norms embodied in the uniform standards (Chatterji and Toffel, 2010; Elsbach and Kramer, 1996; Martins, 2005). The underpinning premise is that a low rate or rank (or an insufficient increase in rate or rank) may challenge, or even repudiate, the positional status of the evaluated organizations (Elsbach and Kramer, 1996; Espeland and Sauder, 2007; Martins, 2005).

Our study expands on these insights in two important respects. First, while our findings largely corroborate the importance of extant positional status, they also suggest that it is not only threats to such status that prompt public responses to imposed ratings and rankings. Rather, drawing upon Sauder and Espeland’s (2009, p. 74) general characterization of ratings and rankings as “engines of status anxiety,” we find that both high and low performers (henceforth referred to as “winners” and “losers”) are likely to publically react to such evaluations.

Second, this study suggests that “ascribed” positional status as manifested through a particular rate or rank is by no means given. On the contrary, we propose that the constitutive quality of commensuration opens up a space for “rhetoric maneuvering” on the behalf of the evaluated organizations which may be skillfully exploited to redefine the ascribed in- and out-groups (i.e. the groups in which they are categorized as a members, or not a member, according to the performance evaluation). For as argued by Espeland et al., commensuration not only simplifies and standardizes “reality,” it is also crucial for how we enact, categorize and make sense of that reality (Espeland and Stevens, 1998; Espeland and Sauder, 2007). In other words, rather than being neutral and objective representations of “actual circumstances,” ratings and rankings contribute to constructing the very entities that should be held accountable, and for shaping the criteria for what is (not) deemed important and/or (ab)normal in the first place (Miller and Power, 2013; Sauder and Espeland, 2009). Or as formulated by Pollock and D’Adderio (2012, p. 566), ratings and rankings “intervene in shaping the reality they attempt to monitor.”

Based on this constitutive quality, we suggest that any commensuration conveys a certain degree of “plasticity” (see also Briers and Chua, 2001). That is, while rates or ranks indeed convey some “perceived substance” that makes them difficult or even impossible to ignore for reasons described in the above section, their constitutive character also implies that actors subjected to these evaluations may publically contest, even seek to redefine, the entities and relations they contribute to shape so as to improve (or maintain) their relative position within the organizational field of which they are part. In fact, we find that the academic institutions studied here could publically contest one or more aspects of the rating device per se, including the comparison object, comparison dimension, and comparison rate. Before we indulge more deeply in these different types of “public commensuration management tactics”, however, we will briefly describe the empirical context and how we have collected and analyzed the data.

4. Research context and method

Context

As suggested above, commensuration is an inherent part of modern society, and a prominent example is the widespread use of ratings and rankings so as to asses various forms of performance in the university sector (Kallio et al., 2016; Lynch, 2015; Mingers and Willmott, 2013; Wedlin, 2007). In order to investigate how organizational representatives can publically “contest” this type commensurability, i.e. try to influence how performance evaluations should be construed among important stakeholders, we investigated how 26 Swedish academic institutions publically responded to a state-initiated assessment of their educational programs in business administration. At least two characteristics of this context made it particularly useful for investigating how the subjects being rated or ranked may publically contest commensurability.

First, at the time of the study, the Swedish university sector was (and still is) quite subdivided in terms of the positional status of different actors (cf. Elsbach and Kramer, 1996; Martins, 2005; Sauder and Espeland, 2009). One group of academic institutions belonged to the six largest and oldest universities. Within the Swedish community, these business administration departments, together with their counterpart at the privately owned Stockholm School of Economics (SSE), were considered high-esteem and officially referred to themselves as members of the “U6-group/network”[1]. The second category consisted of business administration departments from four new universities (awarded university status around the turn of the millennium) and 13 smaller so-called university colleges. These departments could, in principle, give the same type of educational programs as do those in the U6-group (given that they were awarded the qualification by the Swedish National Agency for Higher Education). However, they were generally considered less esteemed and only disposed a very marginal proportion of the state-funded research budget.

Second, the Swedish Government had changed the overall system for the evaluation of educational performance in the university sector. Inspired by a regime of “new public management involving a proliferating culture of audit, accountability and performativity” (Knights and Clarke, 2014, p. 336; see also Espeland and Sauder, 2007; Kallio et al., 2016), it was decided that assessments should be done by a panel of external, independent reviewers who should focus on the achievement of learning outcomes (Govt. Bill 2009/10:139, 2009, p. 139). Unlike previous assessments that had been primarily focused on preconditions (such as the proportion of teachers having a PhD degree) and extant processes for ensuring a good education (e.g. the existence of internal routines for quality assurance), the new system was thus output oriented and even included a small pecuniary reward for those who received the highest grade. Or more precisely, the expert panel should focus on the quality of students’ bachelor/master theses, and students’ experiences of the educational program in question[2]. And on these grounds, the panel assessed all educational programs and graded them on the scale “inadequate quality,” “high quality” and “very high quality.”

This was a quite dramatic performance evaluation system change. For not only did poorly performing academic institutions risk losing their entitlement to award educational programs rated “inadequate quality” if the problems were not satisfactorily addressed within a specified time limit (Report 2011:3 R, 2011), it also resulted in a rather unorthodox grouping of business administration departments. In fact, several of the departments belonging to the highly esteemed U6-group received the rate “inadequate quality” on one or more of their educational programs, while quite a few of the less highly esteemed departments got the rates “high quality” and “very high quality.” This performance evaluation thus provided us with an empirical context where we could expect interesting dynamics as both winners and losers would take the opportunity to influence how their particular rates were publically construed so as to try to improve or maintain their positional status within the organizational field (Elsbach and Kramer, 1996; Martins, 2005; Sauder and Espeland, 2009).

Data collection

Considering our interest in public responses, our primary source of data was the press releases that all universities/university colleges published almost immediately after the results of the new performance evaluation was announced by the National Agency for Higher Education in mid-May, 2012. These press releases were deemed useful for two reasons. First, they granted us access to firsthand, written data where the subjects of the evaluation themselves formulated how they perceived, and wanted important others to perceive, the outcomes of the evaluation. Second, this performance evaluation was deemed very important by all universities/university colleges in question. This increased the likelihood that they would not only respond promptly and decisively, but would also invest considerate effort into formulating the public response (in the form of quite lengthy press releases). And indeed, 26 press releases were immediately published (one per university or university college) amounting to more than 30 pages of text. In about half of the cases, the signing author of the press releases was a press officer. In the other half of the cases, the author was not explicitly specified.

The press releases typically consisted of two parts. One part provided background information on the evaluation as such (e.g. aim, focus and overall results). The other part consisted of statements about the received grade(s) from one or more representatives of the organizations in question. In the great majority of the cases, the Vice Chancellor, a Faculty Head or the Head of the business school commented on the outcomes of the performance evaluation, which further stresses the gravity of the situation. It was also common that representatives from the business administration departments (e.g. the head of department, a professor, or the director of studies) were quoted in the press releases.

In order to get background information on how the performance evaluation and its outcome was perceived by actors in the business administration field, we also conducted semi-structured interviews with representatives from the business administration departments whose educational programs had been evaluated. These interviews included questions like: Was it an important event? Were the outcomes perceived as reliable and fair? and What were the reactions when they acquired the results regarding their own and other business administration departments?

Overall, we conducted interviews with department heads (or similar positions) from 21 of in total 26 units. The remaining five department heads were not available for an interview, typically because of time limits. On average the interviews lasted around half an hour, and were all but one tape-recorded (in that particular case, the respondent did not want the interview to be recorded).

We also collected complementary material concerning the new performance evaluation system as such. Particularly important in this respect was information issued by the National Agency for Higher Education. Such information sources included Power Point presentations showed at so-called pre-evaluation information meetings, and the Report 2011:3 R (2011), which described the background, aim, design and implementation of the newly developed performance evaluation system.

Data analysis

Drawing upon Ahrens and Chapman (2006) and others (Silverman, 2011; Bazeley, 2013), we analyzed the data through working back and forth between the empirical observations, the extant literature and an emergent taxonomy of public response tactics. However, after having reached a preliminary understanding of the material, we conducted the following more systematic coding of it.

Based on the view of ratings and rankings as “engines of status anxiety” (Sauder and Espeland, 2009; see also Elsbach and Kramer, 1996; Martins, 2005), we first looked for in vivo expressions (see e.g. Bazeley, 2013) in the press releases where actors sought to positively differentiate one’s own organization (in-group) in relation to others (out-group) so as to improve their relative positional status. Such expressions could include that “Close to 40% of all programs in the country are severely criticized. […, but] the result is positive for us” (emphasis in original), and “in comparison with other institutions that have been criticized this is ‘mild.’”

In the second phase of the coding, these in vivo expressions were systematically compared to identify commonalities and differences. A striking commonality was that essentially all actors tried to (re)define how the performance evaluation per se should be publically construed among important stakeholders. This overarching observation led us to develop the notion of “contesting commensurability.”

On a more detailed level, however, our analysis of in vivo expressions suggested a number of differences in terms of how this type of “public contesting” of the performance evaluation was manifested. For example, some actors tried to elaborate on the (new) groupings that the performance evaluation had engendered (e.g. through inclusion and exclusion of group members), while others focused on (de-)emphasizing the significance of the ascribed performance rate as such. This type of differences led us to identify three forms of public contesting, focusing on the “comparison object,” the “comparison dimension,” and the “comparison rate,” respectively.

We also found that these attempts at contesting commensurability differed between so-called winners and losers, i.e. whether educational programs given by a particular business administration department were awarded the grade “inadequate quality” or “high quality.” This led to the idea that positional status emerging from the performance evaluation in terms of winners or losers was important for understanding the different forms of public contesting observed.

Based on the three foci of public contesting and the two types of positional status emerging from the performance evaluation, the different manifestations of public contesting were compared and collapsed into a set of second-order concepts referred to as “commensuration management tactics.” In total, 12 different types of commensuration management tactics were identified (see Table I in Section 5).

In a final step of the analysis, these aggregate findings were further contextualized and validated by our other sources of data. For example, our interviews with department heads further stressed the importance of the pre-study groupings of the academic community for understanding the responses to the performance evaluation (e.g. in terms of “we were very surprised to find [the large U6-University NN] among those who failed”). Related to this, the interviews also attested the importance of improving, or at least prevent deterioration of, one’s organization’s positional status in the field (e.g. “we don’t want to be perceived as ‘failed’ […] it’s a matter of long-term survival”).

As a means of further validating the emergent findings, we conducted yet another analysis of press releases that were issued when the educational programs in sociology at Swedish universities were assessed some weeks later. Overall, we were able to identify the larger part of the commensuration management tactics found in the focal coding of the business administration field, which suggests reasonable transferability of findings to other, similar contexts.

5. Empirical findings: a taxonomy of commensuration management tactics

As suggested above, the public announcement of the new performance evaluation changed the since long established grouping of academic institutions within the business administration field, at least temporarily. Indeed, five of the educational programs in the very high quality group were given by members of the U6-group, but no less than three programs were given by so-called university colleges. And this latter result was commented by one of the interviewees in terms of; “when we saw University College NN on the list, we were really surprised.” But the same interviewee also continued by saying “However, even more attention was devoted to those who ‘failed’ [i.e. those who were part of the inadequate quality group]. Take University NN, for example, I mean […] that is one of the largest universities in Sweden.” Along the same lines another interviewee noted with a certain amount of pleasure that the “[U6-university NN] is the only university in Sweden that still has the highest accreditation […] and they failed on five of eight educational programs.” And in fact, half of the U6 universities received the rate inadequate quality on two or more educational programs in business administration. This meant that between 33 percent and fully 60 percent of the total number of programs given by these three U6 universities were considered as not fulfilling all of the learning outcomes stipulated in the Swedish Higher Education Ordinance.

The publication of such largely unexpected results is interesting for two reasons. First, both losers (i.e. that received the rate “inadequate quality” on one or more of their educational programs) and winners (who received the rate “high quality” or “very high quality”) found the performance evaluation as critical. For instance, one loser said that “we take this very seriously,” while a winner belonging to a traditionally high-esteem institution argued that “if the results had been otherwise [i.e. other than very high quality], we would have experienced ‘rough times’ […] but now we just breathe a sigh of relief.” Along these lines, we also find that both winners and losers applied commensuration management tactics which suggests that such tactics may emerge not only from perceived threats to positional status (Elsbach and Kramer, 1996; Martins, 2005), but also from actors perceiving the opposite situation.

Second, the perceived significance of the performance evaluation prompted all universities/university colleges studied to immediately publish quite extensive press releases where they commented on the outcomes. And, we find that they typically did so in manners that contested commensurability. That is, rather than considering the imposed ratings of their educational programs as essentially given, they skillfully exploited the room for rhetoric maneuverings enabled by constitutive and “plastic” qualities of commensuration so as to improve (or maintain) their positional status within the organizational field. Specifically (and again), we found that these so-called commensuration management tactics could have three different foci relating to:

  1. the comparison object (i.e. tactics focused on the grouping of academic institutions imposed on them through the performance evaluation);

  2. the comparison dimension as such (i.e. tactics focused on the particular aspect of performance emphasized in the evaluation, in our case, the attainment of stipulated outcome quality of educational programs); and

  3. the comparison rate per se (i.e. tactics focused on the actual outcome level granted by the performance evaluation).

Table I illustrates the emergent taxonomy of commensuration management tactics structured around the positional status emerging from the evaluation in terms of winners and losers, and foci of commensuration management tactics in terms of comparison object, dimension and rate, respectively. Before going into details, however, it should be noted that the tactics may be closely interrelated empirically. In fact, one and the same expression (in the press releases) could include a mixture of several tactics. For example, an actor could emphasize the significance of a rate as such (Tactic 9) through rhetorically seeking to include one’s own organization with other, traditionally higher valued organizations (Tactic 1). Accordingly, the tactics are only analytically separable.

Commensuration management tactics focused on the comparison object

As argued in Section 2, a basic quality of commensuration in the form of ratings and rankings is that it contributes to constituting entities and their relations (Espeland and Sauder, 2007; Espeland and Stevens, 1998, 2008; see also Miller and Power, 2013). Furthermore, this constituting the quality of commensuration implies that there is at least some space for “rhetoric manoeuvring” where subjects to performance evaluations can seek to publically (re)define how they are grouped together with other organizations (referred to as in-group) and the relations to other groups (out-group). Again, based on the positional status anxiety hypothesis (Sauder and Espeland, 2009), we would expect that actors seek to positively differentiate one’s own organization from others in order to enhance, or at least prevent a deterioration of, its relative positional status within the organizational field of which it is part.

Our case is interesting in this respect insofar as the state-initiated performance evaluation grouped business administration departments in a way that was largely orthogonal with the traditional dichotomization of the sector (see Section 4). And given that this new grouping was perceived as threating for some actors (losers), and as a window of opportunity for others (winners), we find that it sparked two principal forms of tactics relating to the comparison object as such. Specifically, we find that actors would typically mobilize the evaluation as a means to signal (non)membership, below referred to as inclusion (see Cells 1 and 3 in Table I) and exclusion tactics (Cells 2 and 4). As suggested by the table, such tactics would also often imply that actors sought to minimize or maximize the number of members in the new in-group or out-group, respectively, depending on the positional status of the organizations in question and what they wanted to achieve.

One expression of the inclusion tactic in Cell 1 was when traditionally less esteemed actors that came out well in the performance evaluation sought to associate themselves with traditionally high-status, even prototypical, members who were also categorized in the same group. For example, one university college associated themselves with the prestigious Stockholm School of Economics (SSE); “It is only [we] who together with SSE and [University College NN] have received the rate very high quality in the subject Business Administration on the bachelor level” (press release). Moreover, this university college also sought to minimize the new high-status group through specifying “the bachelor level” (see Minimization tactic in Cell 1). For again, several more actors received the grade very high quality on one or more of their educational programs (e.g. on the masters programs), but through delimiting the social comparison to a particular program, the new in-group became even smaller and thereby more exclusive. Another example of this type of “in-group minimization tactic” was used by another university college which solely focused their rhetoric on one-year master programs; “Of 22 one-year master programs in Sweden, only three get the highest rate […] and [we] are one of them” (press release).

However, also many losers mobilized an inclusion tactic – but in this case to mitigate a deterioration of their positional status (see Cell 3 in Table I). One way of doing this was to highlight that also traditionally high-esteem actors (typically U6-members) were part of the new inadequate quality group. For instance, one university college stressed that “Among them who do not fulfill the requirements, there are both university colleges and universities” (press release), while another even explicitly listed all three U6-members (and a number of other academic institutions) “who have also been questioned” (press release).

Parallel to the minimization tactic mobilized by some winners (i.e. where the number of members in the new successful in-group is minimized), we also find substantial evidence of losers seeking to maximize the number of members in the inadequate quality group (see Cell 3). The by far most common way of doing this was to emphasize that “almost 40% of the country’s educational programs get the rate inadequate quality” (press release from a university college). Another way was to focus the presentation of results on business administration departments rather than on educational programs (which was the actual unit of assessment). Because then the percentage of “failures” could be raised from 40 to 50 percent; “The expert group has examined students’ theses […] from 26 universities and found shortages at 13 of these” (press release). In other words, many losers sought to diminish the negative effects of being part of the new low-status group through stressing that one’s own organization was in “good company” (see also the discussion about Tactic 11 below, implying that losers sought to de-emphasize significance of the comparison rate as such).

Notably, a parallel type of rhetoric was also mobilized by many winners. However, they typically used it to enhance their positional status within the academic community through excluding and distancing themselves from the less esteemed out-group(s) (cf. the Exclusion tactic in Cell 2 in Table I). For example, one U6-university sought to reaffirm its position as a winner within the Swedish community through quite harshly commenting on both the quality and the large size of the new low-status group (cf. the Maximization of out-group tactic depicted in Cell 2):

Close to 40% of all programs in the country are severely criticized. […, but] the result is positive for us, in particular when you consider the high proportion of questioned [programs]. At the same time it is worrying that one of the largest educations in Swedish academia gets such a [weak] result. But it is very good news that our educations in Business Administration without doubt have a high quality.

(press release, emphasis in original)

As suggested by Cell 4 in Table I (i.e. when actors mobilize the performance evaluation to signal non-group membership), an exclusion tactic may also be used by losers. That is, as a means of maintaining one’s relative positional status, actors may exclude and distance themselves from other actors in the new grouping imposed on them by the performance evaluation. One common way of conducting such downward social comparison (cf. Elsbach and Kramer, 1996) was to emphasize that one’s own organization nevertheless is better than other relevant comparison objects in the new low-status group:

We have received criticism on one [evaluation] criterion per education of eleven in total. With respect to all other criteria [our] educations are granted high quality. In comparison with other institutions that have been criticized this is mild [suggesting one’s own organization positively differentiates from others in the new, imposed low-status group].

(press release, emphasis added)

All in all then, our data suggest that both winners and losers may skillfully enhance or seek to “save” their positional status through mobilizing a number of tactics focused on the performance evaluation comparison object as such. Specifically, we find that they typically mobilized the performance evaluation to signal group (non-)membership through a variety of inclusion and exclusion tactics. Depending on their positional status (such as winners or losers, traditional position, etc.) and what they wanted to achieve (e.g. to pass from a low-status group into a more esteemed and/or to keep some actors out of new in-group), they would also try to rhetorically minimize and maximize in-/out-groups.

Commensuration management tactics focused on the comparison dimension

As suggested by the mid-column in Table I, a number of commensuration management tactics were also focused on the comparison dimension as such. This meant that actors rhetorically sought to (de-)legitimize the focus on educational outcomes which dominated the newly introduced performance evaluation system (Report 2011:3 R, 2011).

If we start in Cell 5, one obvious tactic used by many winners was to applaud the government’s new grip to try to evaluate the degree of attainment of the learning objectives stipulated in the Higher Education Ordinance (as opposed to the previous system focusing primarily on conditions and processes, respectively, for ensuring high quality, see Section 4). For example, one interviewee whose department came well out of the performance evaluation said that “the focus on bachelor and master theses was essentially very smart.” Likewise, another interviewee thought that “the new performance evaluation system is a step forward,” although s/he also argued that the evaluation of students’ theses “could be complemented with other output-oriented quality criteria.”

However, our data also suggest that winners not only sought to legitimize the comparison dimension as such, but also tried to construct the overall results of the performance evaluation as fact-like (cf. Chua, 1995; Power, 2004). Specifically, we identify two interrelated ways of achieving this, namely, through ambiguity reduction and selective verification, respectively (see Cell 6 in Table I). The first-mentioned tactic sought to attest the validity of the results, e.g. through stating that “all members of the expert panel [who did the evaluation] unitedly stand behind the results published today” (press release, emphasis added). Other expressions of ambiguity reduction with respect to the results are (press releases, emphases added):

Our educational programs have unhesitatingly high quality,

We’re very happy with the results, which recognise the fact that [university college NN] is in the top when it comes to Business Administration in Sweden, or

[O]ur educations in Business Administration without doubt have a high quality.

The second commonly used way of rhetorically “boosting” the validity of the comparison dimension and the overall results was to selectively link them to other, previously conducted performance evaluations (see the selective verification tactic in Cell 6). One way of doing this was to make temporal comparisons. For example, one press release stated that “It is really nice that we also in this evaluation cycle win the gold medal,” while another one argued that “the Swedish National Agency for Higher Education has today awarded [Name of university college] yet a golden star, whose popular business program gets the highest possible mark.” Yet other actors sought to make the focal performance measurement fact-like through comparing it with quality assessments undertaken by other evaluators. As one actor put it, “we are very happy since yet another independent review shows that our priorities and strategic choices have been correct” (press release).

As suggested in Table I, the commensuration management tactics mobilized by losers largely mirror those applied by winners. Notably, however, very few losers sought to de-legitimize the comparison dimension as such. That is, almost no one openly problematized the evaluation of educational output as such (although, indeed, quite a few criticized the sole use of bachelor/master theses to make the overall judgement of educational output quality). Instead, many losers sought to deflect attention of a poor result through highlighting alternative comparison dimensions (see Cell 7, cf. Elsbach and Kramer, 1996). In effect, many actors did this through emphasizing (alternative) organizational strengths that were not explicitly valued in the performance evaluation. For example, one U6-university suggested both in the press release and in the follow-up interview that they had “research of world-class” and that this was not really reflected in the new measurement of performance. Other losers argued that they first of all looked upon themselves as developing skillful, reflexive, and not least, employable practising professionals. For example, one actor argued that “[this program] is developed to increase the number of new business ventures in the country [and therefore] it does not really match the focus of the new evaluation system” (press release). In a similar vein, other actors stressed that “our educations are focused on the professions,” and that “[a key feature of our educations] is a strong cooperation with the region’s working life [and accordingly], our students are very popular on the job market” (press release).

As suggested by Tactic 8 in Table I, however, many involuntary members of the new low-status group also tried to maintain their relative positional status through explicitly questioning the accuracy and validity of the performance measurement. That is, while they (again) hardly criticized the comparison dimension as such (i.e. the evaluation of educational outcomes), they problematized its fact-like status. And largely parallel to winners, they did so by means of two types of tactics, namely, ambiguity enhancement and selective comparisons (Cell 8). Although these two are indeed interrelated, ambiguity enhancement tactics refer to actors seeking to problematize the focal measurement as such. For example, one university college noted that “the result is not clear-cut” and continued through quoting the evaluation report issued by the Swedish National Agency for Higher Education; “there is no evidence of significant shortcomings regarding the education’s content, structure and forms for examination” (press release). Similarly, another actor argued that “a minor shortage in one area can severely affect an education that in all other respects has high quality” (press release), thereby suggesting that the evaluation created seemingly large differences between groups when in fact they are very small (cf. Espeland and Sauder, 2007).

The tactic Selective comparison (Cell 8), in turn, implies that actors problematized the focal performance evaluation through selectively comparing it with other, more successful evaluations. Such tactic could be expressed in terms of “One year ago the same authority awarded [our university] the qualification to give [name of educational program […] and] at that time, the overall judgement was that we fulfilled all quality requirements” (press release). Likewise, two U6 universities stressed in their press releases that “the program in Business Administration does have high quality which has been confirmed in other assessments [link to webpage],” and “we have our own quality assurance system which we trust [and] we score well on these, other criteria,” respectively.

In conclusion then, we identify a number of commensuration management tactics that were directed toward the comparison dimension as such, i.e. toward the very format used to “transform qualities into quantities, difference into magnitude” (Espeland and Stevens, 1998, p. 316). Winners typically strived to enhance their positional status through “establishing” the validity and reliability of their successful outcome of the performance evaluation. Specifically, they tried to legitimize the performance evaluation through explicitly supporting the comparison dimension, and/or through constructing the results as fact-like through ambiguity reduction and selective verification tactics, respectively. In contrast, many losers sought to de-emphasize the importance of the comparison dimension through highlighting alternative organizational qualities (strengths) not valued in the performance evaluation. They also tried to problematize the performance evaluation’s fact-like status through ambiguity enhancement, and selective comparisons with other, more successful evaluations.

Commensuration management tactics focused on the comparison rate

Table I suggests that commensuration management tactics may not only focus on the comparison object (i.e. the new groupings of actors) and comparison dimension (i.e. educational output quality and the way it is measured), but also on the comparison rate as such (i.e. the attained performance level). To begin with, Cell 9 in Table I shows that the winners typically sought to emphasize the significance of the rate as such through results amplification. This tactic was manifested in various ways, but a common one was to stress that the successful rate referred to a core/major activity within the department (as opposed to a marginal and peripheral one). Examples of such expressions in the press releases include:

This is one of the most important and popular programs at the university,

It is the single largest program, and

We are happy that the large amount of students at the bachelor level attains such a good result.

In contrast, many losers tried to de-emphasize the significance of a low rate through relativizing and toning down the results (see Cell 11). A very common way of doing this was to argue that the results were “not relevant any more.” Along these lines, for example, one U6-member stated that “I can conclude that the critical judgements partly concern one-year masters that are now being phased out, partly concern four master programs of which one has been phased out and two have been significantly changed since the performance evaluation” (press release). Other attempts of relativizing a weak rate was to position one’s result in a more successful context, thereby constructing it merely as a “bump in the road.” Examples of such statements from the press releases include that, “the number of students have increased steadily over the years,” “[we] have always had a high number of admissions to our programs,” or to stress that the low rate does not refer to the more prestigious “[NN] program which we know has high quality.”

We also find that actors selectively tried to justify/explain a high and low rate, respectively. As far as winners are concerned, they typically mobilized two tactics (see Cell 10). The first one was to stress proactivity. For example, one actor argued that “the high quality is the result of the long-term efforts we have undertaken,” while another stated that “this is the result of active and continuous quality enhancing work on our educations.” The second tactic in Cell 10 implies that many winners justified/explained the high rate through explicitly linking it to core organizational attributes. Examples from the press releases include that actors could ascribe the success to attributes such as “liberal education,” “multidisciplinary educations,” “a practitioner focus,” and “internationalization” (where the last-mentioned actor further stressed this core attribute through writing the press release in English).

Equally interesting is, however, that losers would typically do the opposite (cf. Cell 12 in Table I). That is, they oftentimes tried to stress that the performance evaluation did not show “who they really were.” And this was done through emphasizing exceptionality. For instance, one university college stressed that the failing programs were “not pure educations in Business Administration, but combined with other subjects and also directed towards practicing professionals [as opposed to more traditional academic programs]” (press release). In a similar vein, another actor emphasized exceptionality through arguing that “this particular program consisted of a high portion of foreign students without previous experiences of writing theses [suggesting that the ‘normal’ programs had high academic quality]” (press release).

So to conclude then, our empirical evidence suggests that many actors sought to enhance (or prevent a deterioration) of their positional status through commenting on the comparison rate as such. In so doing, winners typically tried to amplify the significance of a high rate, while many losers would do the opposite (above referred to as results relativizing). We also find that winners oftentimes sought to justify the rate through stressing proactivity and also to ascribe the perceived success to core qualities/strengths of their organization. In contrast, losers typically stressed exceptionality, and through doing this, suggested that the disappointing comparison rate did not show “who they really were.”

6. Conclusions and contributions

Public performance evaluations have proliferated in the last three decades and are now permeating the environment of many types of organizations (Jeacle and Carter, 2011; Pollock and D’Adderio, 2012; Pollock et al., 2018; Rindova et al., 2017). Research on such ratings and rankings has traditionally been focused on the consequences of commensuration. And an overall conclusion has been that commensuration typically feeds and forms processes of reactive conformance (see also Bogt and Scapens, 2012; Espeland and Sauder, 2007; Gendron, 2008, 2015; Martins, 2005; Parker, 2011; Sauder and Espeland, 2009). More recently, however, it has been argued that while “[s]cholars typically present rankings as a materialised institution patterning action which organisations have little choice but to bend to […, this] no longer seems to suffice in the case of more plural systems” (Pollock et al., 2018, pp. 65, 66, emphasis added). The premise is, Pollock et al. argue, that multiple rankings foster greater interplay and entanglements between ranked organizations and their environments.

Our inductive study extends these insights by suggesting that also single performance evaluations may spark a number of public response tactics through which assessed organizations seek to (re)negotiate (Pollock et al., 2018) how the evaluation in question should be publically construed. We refer to them as “commensuration management tactics” as they all problematize and/or exploit the room for “rhetoric maneuvering” that is inherent in (any) commensuration. For the constitutive character of commensuration not only contributes to shaping various types of entities and their relations (Espeland and Stevens, 1998, 2008; Stevens and Espeland, 2005; see also Miller and Power, 2013), it also leaves some room for interpretation as to how these entities and relations should be publically construed. A room that organizations subjected to performance evaluations may skillfully exploit in a number of ways in order to affect how a particular rate or rank is construed among important stakeholders.

A taxonomy of commensuration management tactics

Overall, our taxonomy of commensuration management tactics (see Table I above) suggests that: these tactics may have at least three different foci related to the rating device as such, and that they are typically mobilized by both losers and winners, although in slightly different ways. More specifically, our findings suggest that actors may try to affect how imposed ratings and rankings are publically construed in the following principal ways. First, actors may elaborate on the comparison object, i.e. on the groupings of entities and the relations between these that commensuration engenders (see e.g. Tactics 1–4 in Table I). For example, they may both seek to expand and contract the boarders of groupings to signal who they (do not) want to be associated with (above referred to as inclusion and exclusion tactics, and maximization and minimizations tactics, respectively). Again, winners often tried to minimize the size of the in-group thereby making it even more exclusive, while many losers sought to maximize the new in-group so as to show that many others, even traditionally high-esteem actors were also part of this unwanted low-status group (see also Elsbach and Kramer, 1996).

However, actors also tried to create new groups all together. A typical example is when one group as defined by the performance evaluation is subdivided into two subgroups by the subjects of this evaluation – one higher esteemed in-group, and one lower esteemed out-group. For example, some actors who were awarded the rate inadequate quality on a particular educational program could argue that they (new in-group) only failed on a single evaluation criterion, while others (new out-group) failed on several criteria.

Second, actors may elaborate on the comparison dimension. As illustrated by Tactics 5 and 7, for example, one way of doing this is to elaborate the standardized format for comparison per se by reintroducing qualities not deemed relevant by the commensuration in the first place. For example, many winners publically ascribed their success to important qualities/strengths of their organizations, while many losers highlighted alternative and more beneficial comparison dimensions. This way of trying to affect how the performance evaluation was publically construed, thus implicitly contested commensuration all together. And actors did so through demonstrating that differences between organizations should not only be seen as a matter of magnitude on a single scale, but also as a matter of kind (cf. Espeland and Stevens, 2008; Stevens and Espeland, 2005).

Another way of contesting the comparison dimension was to elaborate on the representational validity/correctness of the groupings and the relations between these that commensuration engenders (see Tactics 6 and 8). Again, winners would typically seek to construe a successful rate as fact-like, while losers would oftentimes question its validity through selective comparisons with other, more successful ratings and rankings.

Third and finally, actors may elaborate on the significance of the comparison rate (see Tactics 9–12). For example, many winners tried to increase the significance of a high rate through results amplification tactics and through stressing proactivity. In contrast, many losers tried to relativize the significance of a poor rate by arguing that the “results are not relevant any more,” or through emphasizing exceptionality (i.e. that the program(s) that received the rate inadequate quality constituted a peripheral part of the department’s education activities).

In conclusion then, we find that actors may try to contest externally imposed rates or ranks through publically elaborating on the comparison object, the comparison dimension and comparison rate, respectively. We also find that not only are threats to positional status likely to spur decisive responses (Chatterji and Toffel, 2010; Elsbach and Kramer, 1996; Martins, 2005), but also the opportunity to exploit a good rate. In fact, most of the tactics used by actors with poor rates have a mirror-image variant used by those coming well out of the rating.

Contributions

Through analyzing empirical material on public responses to an imposed performance evaluation, and integrating insights from several literatures (Elsbach and Kramer, 1996; Gendron, 2008, 2015; Espeland and Sauder, 2007; Parker, 2011; Pollock and D’Adderio, 2012; Pollock et al., 2018; Wedlin, 2007; Mingers and Willmott, 2013), our study suggests two important contributions.

First, we respond to recent calls for studies that focus more specifically on “the strategic actions that organizations can take to influence their ranking positions” (Rindova et al., 2017, p. 12). We also take account of Pollock et al.’s (2018) call for more research that problematizes what they refer to as the reactive conformance thesis which characterizes most research not only in accounting (see e.g. Bogt and Scapens, 2012; Gendron, 2008; Parker, 2011), but also in organizational studies (Gioia and Corley, 2002; Mingers and Willmott, 2013; Wedlin, 2007) and sociology of commensuration (Espeland and Sauder, 2007; Sauder and Espeland, 2009). For as Pollock et al. (2018, p. 57) argue, “scholars have largely overlooked the possible range of alternative responses.”

Addressing these calls, our paper proposes an empirically derived taxonomy of commensuration management tactics that provides analytical resources for the study of public responses to externally imposed ratings and rankings. That is, our taxonomy identifies a number of ways through which actors subjected to such performance evaluations may try to affect how imposed rates or ranks are construed among important stakeholders. In contrast to the recent work of Pollock et al. (2018), however, our study suggests that such proactive behavior may not only emanate from organizations being exposed to multiple rankings (enabling them to “see” and “reflect” upon different and often conflicting responses asked of them), but also from single performance evaluations.

Second, and related to this finding, this study is one of the first (to our knowledge) that elaborates on how the constitutive quality of commensuration makes it somewhat “plastic” (Briers and Chua, 2001). That is, while externally imposed ratings and rankings contributes to constituting entities and their relations (Espeland and Stevens, 1998, 2008) which are difficult to resist (Espeland and Sauder, 2007; Gendron, 2008; Parker, 2011; Sauder and Espeland, 2009), our study does suggest that their plasticity creates some room for public rhetoric maneuvering on the behalf of the evaluated subjects. Rhetoric maneuvering that could focus on the comparison object, the comparison dimension, and the comparison rate, respectively, as means of improving or maintaining their organization’s relative positional status.

However, this type of research on public responses to ratings and rankings is still in its infancy and more efforts are required to explore if and how such commensuration management tactics actually have an effect on how external stakeholders perceive the relative status of evaluated organizations. We also lack detailed empirical and theoretical accounts of how such processes of (re)negotiation may emerge over time – an interesting and rewarding task for further research on the commensuration management tactics developed in this study.

Emergent taxonomy of commensuration management tactics

Notes

1.

Note that there are also a few more academic institutions considered very high-esteem, not least the Medical University Karolinska Institutet. However, since these actors do not give any educational programs in the Business Administration area focused here, they will not be included in the analysis below.

2.

Indeed, the business administration departments were also obliged to undertake a self-evaluation of their educational programs (including a site visit by the expert panel). However, the main purpose of this self-evaluation was to provide further information about the educational outcomes, although they could also comment on specific conditions that should be taken into account when evaluating the outcomes (e.g. the possible effect of substantially differing minimum entry qualifications between universities) (homepage, Swedish National Agency for Higher Education).

References

Ahrens, T. and Chapman, C.S. (2006), “Doing qualitative field research in management accounting: positioning data to contribute to theory”, Accounting, Organizations and Society, Vol. 31 No. 8, pp. 819-841.

Annisette, M., Cooper, C. and Gendron, Y. (2015), “Living in a contradictory world: CPAs admission to SSCI”, Critical Perspectives on Accounting, Vol. 31, September, pp. 1-4.

Bazeley, P. (2013), Qualitative Data Analysis: Practical Strategies, Sage, Coydon.

Bogt, H.J.T. and Scapens, R.W. (2012), “Performance management in universities: effects of the transition to quantitative measurement systems”, European Accounting Review, Vol. 21 No. 3, pp. 1-47.

Briers, M. and Chua, W.F. (2001), “The role of actor-networks and boundary objects in management accounting change: a field study of an implementation of activity-based costing”, Accounting, Organizations and Society, Vol. 26 No. 3, pp. 237-269.

Butler, N. and Spoelstra, S. (2012), “Your excellency”, Organization, Vol. 19 No. 6, pp. 891-903.

Butler, N. and Spoelstra, S. (2014), “The regime of excellence and the erosion of ethos in critical management studies”, British Journal of Management, Vol. 25 No. 3, pp. 538-550.

Chatterji, A.K. and Toffel, M.W. (2010), “How firms respond to being rated”, Strategic Management Journal, Vol. 31 No. 9, pp. 917-945.

Chua, W.F. (1995), “Experts, networks and inscriptions in the fabrication of accounting images: a story of the representation of three public hospitals”, Accounting, Organizations and Society, Vol. 20 Nos 2/3, pp. 111-145.

Dambrin, C. and Robson, K. (2011), “Tracing performance in the pharmaceutical industry: ambivalence, opacity and the performativity of flawed measures”, Accounting, Organizations and Society, Vol. 36 No. 7, pp. 428-455.

Davies, B. and Petersen, E.B. (2005), “Neoliberal discourse in the academy: the forestalling of (collective) resistance”, Learning and Teaching in the Social Sciences, Vol. 2 No. 2, pp. 77-98.

Elsbach, K.D. and Kramer, R.M. (1996), “Members’ responses to organizational identity threats: encountering and countering the business week rankings”, Administrative Science Quarterly, Vol. 41, September, pp. 442-476.

Espeland, W.N. and Sauder, M. (2007), “The reactivity of rankings: how public measures recreate social worlds”, American Journal of Sociology, Vol. 113 No. 1, pp. 1-40.

Espeland, W.N. and Stevens, M.L. (1998), “Commensuration as a social process”, Annual Review of Sociology, Vol. 24 No. 1, pp. 313-343.

Espeland, W.N. and Stevens, M.L. (2008), “A sociology of quantification”, European Journal of Sociology/Archives Européennes de Sociologie, Vol. 49 No. 3, pp. 401-436.

Gendron, Y. (2008), “Constituting the academic performer: the spectre of superficiality and stagnation in academia”, European Accounting Review, Vol. 17 No. 1, pp. 97-127.

Gendron, Y. (2015), “Accounting academia and the threat of the paying-off mentality”, Critical Perspectives on Accounting, Vol. 26, February, pp. 168-176.

Giddens, A. (1990), “Structuration theory and sociological analysis”, in Clark, J., Modgil, C. and Modgil, S. (Eds), Anthony Giddens: Consensus and Controversy, Falmer Press, London, pp. 297-315.

Gioia, D.A. and Corley, K.G. (2002), “Being good versus looking good: business school rankings and the Circean transformation from substance to image”, Academy of Management Learning & Education, Vol. 1 No. 1, pp. 107-120.

Govt. Bill 2009/10:139 (2010), “Regeringens proposition: Fokus på kunskap – kvalitet i den högre utbildningen”, pp. 1-41.

Guthrie, J., Parker, L.D. and Dumay, J. (2015), “Academic performance, publishing and peer review: peering into the twilight zone”, Accounting, Auditing & Accountability Journal, Vol. 28 No. 1, pp. 2-13.

Jeacle, I. and Carter, C. (2011), “In TripAdvisor we trust: rankings, calculative regimes and abstract systems”, Accounting, Organizations and Society, Vol. 36 No. 4, pp. 293-309.

Kallio, K.-M., Kallio, T.G.J., Tienari, J. and Hyvönen, T. (2016), “Ethos at stake: performance management and academic work in universities”, Human Relations, Vol. 69 No. 3, pp. 685-709.

Knights, D. and Clarke, C.A. (2014), “It’s a bittersweet symphony, this life: fragile academic selves and insecure identities at work”, Organization Studies, Vol. 35 No. 3, pp. 335-357.

Latour, B. (1999), Pandora’s Hope: Essays on the Reality of Science Studies, Harvard University Press, Cambridge, MA.

Lynch, K. (2015), “Control by numbers: new managerialism and ranking in higher education”, Critical Studies in Education, Vol. 56 No. 2, pp. 190-207.

Martins, L.L. (2005), “A model of the effects of reputational rankings on organizational change”, Organization Science, Vol. 16 No. 6, pp. 701-720.

Miller, P. and Power, M. (2013), “Accounting, organizing and economizing: connecting accounting research and organization theory”, Academy of Management Annals, Vol. 7 No. 1, pp. 557-605.

Mingers, J. and Willmott, H. (2013), “Taylorizing business school research: on the ‘one best way’ performative effects of journal ranking lists”, Human Relations, Vol. 66 No. 8, pp. 1051-1073.

Modell, S. (2003), “Goals versus institutions: the development of performance measurement in the Swedish university sector”, Management Accounting Research, Vol. 14 No. 4, pp. 333-359.

Northcott, D. and Linacre, S. (2010), “Producing spaces for academic discourse: the impact of research assessment exercises and journal quality rankings”, Australian Accounting Review, Vol. 20 No. 1, pp. 38-54.

Parker, L.D. (2011), “University corporatisation: driving redefinition”, Critical Perspectives on Accounting, Vol. 22 No. 4, pp. 434-450.

Pollock, N. and D’Adderio, L. (2012), “Give me a two-by-two matrix and I will create the market: rankings, graphic visualisations and sociomateriality”, Accounting, Organizations and Society, Vol. 37 No. 8, pp. 565-586.

Pollock, N., D’Adderio, L., Williams, R. and Leforestier, L. (2018), “Conforming or transforming? How organizations respond to multiple rankings”, Accounting, Organizations and Society, Vol. 64, January, pp. 55-68.

Power, M. (2004), “Counting, control and calculation: reflections on measuring and management”, Human Relations, Vol. 57 No. 6, pp. 765-783.

Report 2011:3 R (2011), “The Swedish national agency for higher education’s quality evaluation system 2011-2014”, Högskoleverkets rapportserie, Stockholm.

Rindova, V.P., Martins, L.L., Srinivas, S.B. and Chandler, D. (2017), “The good, the bad, and the ugly of organizational rankings: a multidisciplinary review of the literature and directions for future research”, Journal of Management, Vol. 44 No. 6, pp. 2175-2208.

Sauder, M. and Espeland, W.N. (2009), “The discipline of rankings: tight coupling and organizational change”, American Sociological Review, Vol. 74 No. 1, pp. 63-82.

Silverman, D. (2011), Interpreting Qualitative Data: A Guide to the Principles of Qualitative Research, Sage, London.

Stevens, L.S. and Espeland, W.N. (2005), “Commensuration”, in Kempf-Leonard, K. (Ed.), Encyclopedia of Social Measurement, Elsevier, Amsterdam, pp. 375-378.

Wedlin, L. (2007), “The role of rankings in codifying a business school template: classifications, diffusion and mediated isomorphism in organizational fields”, European Management Review, Vol. 4 No. 1, pp. 24-39.

Willmott, H. (1995), “Managing the academics: commodification and control in the development of university education in the UK”, Human Relations, Vol. 48 No. 9, pp. 993-1027.

Acknowledgements

The authors gratefully acknowledge the useful comments of two anonymous reviewers, Gun Abrahamsson and participants at workshops for CEROC (Centre for Empirical Research on Organizational Control) at Örebro University. Financial support was received by The Swedish Research Council (Project No. 421-2014-740), the Jan Wallander and Tom Hedelius Foundation (Project No. 2015-0031:1) and Örebro University.

Corresponding author

Jonas Gerdin can be contacted at: jonas.gerdin@oru.se