PMM and beyond – reflections on the paper “new developments in institutional research on performance measurement and management in the public sector”

Abstract

Purpose

This paper reflects on Sven Modell's (2022) study discussing uses of institutional theorising for studying performance measurement and management (PMM) in the public sector context. The paper provides arguments for critically analysing the assumptions and characteristics of PMM research.

Design/methodology/approach

First, the paper addresses PMM as a field of research linking scientific disciplines, schools of thought and academic scholars. Second, the paper discusses the role of institutional theorising in PMM research. Third, the paper analyses and reviews Modell's ideas on the future prospects of PMM research. The paper also elaborates on the ideas presented in Modell's paper.

Findings

Modell's paper suggests sociology of valuation and the discussion on hybrid governance as future developments for PMM research. This paper provides a conceptual perspective to link these areas together. Furthermore, the paper contributes to understanding PMM as a multi-disciplinary, inter-disciplinary and trans-disciplinary research area.

Originality/value

This paper contributes to the conceptualizations of values, valuation and hybridity in PMM research from the viewpoint of institutional theory.

Citation

Vakkuri, J. (2022), "PMM and beyond – reflections on the paper “new developments in institutional research on performance measurement and management in the public sector”", Journal of Public Budgeting, Accounting & Financial Management, Vol. 34 No. 4, pp. 501-511. https://doi.org/10.1108/JPBAFM-12-2021-0168

1. Introduction

I am grateful for the invitation by the Journal of Public Budgeting, Accounting and Financial Management and its editor Giuseppe Grossi to write a rejoinder reflecting upon Sven Modell's paper (2022). Modell's paper poses the question of how to understand uses of institutional theory and theorising for studying performance measurement and management (hereafter PMM), especially in the public sector context. The aim of my paper is to focus on a limited number of issues that could be instrumental in analysing and reconsidering assumptions of PMM research. For that, first, I will discuss PMM as a field of research linking together different scientific disciplines, schools of thought and academic scholars. Second, inspired by Modell’s (2022) ideas, I will discuss the role of institutional theorising in the context of PMM research with an emphasis on specifying what institutional theory we are actually referring to. Third, I will analyse and review Modell's ideas on the future prospects and developments of PMM research. The suggestions made in his paper have important potential; however, it is also possible to elaborate on these ideas further. The final section of the paper provides conclusive arguments.

2. PMM as a research topic

It is important to recognize the fundamental characteristics of PMM as a research topic. As institutional carriers of scientific practices, PMM researchers are significantly conditioned and influenced by theoretical underpinnings and methodological choices from their academic tribes (Becher and Trowler, 2001). Let us consider the institutional landscape of PMM research from three viewpoints to put Modell's ideas into perspective.

First, PMM is a multi-disciplinary area of scientific research in which several disciplines collectively contribute to the common body of knowledge, both knowingly and unknowingly. However, there does not exist a unified common body of knowledge on PMM. Rather, we deal with a complicated assemblage of scientific insights coming from different disciplines. Those insights have organised into parallel streams of research knowledge that several scholars have started to name PMM. Perhaps there is even a silent, loosely coordinated ethos for creating a “performance science” that would transcend extant research disciplines and their frameworks of rationality, measurement principles and vocabularies. The theories and models of PMM that we have come across in individual papers and books are multi-faceted, nested combinations of intellectual ideas initiated, revised and debated in (applied) neoclassical economics, sociology, public policy and management, business studies and accounting, political science and philosophy, to name a few (Whitley, 2000).

Second, PMM research has unique “local”missions and purposes in different disciplines that aim at contributions to the common body of knowledge, but within the boundaries of their own academic tribe. Thus, PMM can also be considered an inter-disciplinary area of scientific research. Personally, I learned this through writing a co-authored paper on PMM with Jan van Helden and Åge Johnsen (Van Helden et al., 2008). Our goal was to compare two groups of scholars studying PMM: accounting and public management scholars. We concluded that although it is easy to understand the institutional logics of these two academic tribes and the respective forms of “internal” accountabilities and commitments, leading to specialization, it is equally easy to observe the boundaries of academic tribes as limiting the exploration of novel ideas (also Johnsen et al., 2012). The potential for looking across academic tribes in understanding PMM is still very much unrealised. The conscious decision to cite scholars from one's own academic tribe, and of course the deliberate choice not to cite scholars from “other” tribes, despite them being crucial to argumentation, remains a fairly institutionalized practice of PMM research (Van Helden et al., 2008, p. 647). Whether it makes sense in advancing PMM research is worth constant reconsideration.

Third, PMM is a trans-disciplinary area of research (Gibbons et al., 1994). Many topics of PMM research are not merely driven by the accumulation of scientific knowledge, but by complex interactions between theoretical design and institutional practices of organizations in governments, business fields and civic activities. The development of PMM research can be seen as a reciprocity between theory building vis-à-vis distinct practice theories (Orlikowski, 2000, 2002) and forms of administrative pragmatism (Johanson and Vakkuri, 2017). Moreover, academic scholars are not only influenced by the prior theoretical body of knowledge, but also by engaging themselves with practices and contexts of the institutional action that they study (Power, 2015). This is not something we should be ashamed of. Instead, we should nurture this aspect while simultaneously being aware of the implications of those assumptions on research efforts. For instance, consider March and Sutton (1997) critically discussing why and how several influential studies of organizations treat performance as a dependent variable.

Organizational researchers live in two worlds. The first demands and rewards speculations about how to improve performance. The second demands and rewards adherence to rigorous standards of scholarship. In its efforts to satisfy these often conflicting demands, the organizational research community sometimes responds by saying that inferences about the causes of performance cannot be made from the data available, and simultaneously goes ahead to make such inferences. (March and Sutton, 1997, p. 698)

In responding to the institutional pressures for being useful and producing research impacts, PMM research focuses on explanations for how to distinguish “good” performance from “poor” performance. With that, scholars are not only driven by intended contributions to theory building, but also by perceived expectations for doing research that they consider relevant for practice and thus legitimate.

Modell's paper invites me to reflect on the role of “public” in relation to PMM research. I very much agree with the observation that there are limitations in the current excessively decontextualized depictions of how, for instance, institutional logics evolve and how this evolution may be de facto shaped by the interaction between institutional levels. Consider the following quote from the paper (Modell, 2022, XXX):

Whilst institutional logics often include distinct conceptions of performance as a criterion for determining how legitimacy is bestowed on organizations, this body of research has been dominated by a view of such logics as given within institutional fields and has mainly examined how the organizations that are situated in such fields respond to competing logics. Hence, such logics are treated as relatively monolithic entities and little attention is paid to their reciprocal interplay with the PMM practices that evolve across multiple levels of analysis. This limits our understanding of how PMM practices are implicated in the shaping of institutional logics over time and how this contributes to the evolution of institutional fields.

Currently, I am especially considering the research object of PMM with a particular emphasis on “public” (Modell, 2022). How are the levels defined in PMM research, and to what extent are such definitions able to address the characteristics of public administration systems (cf. Johanson and Vakkuri, 2017)? How should we acknowledge the macro-level policies of governments influencing the allocation of taxpayers' money, thereby shaping the design and contents of PMM systems and practices? How may the level analysis be related to the micro-meso-macro distinction of levels present in research discussions of public policy and management (Capano et al., 2019)? Organizations vis-à-vis institutional field typologies may be useful to some extent, but they also include limitations in understanding the specificity of PMM in public politics, policies and public administration systems. We should be able to include publicness more comprehensively in PMM research (Steccolini, 2019). Moreover, PMM research could explore publicness as a broad, more inter-disciplinary construct (Meynhardt, 2009; Bozeman, 2020). With that, it can be ensured that the emphasis on ‘public’ in PMM research does not remain institutionally hollow.

3. The role of institutional theory in PMM

Institutional theory has become an extremely powerful collection of theoretical ideas to analyse and explain PMM topics. It is even surprising to see such a success. In Modell’s (2022) paper, this is primarily associated with NPM-type reforms in the OECD countries where institutional theorising has constituted an important, seemingly credible alternative system of concepts for rational choice-driven theories of how government systems ought to be governed, measured and controlled (cf. Hood and Dixon, 2015). This may be one explanation. Scholars have been interested in isomorphism; particularly, they are interested in how reforms are products of mimesis, whether they are instigated by professional codes of ethics and to what extent PMM development could be seen as coercive modes of regulatory regimes (DiMaggio and Powell, 1983). Moreover, the classic idea discussed originally by Meyer and Rowan (1977) on how rational modes of action could be distinguished from ritualistic modes of action have gained huge popularity. In fact, the same idea remains a justified part of PMM research, often with an emphasis on to what extent the two worlds would coalesce or how they would be decoupled from each other. Organizations are ripe with modes of behaviour for saying (reporting) one thing and doing another (Bromley and Powell, 2012).

The link between institutional theory and PMM calls for more in-depth scrutiny. In particular, the “accounting school” of PMM research has been influenced by institutional economics, even more by institutional sociology. This may be one explanation as to why institutional theorising in general has become influential for researching PMM. On the other hand, my interpretation is that the “public management” school of PMM research is not equally influenced by institutional theories. Nevertheless, it would be important to trace the development of institutional theories in PMM research in public management and accounting studies to be aware of the rationales for applying institutional theorising in different contexts. For reference, consider Klaes and Sent's (2005) paper on the conceptual history of bounded rationality that intelligibly demonstrates how important it is to encapsulate the temporality of the theoretical concepts that we use. For PMM research, the utilization of Klaes and Sent's ideas would help us put short and long term developments of institutional theorising into an elaborate perspective.

Modell’s (2022) argumentation regarding the one-sidedness of uses of institutional theories is excellent. I will focus on the interesting association between the concepts of institutional logic and performance. It has become too simple an argument to explain performance variations with different institutional logics. An important argument of Modell's paper is that there is much room for reconsidering and re-conceptualising the direction of causality. In other words, how can we explain fundamental principles and behaviours of public organizations with the intended rationalities among those organizations interpreted through distinct PMM schemes and models? Following this direction of perceived causalities, institutional logics do not explain performance; instead, (intended) performance explains institutional logics (Modell, 2022). Public organizations are regarded as intentional subjects influencing performances through the processes of their use of and engagement with PMM models. Interestingly enough, public organizations pursue understanding of the ways not only to simplify the world with performance schemes, but also the world of performance schemes (Vakkuri, 2010). Thus, Modell's paper encourages an explanation of the ways in which designers, users and uses of performance schemes intervene in the process of public service delivery. The notion of performance in public organizations incorporates both the aspects of performance and performativity, i.e. both intended and realised rationalities of institutional action as well as social constructions of those rationalities (Vosselman, 2014; Vakkuri, 2013).

The notion of institutional(ised) performance includes interesting potential for PMM research to find more intelligible research designs where important multi-faceted complexities in our rationality ideals would be analysed in eloquent theoretical settings. This simultaneously calls for more explicit attention to performance through the lenses of social contructivist thoughts, through diverse frames and framing processes as well as the scrutiny of performances of different levels and boundaries. Here, PMM practices are not analysed as instruments of adaptation to institutional environments. They are understood as holistic descriptions that may actually shape and reshape the institutional conditions in the long run (Modell, 2019).

4. Values, valuation and value creation in PMM

Modell’s (2022) paper introduces two important future research agendas. These are the closer engagement with the sociology of valuation research as well as the research on hybrid governance and organizations. Both of these suggestions for further research are extremely promising and deserve close attention from PMM researchers. Let me provide reflections by combining the two aspects of Modell's paper.

4.1 Understanding value creation in PMM

PMM is a representation of values, value propositions, and value-creation mechanisms in organizations and institutional fields. PMM manifests, often through (limited) quantifications, how market, public and social values are demonstrated, measured and enacted (Grossi et al., 2022; Vakkuri et al., 2021a). Irrespective of this general starting point, performance quantifications are often considered neutral, value-free techniques. While by no means are quantifications considered a panacea as such, they permeate the activities of society and economy by providing a seemingly objective ground for complex value-laden resource allocation decisions. They may serve as centres of calculation or technologies of distance to contribute to political legitimacy (Porter, 1995; Boltanski and Thévenot, 2006; Miller and Rose, 2008). Societies have learned to trust performance quantifications as sensible, useful and real. At the same time, it has become easy to neglect the characteristics of PMM as a constellation of competing and conflictual values.

There are two relevant problems in understanding the value creation of public sector activities: let us call these the problems of “knowing” and “doing” (Vakkuri and Johanson, 2020a). PMM research should be interested in knowing the contents of value(s) in the public sphere. With this approach, the focus is on the implications of market, public and social value configurations in PMM systems. What type of value propositions do PMM systems reflect? The publicness perspective of value is one crucial perspective that is growing in importance, which I strongly advocate (Meynhardt, 2009; Steccolini, 2019; Bozeman, 2020). To take this further, it becomes even more important to discuss the hybridity of values, including simultaneous influences of market, public and social values, and how the mixes of these influences condition institutional processes (Vakkuri et al., 2021a; Vakkuri and Johanson, 2020a). The links between hybridity and value creation will be discussed in more detail in the next section.

Modell’s (2022) paper refers to one possible extension to the PMM literature; namely, that it would be possible to study the institutionalization of various conceptions of values. As argued in the paper (Modell, 2022, XXX):

… what comes to be valued and not valued along these lines can foreground the socio-political processes and power struggles associated with the institutionalisation of various conceptions of performance. By examining which actors gain and lose power through such processes, public sector accounting and management scholars can perhaps take research on public value creation forward whilst, at the same time, imbue institutional research on PMM with more explicit, critical intent.

In fact, this concerns the above-mentioned “doing” problem of value creation with an emphasis on action spaces where different forms of value are imagined and enacted in practice (Stark, 2009). Such a doing process, which is also called valorization, may take place irrespective of the contested and conflicted characteristics of values (Vatin, 2013). In other words, one does not need a perfectly solid measure of value for enacting a process that is called value creation. For assessing the validity of my observation, consider the extensive proliferation of doing mechanisms in the value creation literature of organizations and institutions. Societies and organizations pursue the creation of value before they know exactly what the ultimate values may be. The process is unexpected and often serendipitous (for an overview, see Vakkuri and Johanson, 2020a).

This tradition incorporates a plethora of verbs to conceptualize forms of doing value. We can think of the process of “creating” value where an actor develops innovative services to provide novel solutions to social problems (Lepak et al., 2007), “capturing” value where organizations that have not been involved in initiating value retain some of the value created earlier (Jacobides et al., 2006), or “exchanging” value as the realization of values through the exchange mechanism between buyers and sellers. For PMM, these constitute interesting institutional facets for discussing who creates, who captures value and whether the process of exchanging value (e.g. in Public-Private Partnership (PPP) arrangements) accounts for effective, just and democratic impacts on service users and citizens (Mazzucato, 2018). Value-creation processes may sometimes increase the worth of existing systems but also destroy the fruits of previous value generation (Plé and Caceres, 2010). For instance, the COVID-19 pandemic has destroyed or “co-destroyed” distinct forms of value in societies but has also created new value through innovative practices (Engen et al., 2021).

Value creation mechanisms also incorporate different constellations of actors, processes and outcomes. The system incorporates value creators and value users as subjects. Somebody produces value while someone else uses value. There may often exist several producers collectively collaborating (“co-producing”) on value creation (Ramirez, 1999). The research tradition also informs us about “balancing” between different logics of value creation, or “sharing some of the value with other stakeholders and communities, who may not have been the targeted beneficiaries in the first place (Porter and Kramer, 2011). Finally, prior research has also widely reflected upon the myriad of outputs of value creation. For more in-depth understanding on the notion of performance, PMM research needs to develop more sophisticated integration of the perspectives knowing and doing in value creation.

4.2 Linking value creation with the hybrid institutional contexts of PMM

Let us discuss in more detail how to understand values, valuation and value creation in the contexts of hybrid organizations and systems, and how such a combination could be important for PMM research. Hybridity and hybrid forms of governance pose an important question: If PMM research is supposed to scrutinize performance measurement and management in the public sector context, what do we then actually mean by “public”? Even rudimentary observations of the current policy problems make it difficult to limit the discussion to a monolithic notion of the “public” sector, let alone the black-and-white distinction into “public” and “private” sectors. Governing global problems, such as combating climate change, surviving pandemics, alleviating social exclusion and developing smart and sustainable cities has become (or in fact has probably always been) a collaborative exercise between public policies and agencies, private business, economic institutions and civic activities (Vakkuri et al., 2021a). While PMM research may acknowledge that public, private and civil sector domains together play a role in how important forms of market, public and social value are created, it still struggles with understanding why, through what mechanisms, and with what impacts this interaction takes place. The hybridity problem, more explicitly, refers to the interfaces among public, private and civil society via distinct modes of ownership, competing institutional logics, diverse funding bases and various forms of social and institutional control in contributing to societal impacts and value (Johanson and Vakkuri, 2017).

It could be argued that in the PMM research market, public and social forms of value have been regarded as distinctive firewalls that separate sectors and agencies from each other (Thacher and Rein, 2004). In such an institutional system with high specialization and compartmentalization—the assumption holds—market value is created by firms, entrepreneurs and businesses; public value by the government and public sector; and social value by non-profits, social enterprises and civic organizations. It is interesting how this type of thinking has remained an important strategy for societies to manage the value conflicts inherent in governance efforts. According to classic ideas by Weaver (1948), such compartmentalization of value creation efforts into smaller problems constitutes an important heuristic for governing organized complexity, partly because it has always been easier to demonstrate vertical rather than horizontal accountabilities. However, it simultaneously limits societies' understanding of how public policies could at best address inter-organizational value-creation mechanisms to design and implement policies (Vakkuri et al., 2021a, 2021b).

How can we connect the problems of knowing and doing in the value-creation processes of hybrid organizations? Let us consider three value-creation mechanisms originating from the important characteristics of hybridity and hybrid organizations: mixing, compromising and legitimizing (Vakkuri and Johanson, 2020; Vakkuri et al., 2021a, b).

First, in hybrid settings, the process of creating value entails important complexities, as actors should simultaneously provide distinct categories of value. Mixing as a value-creation mechanism refers to the fundamental process of combining previously created or existing value categories with the aim of creating novel variants of value (Nicholls, 2009; Polzer et al., 2016). Consider the process of mixing values between shareholders' interest and public interest in state-owned enterprises, or the process of blending mission orientations with for-profit logics in social enterprises (Costa and Andreus, 2020).

Second, managing competing values in hybrid settings necessitates innovative mechanisms of compromising. Often, organizations are institutionally forced to reconcile competing value-creation logics by reaching compromises between different value propositions. In PMM research, compromising logics materialize in different ways. While accounting research has discussed compromising accounts (Chenhall et al., 2013), public management research has addressed performance dialogues as instruments of compromising logics (Moynihan, 2005). Forms of compromising can be applied to different types and levels of accountabilities that may exist within intra- or inter-organizational settings (Rajala and Kokko, 2022). Naturally, a performance measurement system is already a compromise of multiple rationalities enacted in hybrid settings (De Waele et al., 2021).

Third, legitimacy is a crucial concern for hybrid organizations—there are legitimacy demands from external audiences and society that may impose additional constraints. However, the same demands may provide opportunities for hybrid organizations that can benefit from the fluidity of legitimization requirements (Karré, 2020). For instance, although university organizations struggle with measurement ambiguities and multiple audiences for legitimization, in many cases they have opportunities to choose the modes of value they wish to demonstrate and to not disclose those forms of value they wish to hide (Vakkuri and Johanson, 2020b). They may sometimes legitimize their performance by selecting the audiences to which they wish to be visible and transparent.

5. Conclusions—value of PMM (research)

Sven Modell's paper (2022) looks into institutional theorising in the context of PMM by intelligibly encouraging PMM researchers to avoid one-sided interpretations of institutional theory ideas, including institutional logics, levels of analysis, institutional constructs of value, valuation and performance. More theoretical understanding of the links between valuation and performance and deeper analysis regarding the links of the institutional levels and the contextual enactments of PMM is required. To realise these research prospects, I would further specify and extend the reflection.

1. Research should become more cognizant of the role of the public in PMM. More sophisticated institutional theorising is needed to integrate publicness, the characteristics of politico-administrative systems, and public value into PMM discussions.

2. Future research should be aware of the limitations in our assumptions on how “public sector” systems work and operate in comparison with, in relation to, and in collaboration with other societal sectors and agencies. The impact of PMM research could be significantly improved by addressing the comprehensive role of performance measurement and management in different institutional contexts and in the wider society.

3. Expanding upon the notion of hybridity in PMM research has important opportunities. This area would significantly benefit from systematic dialogue and inter-disciplinary efforts, and it could adopt several forms. PMM researchers may examine their performance-related topics by using hybridity as institutional conditions for the modes of operation. This could be related to understanding the impacts of hybridity on PMM design, implementation and use. Moreover, hybridity may be used as a more profound theoretico-conceptual framework for enabling the actions through PMM principles and schemes. Studies may concentrate on understanding the implications of distinct hybrid logics and value-creation mechanisms on the ways how hybrid organizations are managed and measured.

Links between value creation and performance management introduce several research puzzles, one of which is associated with the fundamental tension between the complexity of performance and the governability of performance. A multidimensional measurement of performance eludes the goal of simplicity and forms of administrative pragmatism (Johanson and Vakkuri, 2017). In doing so, it hinders swift practices for performance improvements (Vakkuri et al., 2021b). The question centres on what we expect from performance management systems, whether that consists of sophisticated representations of “performance realities” or easy-to-use, easily manageable instruments to enable performance improvements. This is an important dilemma in the context of PMM, where it has become more vital to understand a multiplicity of rationalities. This is one approach as to how Modell (2022) conceptualizes institutional(ised) performance. Perhaps it is also close to the ways in which we can understand the fundamental tensions and lines of division within PMM research.