Abstract
Purpose
Public health strategies and activities are intrinsically complex. According to the literature, this “wickedness” depends on the different interests and expectations of the stakeholders and the community, the fragmented governance of the related services and the challenges in measuring and assessing public health outcomes. Existent performance measures and management systems for public health are not designed to cope with wickedness since they are mainly focused on inputs and outputs, neglecting broader outcomes because of their long-term impact and the poor accountability of results. This research aims to tackle this shortfall by adopting a dynamic performance management (DPM) approach.
Design/methodology/approach
This research explores the case of the vaccination campaign of a Regional Health System. Through the analysis of an illustrative case study, the research discusses both opportunities and limits of the proposed approach.
Findings
This research highlights that DPM supports performance management (PM) in wicked contexts, thanks to the adoption of a system-wide perspective and the possibility of using simulation to experiment with alternative strategies and benchmarking performance results with simulated trends.
Originality/value
This article tackles a gap related to the management of wicked problems both from a theory and a practical perspective. In particular, this research suggests the adoption of DPM as an approach that may support policymakers in tackling social pluralism, institutional complexity and scientific uncertainty all at once.
Keywords
Citation
Noto, G., Prenestini, A., Cosenz, F. and Barresi, G. (2023), "Tackling wicked problems in performance management and governance of public health: an empirical analysis of COVID-19 vaccination strategies", International Journal of Public Sector Management, Vol. 36 No. 2, pp. 130-151. https://doi.org/10.1108/IJPSM-07-2022-0163
Publisher
:Emerald Publishing Limited
Copyright © 2023, Guido Noto, Anna Prenestini, Federico Cosenz and Gustavo Barresi
License
Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Introduction
Our society and its current challenges are growing in complexity, so modern literature defines these last ones as “wicked” (Rittel and Webber, 1973; Head and Alford, 2015). Wicked does not mean “evil” but refers to the impossibility of sharing an understanding of societal problems and their definition. According to Head (2008) and Head and Alford (2015), “wickedness” mainly depends on three key factors: social pluralism – i.e. the presence of multiple interests and values of stakeholders; institutional complexity – i.e. the horizontal fragmentation and multilevel governance characterizing public administrations; and scientific uncertainty – i.e. the existence of gaps in reliable knowledge.
The emergence and persistence of wicked problems pose important challenges to performance management (PM) in the public sector. PM is a methodological approach whose main purpose is to guide organizations toward their objectives according to effectiveness, efficiency and viability principles (Ouchi, 1979; Ferreira and Otley, 2009).
However, traditional PM systems are considered unable to cope with wicked problems and their characteristics due to a static approach that does not account for feedback-loops and non-linearities; and the inter-organizational fragmentation characterizing the public sector that leads to poor accountability of results (Drury, 2014; Noto and Bianchi, 2015; Bianchi, 2016). To manage performance in wicked contexts, key solutions proposed by the literature refer to the implementation of collaborative arrangements and co-production (Cristofoli et al., 2017; Loeffler and Bovaird, 2019; Bianchi, 2021). These solutions have been designed and tested to address the source of complexity deriving from institutional fragmentation and social pluralism. However, to the best of the authors' knowledge, only some scholars proposed solutions that also tackle scientific uncertainty.
Based on this gap, this work aims to illustrate how dynamic performance management (DPM) can be used to cope with wicked problems in managing performance by tackling the key limits characterizing traditional PM schemes.
DPM is a scientific approach combining system dynamics (SD) with PM theory (Bianchi, 2016). SD is a modeling technique that allows analysts to frame and simulate complex systems and experiment with the models to design strategies for management and change (Forrester, 1958).
The adoption of SD to tackle wicked problems finds its foundation in the methodological opportunity to explore with simulation and engage with stakeholders (Vennix, 1999; Sterman, 2000), thus providing more robust decision support to inter-institutional settings. As such, SD allows analysts to adopt a holistic perspective to frame social pluralism, institutional complexity and scientific uncertainty as the key factors characterizing wicked contexts. As a result, it supports collaborative PM systems fostering a shared understanding of their operating principles and processes.
This article focuses on public health and health prevention to explore and analyze the contribution of SD in the case of wicked issues. Public health is considered fundamental in tackling current social challenges such as the COVID-19 pandemic. In particular, the process of planning and implementing vaccination campaigns in western Countries represents one of the most challenging wicked problems in the current era.
The research begins with a literature review on wicked problems and PM in the public sector. A second theoretical background explores the literature on PM in public health. Building on this literature review, the article proposes and illustrates the DPM approach (Bianchi, 2016, 2021) as a systemic method for PM and governance in wicked contexts. Then, the approach is tested on a case study exploring the vaccination campaign in Lazio – an Italian regional health system that reported a successful experience with vaccination operations – to explore how this health system could benefit from the adoption of DPM. The case is developed by collecting empirical data, active engagement and discussing results in focus groups with the Lazio regional managers. In the last section of the article, the case study results enable us to discuss the main advantages and limitations of the proposed approach also in comparison with the gaps found in the literature. Eventually, the article outlines critical implications for theory and practice and concludes with future research perspectives.
Theoretical background
Wicked problems and performance management
In the public administration context, the last years have been characterized by a growing and renewed interest in exploring the so-called “wicked” problems and the possible countermeasures public policy research proposes to deal with their negative implications. “Wicked” problems were defined back in 1973 as issues that are hard to define and manage due to their complexity (Rittel and Webber, 1973).
Wicked problems are counterposed by the literature to tame ones (Roberts, 2000; Weber and Khademian, 2008; Head and Alford, 2015). Differently from these lasts – for which both the definition of the problem and the likely solution are clear to decision-makers – wicked problems have no definitive formulation and may have multiple solutions which are necessarily open to further interrogation and adaptation (Rittel and Webber, 1973; Head and Alford, 2015). Moreover, robust evaluation frameworks to assess the success of initiatives and solutions implemented are not suitable to support public managers in addressing them (Rittel and Webber, 1973; Head and Alford, 2015).
In spite of these challenges, in the last two decades, we witnessed an emergence of research focusing on how to deal with these problems through the public administration and management lenses (Roberts, 2000; Weber and Khademian, 2008; Ferlie et al., 2011; Head and Alford, 2015; Cristofoli et al., 2017; Bianchi et al., 2017; Massey, 2022). As previously mentioned, Head (2008) and Head and Alford (2015) framed the sources of wickedness in three factors, i.e. social pluralism, institutional complexity and scientific uncertainty. Public managers and leaders are thus called to cope with these factors to guide their organizations in dealing with wicked problems. Decision-makers operating in wicked contexts are indeed exposed to multiple trade-offs between divergent and sometimes conflicting results (Belle and Cantarelli, 2022).
An emerging topic on wicked problems literature is promoted by a group of scholars who focused their research on how to manage performance in wicked contexts (Blackman et al., 2006; Drury, 2014; Noto and Bianchi, 2015; Cepiku, 2017; Bianchi et al., 2017; Herrera et al., 2019; Costumato, 2021; Noto et al., 2022). PM is defined as the activity that guides an organization, or a social system, toward its objectives and targets (Ouchi, 1979; Lebas, 1995; Ferreira and Otley, 2009). This first definition clarifies how challenging may be the design and implementation of PM systems in contexts whose problems and issues are not even definable and, consequently, desired outcomes are ambiguous or unclear (Rittel and Webber, 1973; Head and Alford, 2015).
To deal with wickedness and lead organizations and networks towards desirable performance, extant literature strongly suggested the adoption of collaborative (Roberts, 2000; Jackson and Stainsby, 2000; Christensen and Lægreid, 2007; Ferlie et al., 2011; Lagreid and Rykkja, 2015; Cristofoli et al., 2017; Noto et al., 2022) and co-production practices (Bianchi et al., 2017; Loeffler and Bovaird, 2019; Steen and Brandsen, 2020). The idea beyond these studies is that collaboration and co-production may foster a shared understanding of problems. Indeed, such countermeasures find their rationale in the institutional fragmentation characterizing the current governance structure of many public sectors in Western countries and in need to involve the public at large to comprehend and address the multiple needs and interests carried out by the community (Christensen and Lægreid, 2007; Turrini et al., 2010). Collaboration and co-production may foster a shared understanding of problems and align stakeholders' expectations toward preferred solutions and outcomes (Cristofoli et al., 2017; Bianchi et al., 2017). As such, the suggested solutions fit well with two of the sources of wickedness previously mentioned – i.e. social pluralism and institutional fragmentation. However, these kinds of solutions do not necessarily directly tackle the third cited source of wickedness – i.e. scientific uncertainty.
Uncertainty is a phenomenon that is part of existence and public administration and management are disciplines that necessarily involve uncertainty (Weber, 1999; Cairney et al., 2016). There are different sources and types of uncertainty. Regarding scientific uncertainty, we refer to gaps in reliable knowledge (Reckhow, 1994; Head and Alford, 2015).
Among the main approaches used to cope with uncertainty, and in particular scientific uncertainty, research in multiple disciplines widely relies on policy models and simulation techniques (see, among others, Meadows et al., 1972; Reckhow, 1994; Papadopoulos and Yeung, 2001; Welter and Kim, 2018). Simulation models could provide practical support to explore and experiment with complex social systems and thus comprehend their functioning (Forrester, 1958; Sterman, 2000, 2002, 2014; Kim et al., 2013; Cosenz, 2018; Noto and Cosenz, 2021; Noto, 2022). Moreover, simulation allows one to experiment with alternative scenarios and test the sensitivity of a system regarding specific variables or parameters. Such an approach is of great help when scientific knowledge about variables and value parameters is unclear or incomplete (e.g. the contagiousness of a virus variant). In the words of Sterman (2002, pp. 524), “simulation is essential for effective systems thinking, even when the purpose is insight, even when we are faced with a “mess” rather than a well-structured problem”.
Assessing and managing the performance of public health services
Since the implementation of New Public Management reforms, PM has been extensively applied in public administrations and, more generally, in the public sector (Hood, 1991; Bouckaert and Halligan, 2007; Rajala et al., 2018). The public health sector has not escaped this cultural wave, implementing PM systems and mechanisms (Aidemark, 2001; Lega et al., 2013; Nuti et al., 2018; Noto et al., 2021).
In the realm of public services, health service has been an interesting field of implementing PM practices due to its high input and output measurability. The importance of measuring health service end-results emerged already in the seminal article of Donabedian (1966), which highlighted the need to assess the quality of care provided through the assessment of structures, processes and outcomes. The Donabedian performance assessment framework has become one of healthcare's most diffused performance measurement frameworks. It has successfully been adopted in the hospital setting, both in the case of inpatient and outpatient services (Berwick and Fox, 2016; Lenzi et al., 2020). Donabedian's framework also stands at the basis of the performance measurement schemes of public health (Deber and Schwartz, 2016; Riley et al., 2012; Scutchfield et al., 2004; Handler et al., 2001) that, in the USA, had led to the development of the Turning Point PM Conceptual framework in 2003 and the release of a new version in 2013. The five components of the Turning Point are (Landrum and Baker, 2004; DeAngelo et al., 2014): performance standards (set goals, targets and relevant indicators to improve public health performance), performance measures (refinement and application of performance indicators and measures), reporting of progress (analysis of data and feedback to stakeholders), quality improvement (use of data for decisions to improve policies and programs, manage change and achieve quality outcomes) and visible leadership. Following this stream of thought, other research works tried to develop Balanced Scorecards for Public Health (Robinson et al., 2003; Weir et al., 2009). In the studies mentioned above, PM systems were not designed to provide routine data to monitor ongoing public health strategies but only to give a static view of public health performance (Schwartz and Deber, 2016). As argued by Schwartz and Pais (2009), these frameworks are poorly designed to assess the inherent complexity of the public health sector characterized by policy fragmentation, feedback loop and non-linearity.
Therefore, PM still encounters complex challenges when implemented in public health and health prevention services. Moreover, public health performance measurement systems rarely link broad outcomes to measures of processes and output controlled by public health organizations (Schwartz and Deber, 2016). This is mainly due to the specific mission of public health and, consequently, the characteristics that distinguish public health services from others in the healthcare sector. The mission of public health is indeed multifaceted. It depends on the different healthcare governance systems and infrastructures; however, there are some common traits across the public health systems worldwide. Public health has the aim to improve the health of populations through (Ingram et al., 2012; Weir et al., 2009): the prevention and monitoring of diseases; the development of policies to inform, educate and empower people and community about health issues; the enforcement and application of regulations that protect health and ensure safety.
Due to the complexity of its mission, multiple features make public health PM challenging.
First, many factors that influence population outcomes are beyond the direct control of public health organizations, such as economic factors, lifestyle, gender and genetics (Weir et al., 2009; Deber and Schwartz, 2016). This is different in the hospital setting, where the outcome mainly depends on organizational and individual performance.
Second, public health services are usually delivered by a network of organizations (e.g. health authorities, police forces, laboratories and other public organizations) that collaborate (Schwartz and Pais, 2009; Noto et al., 2022).
Third, it is difficult to directly measure the impact of public health actions on population health outcomes due to the time lag between public health interventions and their effects. Outcomes of prevention activities – such as education, inspections and vaccinations – can only be appreciated in the long run. As such, it is not possible to assess their performance in time to put corrective actions in place (Hunter, 1990; Ingram et al., 2012; Cinquini et al., 2014). Another temporal lag is related to the long latency of some diseases that may obscure the impact of performance on measures that come out later, like mortality rates (Ingram et al., 2012).
Last, public health initiatives and outcomes affect other domains such as economic development.
These characteristics describe a wicked context and pose essential challenges to PM in public health. Performance measures and PM systems for health prevention services usually focus on inputs and outputs, neglecting broader outcomes due to their long-term impact and the poor accountability of results. Namely, due to governance fragmentation, it is impossible to identify a single institution responsible for the outcomes obtained.
The traditional performance measurement and management frameworks (from Donabedian's to Turning Point's systems) present some gaps related to the wicked nature of public health. However, the basic elements of their conceptualization are still valuable for elaborating new models for managing scientific uncertainty alongside social pluralism and institutional complexity. Therefore, the instrumental view of DPM conceptualized by Bianchi (2016) is appropriate to repropose the basic elements of the PM frameworks for public health. This view identifies and connects strategic resources (e.g. human resources, equipment, pharmaceutical products), intermediate results or processes outputs (e.g. number of inspections delivered, number of people vaccinated, etc.) and end-results or outcomes (e.g. epidemic outbreaks, disease morbidity, etc.).
In addition, the DPM supports decision-makers by allowing them to experiment and simulate alternative strategies with the SD model. This feature enables the switch from feed-back to feed-forward control of results. Namely, decision-makers can monitor the progress toward the achievement of desired outcomes by periodically comparing results with simulated behaviors and estimating projections for subsequent periods to eventually anticipate adjustments to the objectives or to the pattern of actions.
Method
This article proposes the DPM as a method to tackle PM in wicked issues. This scientific approach combines traditional PM theory with SD modeling (Bianchi, 2016).
SD is a methodological approach for modeling and simulating complex physical and social systems and experimenting with the models to design strategies for management and change (Forrester, 1958). It provides a systemic perspective and a set of conceptual tools that enable one to frame complex, non-linear and multi-loop feedback systems (Forrester, 1958; Meadows, 1980; Sterman, 2000).
To represent complex and dynamic systems, SD models adopt a graphical syntax in which flow (rate) and stock (level) variables interact with each other through a set of causal relationships mathematically modeled as differential equations that can be simulated (Sterman, 2000; Größler et al., 2008). This means that a variable influences another variable (1) positively (i.e. an increase of the one corresponds to an increase of the other and vice versa), (2) negatively (i.e. an increase of the one corresponds to a decrease of the other and vice versa) and (3) according to a non-linear relation between them. If such relations originate closed circuits, i.e. the feedback loops could be reinforcing (R), i.e. they produce exponential trends of the system over time; or balancing (B), i.e. they limit such an effect by tending to a steady-state.
According to an instrumental view of performance (Bianchi, 2016), performance can be framed into strategic resources, performance drivers (or intermediate results) and end-results. Strategic resources are those stock variables whose activation and use allow the implementation of specific actions. Performance drivers are those indicators, usually computed as ratios, which display the progress of an organization or a system toward achieving desired results. End-results represent the system's goals whose achievement allows to foster back the strategic resources – i.e. generating accumulation and/or depletion processes. Figure 1 illustrates how performance drivers are built and connected to end-results and strategic resources according to a causality-based perspective.
Due to the above considerations, this research argues that a DPM approach is suitable to address PM in wicked contexts – such as public health – since it combines (1) a PM framework, (2) a system-wide perspective in which the various interests and perspective of the various stakeholder, as well as the multilevel governance, are represented and (3) it allows to cope with scientific uncertainty through the use of simulation.
Consistently with the purpose of this research, DPM is applied to an illustrative case study (Scapens, 2004). A case study strategy is particularly valuable for addressing theory-building research and demonstrating that the existing research does not properly address the investigated propositions (Eisenhardt and Graebner, 2007). The case study focuses on the adoption of the COVID-19 vaccine strategy and its implementation in the best performer Italian regional system identified through the data on vaccine delivery (National Health Institute data [1]), i.e. Lazio. The case has been selected to explore and discuss the potential advantages and risks associated with the adoption of DPM to foster performance in the health prevention sector and, more generally, when operating in wicked contexts.
In vaccination activities, we can find the three sources of wickedness. First, vaccination campaigns embrace multiple interests and values (social pluralism), e.g. this pandemic created trade-offs between public health and the competitiveness of economic activities due to lockdowns and restrictions. Second, health prevention services are provided through the collaboration of multiple public and private organizations (institutional complexity), i.e. national and regional governments, local health authorities, hospitals, private health providers, general practitioners, national armies, pharmaceutical companies, etc. Last, health prevention results are usually obtained in the long term, and it is not possible to univocally attribute these results to activities executed in the past (scientific uncertainty). Thus, it appears almost impossible to assess in advance the effectiveness of specific measures (i.e. vaccination strategy) as it depends on the other actors' behavior (virus – and its variants – included).
The case study was based on combined data collection tools obtained from a variety of sources: government open data about vaccines and epidemiological trends, official documents, institutional reports and data from the health authorities and other public entities (Lazio region, National Health Institute, Ministry of Health [2]) and three focus groups with key informants (the General Director of the Lazio Health System and his staff). In particular, government open data about vaccines (e.g. delivered doses) were used to develop the vaccines production and distribution section of the model. The epidemiological dynamics of the model (referring to variables such as population, deaths, healed, etc.) originate from open data on epidemiological trends, official documents and institutional reports. The Laboratorio Mes and Agenas (2021) report was employed to build up the communication section of the model. Two focus groups with key informants were held in October 2021. Each lasted about one hour and served to understand the regional health authority's organizational structure and the related model section (regarding hubs and human resources). A third two-hours-focus group was run in November 2021 to explain and discuss with key informants the research results, the strengths and weaknesses of the simulation model compared to the PM systems currently used to monitor the healthcare system and the results of vaccination strategies and – eventually – further application of this methodology in public health.
The DPM simulation model was developed and calibrated based on the data referred to the first six months of the vaccination campaign (1st January 2021–30th June 2021). The software used was Stella Architect®.
The model validation was performed according to the SD literature's requirements (Barlas, 1996; Sterman, 2000; Homer, 2012). An SD simulation model is indeed validated both in its structure and behavior. Structure validation tests assess the validity of the model structure by direct comparison with knowledge about real system structure (Barlas, 1996). In this regard, the model has been compared with knowledge about the system analyzed as emerging by the data, documents and reports used to develop the case study. The model has also been discussed in its sections during the focus groups with the key actors of the Lazio health system. Behavior validity is assessed by comparing model output with empirical data (Barlas, 1996; Homer, 2012). The validation process verifies the ability of the model to replicate the actual behavior of the variables considered during the reference period, performing extreme condition tests (Barlas, 1996) and partial model testing (Homer, 2012). Once the simulation model was developed, validated and tested to determine whether it realistically behaves, inputs were modified to conduct scenarios and sensitivity analyses of how short- and long-term results would change in response to alternative strategies and actions (Kunc and O'Brien, 2017; Torres et al., 2017; Noto and Cosenz, 2021).
Results
A high complexity characterizes the analysis of the vaccination campaign due to its numerous physical and social factors. A causal loop diagram (CLD) was designed to highlight the relationships between the key variables that emerged during the case study's development. CLD is a qualitative representation of a complex system often used in system thinking applications (Sahin et al., 2020). Figure 2 displays the vaccination campaign CLD.
As portrayed in Figure 2, the system analyzed is characterized by four key balancing loops (which depict dynamics tending to counteract any disturbance and move the system toward an equilibrium point) and five key reinforcing loops (dynamics driving toward exponential growth).
Table 1 explains the key system's dynamics represented in Figure 2.
While CLD is effective in representing the structure of a complex system, understanding the combined effect of the loops and their dominance requires building a proper simulation model, i.e. the so-called stock-and-flow diagram (SFD). This also allows adopting a DPM perspective by framing the abovementioned variables in terms of strategic resources, end-results and performance drivers (or indicators) (Bianchi, 2016).
The whole model structure comprehends four key sections: epidemiological (related to the infection diffusion model); production (related to the production of the vaccines); organizational (related to the creation and management of vaccination hubs and the recruitment of staff); and communication (related to the communication strategy and its effectiveness).
The graph in Figure 3 shows a synthetic version of the SFD framed according to the DPM logic – i.e. strategic resources, performance drivers and end-results. The complete list of variables used is reported in Appendix.
Table 2 describes in detail the key variables reported in Figure 3.
The results of the model are portrayed in Figures 4 and 5. Figure 4 shows the behavior of the key model variables, i.e. number of infected, number of death, number of doses delivered and the flows of vaccine administration (first dose, second dose and Janssen).
Figure 5 shows the result of the five performance indicators. The behaviors of the performance drivers allow one (e.g. policymakers, leaders, managers) to monitor the progress toward the success of the vaccination campaign.
Experimentation
The critical feature that allows DPM to deal with scientific uncertainty is the possibility of experimenting with the model and simulating scenarios based on alternative policies or model sensitiveness concerning specific parameters.
In the case analyzed, four alternative scenarios have been tested through simulation (see Figure 6) based on three key policy levers: communication effectiveness, vaccination production and restrictions.
Run 1 represents the simulation as is, i.e. the behavior validated through empirical data.
Run 2 shows the system behavior in case of low communication effectiveness on the vaccine campaign (from the original value of 0.7 estimated through Laboratorio Mes and Agenas (2021) data to 0.2), other conditions being equal. In this case, we may notice a worse performance of the vaccination campaign in terms of both outputs (doses delivered) and outcomes (infected and deaths).
Run 3 portrays the extreme scenario in which vaccines are not available. The only constraint to the virus is represented by the restriction policies leading to related impacts on the economy and the social well-being of the population. In this case and the previous one, it is possible to notice a worse performance in terms of infected people and deaths compared to the original scenario.
Run 4 shows cases where vaccines are unavailable, and no restriction policies are applied. Such a scenario shows an uncontrolled diffusion of the virus, especially in the first months, dramatically impacting the number of deaths.
Last, Run 5 portrays a scenario in which, other conditions being equal, restriction policies are strengthened. In this case, health outcomes show improved behavior at the expense of potential damage to the economy and the social well-being of the population.
The ones portrayed in Figure 6 are some of the possible simulations we can obtain through the DPM model developed to represent the vaccination campaign of the Lazio Region. Such experimentation allows us to appreciate the typical trade-offs implicit in health prevention activities and understand the system's functioning to design policies and monitor their performance through the set of performance drivers previously commented on.
Discussion
Wickedness is today recognized by both literature and practice as a key trait of current policy problems (Rittel and Webber, 1973; Head and Alford, 2015). As such, facing wickedness has become pivotal for every public manager (Massey, 2022).
Since the implementation of NPM, PM has been the most used approach to pursue efficiency and effectiveness in using resources and creating public value (Moore, 1995; Bryson et al., 2014). PM schemes have been widely applied in multiple settings of the health sector, from hospitals to public health and prevention (Vainieri et al., 2020).
Managing performance in wicked contexts is controversial. PAs and their managers are called to achieve targets to address problems that are not even definable – or, at least, there is no shared understanding of their traits. As such, public managers are called to deal with multiple trade-offs deriving (Head, 2008; Head and Alford, 2015) (1) from the social pluralism characterizing them, (2) from the presence of several institutions having a stake and an influence on their management and (3) from gaps on reliable knowledge about their implication.
Previous literature suggested collaboration and co-production for addressing those trade-offs deriving from social pluralism and institutional fragmentation (Cristofoli et al., 2017; Loeffler and Bovaird, 2019; Bianchi, 2021) thus trying to cope with the shared understanding of policy problems. However, a research gap still exists in dealing with scientific uncertainty.
According to Cairney et al. (2016), public administrations should rely on evidence to cope with uncertainty. However, some of the issues public organizations are dealing with are new, and there is no past (or similar) evidence that managers and leaders can rely on. As such, simulation may be a powerful tool to deal with knowledge gaps and learn how complex systems work (Kim et al., 2013; Sterman, 2014; Noto, 2022).
Combining simulation with PM schemes and system-wide perspective (e.g. a perspective taking into account the role, influence and interests of the stakeholder of a system) is proposed in this article as a possible solution to address wickedness. More in detail, the article suggests adopting the DPM approach (Bianchi, 2016). Based on the illustrative case developed, it is possible to discuss the potential benefits and pitfalls of the proposed approach.
Through the use of SD, DPM may bring in the perspectives and mental models of different stakeholders participating in the system analyzed (Vennix, 1999). As in our case study, the model “challenges the clouds” including and “endogenizing” in the model all the elements having an influence on the system and being influenced by it (Sterman, 2002; Noto and Bianchi, 2015; Nabavi et al., 2017).
DPM “speaks the same language” of healthcare and public health PM schemes. The instrumental perspective integrating strategic resources, performance drivers, and end-result can easily be overlain with the Donabedian (1966) and the Turning Point Performance Management Collaborative (2003) frameworks. Nevertheless, these traditional PM schemes suffer from several weaknesses, which the DPM may overcome.
First, they were developed to emphasize the role of PM systems in implementing quality improvement projects. At the same time, the DPM is flexible and can be adapted to several different health policy strategies.
Second, traditional public health PM schemes adopt typically backward-looking approaches (i.e. showing the effectiveness of past decisions) and are composed mainly of lagging (outcome) indicators. Therefore, they need to be accompanied by the development of intermediate results measures (i.e. performance drivers) for creating sustained performance improvement (Riley et al., 2010). The DPM, instead, is a comprehensive method based on a forward-looking approach suggesting expected trajectories of future performances and supporting strategy execution. The DPM model is designed to identify, monitor and simulate the behavior of those performance drivers representing intermediate results over time. These performance drivers are defined as causally linked to the system's end-results - or outcomes (Bianchi, 2016); due to this, they represent the key levers policymakers should act on to manage the system's performance. Decision-makers are indeed enabled to benchmark effective drivers' behavior with the simulated one and to put in place corrective actions in case of significant gaps. This is not possible when focusing exclusively on the long-term outcomes characterizing public health strategies (Hunter, 1990; Cinquini et al., 2014). Neither it is possible to effectively monitor intermediate results (e.g. vaccine doses delivered, number of inspections performed, etc.) without a target to benchmark with. Simulation may provide these benchmarks as intermediate results values are causally related to simulated expected outcomes. Decision-makers of the Lazio health system could have used this tool to evaluate policy trade-offs and monitor policy implementation processes by comparing results and simulated trends.
Third, the Turning Point Model, as well as the other approaches considered in the theoretical background, is mainly used for guiding the single public health department and, consequently, it cannot work appropriately for addressing public health issues or programs on which the actions of several different institutions may converge. On the contrary, DPM simulates complex systems on which public and private institutions, along with the preferences of the different population segments, can influence the achievement of results. As such, DPM embraces the suggestion provided by collaborative governance literature which asks for the engagement of multiple institutions and players having a stake in the public health issue (Weber and Khademian, 2008; Cristofoli et al., 2017).
Last, as highlighted by Schwartz and Pais (2009), the traditional assessment methods provide little insight into synergies, feedback loops and non-linearity in the complex interweaving of public health interventions. The result is that “those wishing to develop and refine better strategies therefore have insufficient knowledge as to which interventions to deploy in what sequence and in what combinations under varying contexts” (Schwartz and Pais, 2009, p. 20).
To overcome this latter weakness, DPM recognizes the feedback loops and non-linearity of interventions, simulates alternative scenarios (e.g. lockdown vs. vaccination) and tests parameters' sensitiveness (e.g. communication effectiveness). This feature allows decision-makers to experiment with the model and explore the trade-offs characterizing wicked contexts.
Conclusions
This article aims to fill a research gap related to PM in a wicked context such as public health. Based on this theoretical gap, this research suggests the adoption of DPM as an approach that may support policymakers in tackling social pluralism, institutional complexity and scientific uncertainty all at once. The result of this research highlights that this support comes from the following features: (1) system-wide perspective, (2) PM framework and (3) simulation.
The limits of the approach are mainly related to the inner characteristics of SD methodology. This last aims to comprehend the overall dynamic behavior of a system rather than performing a precise quantitative prediction (Meadows, 1980). Moreover, while SD adopts a system-wide perspective, it could be interesting to understand how single agents interact in determining the system performance (thus combining a system-based perspective with an agent one).
Other limitations of this study are related to the illustrative case developed. We chose the Italian Regional Health System case study, which best performs in the vaccination campaign in terms of vaccines delivered. However, other case studies – such as worst performers – could have brought interesting insights to the discussion.
As the limitations represent an opportunity to make suggestions for future research, we believe that future studies may develop multiple case studies to compare the strategies adopted to address the vaccination campaign. Additionally, comparing or combining the suggested approach with other simulation techniques, such as agent-based modelling or discrete event simulation, could be insightful.
Figures
Loops description
Loop | Description |
---|---|
R1 | R1 loop explains the dynamic according to which the more vulnerable population, the more people get infected. Once healed, these people get antibodies and can be considered immune for the following period |
R2 | This loop shows that the more people get vaccinated, the more population can be considered non-vulnerable. The vaccine effect will last for about 6 months, and then the vaccinated will be considered vulnerable again |
R3 | It shows the dynamic according to which the more vaccines are available, the more people get vaccinated. On the other hand, vaccination decreases the stock of vaccines available |
R4 | According to this loop, the more people get vaccinated, the fewer restrictions the health authorities impose on the population. These restrictions reduce contact between people and, thus, infections |
B1 | This loop portrays the dynamics according to which the more people get infected, the less the stock of vulnerable people is |
B2 | According to this loop, the more people get vaccinated, the less the stock of vulnerable people is |
B3 | This loop shows that the more people need vaccines, the more hubs, and human resources are activated for the vaccination campaign. This positively influences the number of vaccines inoculated, decreasing the stock of vulnerable people |
B4 & B5 | This loop portrays the dynamics according to which the more people get infected (or hospitalized), the more restrictions to reduce contact will be introduced |
Key variables specification
Name | Type | Description |
---|---|---|
Infected | Strategic resource | This represents the stock of people infected by COVID-19. Data to initialize the variable (and compare behavior) gained from the Ministry of Health website |
IC hosp | Strategic resource | The number of people infected and hospitalized in Intensive Care (IC) units. Data to initialize the variable (and compare behavior) gained from the Ministry of Health website |
Vulnerable | Strategic resource | The stock of healthy people, non-vaccinated, who can be infected. Variable initialized as 0, since the vaccination campaign started at the beginning of the period considered. Data to compare behavior obtained from government open data |
Vaccinated | Strategic resource | The stock of people vaccinated. Variable initialized as 0, since the vaccination campaign started at the beginning of the period considered. Data to compare behavior obtained from government open data |
Hub & HR | Strategic resource | This variable represents the health system's strategic resources consisting of the number of vaccination hubs and human resources recruited or assigned to the vaccination campaign. In the full running model hubs and staff are split into two stock variables. Data gained from the Lazio health system |
Comm. Level | Strategic resource | This is a stock variable representing the communication level of the vaccination campaign (initialized through Laboratorio MeS and Agenas (2021) data) |
Intensive care | Performance Driver | This variable is computed as the ratio between the stock of people hospitalized in intensive care departments and the intensive care beds available. A higher value of this indicator brings to the adoption of restriction policies to limit contact between people with a consequent effect on the economic activity and productivity of the region |
Vaccination | Performance driver | This driver compares the number of people fully vaccinated with the number of vulnerable people. Together with the production one, this driver is related to the number of hubs activated to vaccinate the population |
Production | Performance driver | This variable focuses on the ability of the pharmaceutical providers to respond to the vaccine demand computed as the population of vulnerable people. It is the ratio between the vaccines available and the vulnerable population and represents the production capacity constraint of the vaccination campaign |
Accessibility | Performance driver | It measures the adequacy of the organizational structure to perform a successful vaccination campaign. It directly impacts the accessibility of vaccines for the population in terms of availability and capillarity of the distribution network |
Communication | Performance driver | This driver measures the communication's effectiveness that incentivizes people to accept vaccination by comparing the actual communication level with the desired one |
Infection | End result | This variable measures the infection rate and mainly depends on the vaccination campaign results and the restriction policies |
Chg in hub & HR | End result | This result portrays the change in the strategic organizational resources (hubs and human resources) and depends on the magnitude of the pandemic |
Immunization | End result | This rate represents the flow of vaccines delivered to the vulnerable population, which is the direct result of the vaccination campaign |
Variables and equations
Run specs | |
---|---|
Start time | 0 |
Stop time | 27 |
DT | 1 |
Time units | Week |
Integration method | Euler |
Total | Count | Including array elements |
---|---|---|
Variables | 92 | 92 |
Stocks | 17 | 17 |
Flows | 26 | 26 |
Converters | 49 | 49 |
Constants | 26 | 26 |
Equations | 49 | 49 |
Graphicals | 7 | 7 |
Notes
https://www.epicentro.iss.it/vaccini/archivio (accessed on October 2021)
https://www.salute.gov.it/portale/nuovocoronavirus/dettaglioContenutiNuovoCoronavirus.jsp?lingua=italiano&id=5351&area=nuovoCoronavirus&menu=vu (accessed on February 2022), https://www.epicentro.iss.it/vaccini/archivio (accessed on February 2022), https://githu.com/pcm-dpc/COVID-19/tree/master/dati-regioni (accessed on February 2022)
Stocks
Communication_level(t) = Communication_level(t–dt) + (Chg_in_comm) * dt.
Deaths(t) = Deaths(t–dt) + (Death_rate) * dt.
Delivered_doses(t) = Delivered_doses(t–dt) + (doses) * dt.
First_dose_vaccinated(t) = First_dose_vaccinated(t–dt) + (Immunization_1_dose–Total_immunization - “1_vaccinated_infection”) * dt {NON-NEGATIVE}
Healed(t) = Healed(t–dt) + (Healing–Loosing_immunity) * dt {NON-NEGATIVE}
Hubs_-_lines(t) = “Hubs_-_lines” (t–dt) + (Chg_in_hubs) * dt.
Infected(t) = Infected(t–dt) + (Infection–Death_rate–Healing) * dt {NON-NEGATIVE}
Janssen_available(t) = Janssen_available(t–dt) + (Jansenn_production–Janssen_consumption) * dt.
Moderna_available(t) = Moderna_available(t–dt) + (Moderna_production–Moderna_consumption) * dt.
Pfizer_available(t) = Pfizer_available(t–dt) + (Pfizer_production–Pfizer_demand) * dt.
Staff(t) = Staff(t–dt) + (Recruitment) * dt.
Vaccinated(t) = Vaccinated(t–dt) + (Immunization_Janssen + Total_immunization + Vaccinated_healed–Vaccinated_infection–Loosing_protection) * dt {NON-NEGATIVE}
Vaccinated_deaths(t) = Vaccinated_deaths(t–dt) + (Vacc_death_rate) * dt.
Vaccinated_infected(t) = Vaccinated_infected(t–dt) + (Vaccinated_infection + “1_vaccinated_infection” - Vaccinated_healed–Vacc_death_rate) * dt {NON-NEGATIVE}
Vaccines_available(t) = Vaccines_available(t–dt) + (Vaccines_incoming–Vaccination) * dt.
Vaxevria_available(t) = Vaxevria_available(t–dt) + (Vaxevria_production–Vaxevria_consumption) * dt.
Vulnerable_population(t) = Vulnerable_population(t–dt) + (Loosing_immunity + Loosing_protection – Immunization_1_dose–Immunization_Janssen–Infection) * dt {NON-NEGATIVE}
Flows
1_vaccinated_infection = Contagious_contacts*Contagiousness* Probability_of_infection_compared_to_non_vaccinated_1.
Chg_in_comm = (Communication_lever-Communication_level)/Time_to_chg_comm
Chg_in_hubs = (Planned_hubs-“Hubs_-_lines”)/Time_to_build_hubs
Death_rate = Deadliness*Infected
Doses = Immunization_Janssen + Total_immunization + Immunization_1_dose.
Healing = Healing_rate*Infected.
Immunization_1_dose = IF Vaccines_available*Converter > Overall_productivity THEN MAX((Overall_productivity-“1_vaccinated_infection”)*Community_acceptance; 0) ELSE MAX(Vaccines_available*Converter*Community_acceptance; 0).
Immunization_Janssen = IF Janssen_available*Converter > Overall_productivity THEN MAX((Overall_productivity-“1_vaccinated_infection“-Immunization_1_dose)*Community_acceptance; 0) ELSE Janssen_available*Converter.
Infection = Contagious_contacts*Contagiousness.
Jansenn_production = LOOKUP(Janssen_capacity; TIME)*Vaccination_policy
Janssen_consumption = Immunization_Janssen
Loosing_immunity = Healed*Loosing_immunity_rate.
Loosing_protection = Loosing_immunity_rate*Vaccinated.
Moderna_consumption = MAX(Moderna_available/Delivery_time; 0).
Moderna_production = LOOKUP(Moderna_capacity; TIME)*Vaccination_policy
Pfizer_demand = MAX(Pfizer_available/Delivery_time; 0).
Pfizer_production = LOOKUP(Pfizer_capacity; TIME)*Vaccination_policy
Recruitment = (Desired_staff-Staff)/Time_to_recruit
Total_immunization = IF Vaccines_available*Converter > Overall_productivity THEN MIN(Overall_productivity; First_dose_vaccinated/Time_to_get_second_dose) ELSE Vaccines_available*Converter.
Vacc_death_rate = Vaccinated_deadliness*Vaccinated_infected
Vaccinated_healed = Vaccinated_infected*Healing_rate
Vaccinated_infection = Contagious_contacts*Contagiousness*Probability_of_infection_compared_to_non_vaccinated
Vaccination = (Immunization_1_dose + Total_immunization)*Vaccine_per_person
Vaccines_incoming = Moderna_consumption + Pfizer_demand + Vaxevria_consumption.
Vaxevria_consumption = MAX(Vaxevria_available/Delivery_time; 0).
Vaxevria_production = LOOKUP(Vaxevria_capacity; TIME)*Vaccination_policy
Converters and constants
Accessibility_PD = (“Hubs_-_lines”/Planned_hubs)*Human_resources_PD
Communication_lever
Communication_PD = Communication_level/Desired_communication_level
Community_acceptance = People_accepting_vaccination/Vulnerable_population
Contacts = Infected*Sociability.
Contagious_contacts = Contacts*Potential_concentration
Contagiousness.
Converter.
Deadliness.
Delivery_time
Desired_communication_level
Desired_staff = “Hubs_-_lines”*“People_per_hub-line”.
Factors = Communication_PD*Accessibility_PD
Fully_vaccinated = First_dose_vaccinated + Vaccinated + Vaccinated_infected + Vaccinated_deaths
Healing_rate
Hosp_rate
Hospetalizations = Infected*Hosp_rate
Human_resources_PD = Staff/Desired_staff
IC_capacity.
IC_rate.
Intensive_care = Infected*IC_rate.
Intensive_care_PD = Intensive_care/IC_capacity.
Janssen_capacity = GRAPH.
Loosing_immunity_rate.
Moderna_capacity = GRAPH.
Overall_productivity = Productivity*“Hubs_-_lines”.
People_accepting_vaccination = Vulnerable_population*Factors.
People_per_hub-line.
Pfizer_capacity = GRAPH.
Planned_hubs = Planned_hubs_1*Production_constrain
Planned_hubs_1 = GRAPH(Vaccination_PD).
Potential_concentration = Vulnerable_population/Potential_infection
Potential_infection = Infected + Vulnerable_population + Healed + First_dose_vaccinated + Vaccinated + Vaccinated_infected
Probability_of_infection_compared_to_non_vaccinated
Probability_of_infection_compared_to_non_vaccinated_1.
Production_constrain = GRAPH(Production_PD).
Production_PD = (Vaccines_available + Janssen_available)/Vulnerable_population
Productivity.
Restriction_policies = GRAPH(Intensive_care_PD).
Sociability = Restriction_policies
Time_to_build_hubs
Time_to_chg_comm
Time_to_get_second_dose
Time_to_recruit
Vaccinated_deadliness
Vaccination_PD = Vaccinated/Vulnerable_population
Vaccination_policy
Vaccine_per_person
Vaxevria_capacity = GRAPH.
References
Aidemark, L.G. (2001), “The meaning of balanced scorecards in the health care organisation”, Financial Accountability and Management, Vol. 17 No. 1, pp. 23-40.
Barlas, Y. (1996), “Formal aspects of model validity and validation in system dynamics”, System Dynamics Review, Vol. 12 No. 3, pp. 183-210.
Belle, N. and Cantarelli, P. (2022), “Your money, your life, or your freedom? A discrete‐choice experiment on trade‐offs during a public health crisis”, Public Administration Review, Vol. 82 No. 1, pp. 59-68.
Berwick, D. and Fox, D.M. (2016), “Evaluating the quality of medical care": donabedian's classic article 50 years later”, The Milbank Quarterly, Vol. 94 No. 2, p. 237.
Bianchi, C. (2016), Dynamic Performance Management, Springer, Berlin.
Bianchi, C. (2021), “Fostering sustainable community outcomes through policy networks: a dynamic performance governance approach”, in Handbook of Collaborative Public Management, Edward Elgar Publishing.
Bianchi, C., Bovaird, T. and Loeffler, E. (2017), “Applying a dynamic performance management framework to wicked issues: how co-production helps to transform young people's services in Surrey County Council, UK”, International Journal of Public Administration, Vol. 40 No. 10, pp. 833-846.
Blackman, T., Greene, A., Hunter, D.J., McKee, L., Elliott, E., Harrington, B., Marks, L. and Williams, G. (2006), “Performance assessment and wicked problems: the case of health inequalities”, Public Policy and Administration, Vol. 21 No. 2, pp. 66-80.
Bouckaert, G. and Halligan, J. (2007), Managing Performance: International Comparisons, Routledge, London.
Bryson, J.M., Crosby, B.C. and Bloomberg, L. (2014), “Public value governance: moving beyond traditional public administration and the new public management”, Public Administration Review, Vol. 74 No. 4, pp. 445-456.
Cairney, P., Oliver, K. and Wellstead, A. (2016), “To bridge the divide between evidence and policy: reduce ambiguity as much as uncertainty”, Public Administration Review, Vol. 76 No. 3, pp. 399-402.
Cepiku, D. (2017), “Collaborative governance”, in The Routledge Handbook of Global Public Policy and Administration, Routledge.
Christensen, T. and Lægreid, P. (2007), “The whole‐of‐government approach to public sector reform”, Public Administration Review, Vol. 67 No. 6, pp. 1059-1066.
Cinquini, L., Campanale, C. and Vainieri, M. (2014), “Measuring Efficiency in Public Health: the case of the final product system”, International Journal of Management Accounting Research, Vol. 4 No. 2, pp. 183-199.
Cosenz, F. (2018), “Supporting public sector management through simulation-based methods: a dynamic performance management approach”, International Review of Public Administration, Vol. 23 No. 1, pp. 20-36.
Costumato, L. (2021), “Collaboration among public organizations: a systematic literature review on determinants of interinstitutional performance”, International Journal of Public Sector Management, Vol. 34 No. 3, pp. 247-273.
Cristofoli, D., Meneguzzo, M. and Riccucci, N. (2017), “Collaborative administration: the management of successful networks”, Public Management Review, Vol. 19 No. 3, pp. 275-283.
DeAngelo, J.W., Beitsch, L.M., Beaudry, M.L., Corso, L.C., Estes, L.J. and Bialek, R.G. (2014), “Turning point revisited: launching the next generation of performance management in public health”, Journal of Public Health Management Practice, Vol. 20 No. 5, p. 463.
Deber, R. and Schwartz, R. (2016), “What's measured is not necessarily what matters: a cautionary story from public health”, Healthcare Policy, Vol. 12 No. 2, p. 52.
Donabedian, A. (1966), “Evaluating the quality of medical care”, The Milbank Memorial Fund Quarterly, Vol. 44 No. 3, pp. 166-206.
Drury, I. (2014), “Performance management for wicked problems: reflections from child welfare's front line”, Administrative Theory and Praxis, Vol. 36 No. 3, pp. 398-411.
Eisenhardt, K.M. and Graebner, M.E. (2007), “Theory building from cases: opportunities and challenges”, Academy of Management Journal, Vol. 50 No. 1, pp. 25-32.
Ferlie, E., Fitzgerald, L., McGivern, G., Dopson, S. and Bennett, C. (2011), “Public policy networks and 'wicked problems': a nascent solution?”, Public Administration, Vol. 89 No. 2, pp. 307-324.
Ferreira, A. and Otley, D. (2009), “The design and use of performance management systems: an extended framework for analysis”, Management Accounting Research, Vol. 20 No. 4, pp. 263-282.
Forrester, J. (1958), Industrial Dynamics, MIT Press, Cambridge, MA.
Größler, A., Thun, J.H. and Milling, P.M. (2008), “System dynamics as a structural theory in operations management”, Production and Operations Management, Vol. 17 No. 3, pp. 373-384.
Handler, A., Issel, M. and Turnock, B. (2001), “A conceptual framework to measure performance of the public health system”, American Journal of Public Health, Vol. 91 No. 8, pp. 1235-1239.
Head, B.W. (2008), “Wicked problems in public policy”, Public Policy, Vol. 3 No. 2, pp. 101-118.
Head, B.W. and Alford, J. (2015), “Wicked problems: implications for public policy and management”, Administration and Society, Vol. 47 No. 6, pp. 711-739.
Herrera, M.M., Cosenz, F. and Dyner, I. (2019), “How to support energy policy coordination? Findings from the Brazilian wind industry”, The Electricity Journal, Vol. 32 No. 8, 106636.
Homer, J.B. (2012), “Partial‐model testing as a validation tool for system dynamics (1983)”, System Dynamics Review, Vol. 28 No. 3, pp. 281-294.
Hood, C. (1991), “A public management for all seasons?”, Public Administration, Vol. 69 No. 1, pp. 3-19.
Hunter, D.J. (1990), “‘Managing the cracks’: management development for health care interfaces”, The International Journal of Health Planning and Management, Vol. 5 No. 1, pp. 7-14.
Ingram, R.C., Scutchfield, F.D., Charnigo, R. and Riddell, M.C. (2012), “Local public health system performance and community health outcomes”, American Journal of Preventive Medicine, Vol. 42 No. 3, pp. 214-220.
Jackson, P.M. and Stainsby, L. (2000), “The public manager in 2010: managing public sector networked organizations”, Public Money and Management, Vol. 20 No. 1, pp. 11-16.
Kim, H., MacDonald, R.H. and Andersen, D.F. (2013), “Simulation and managerial decision making: a double‐loop learning framework”, Public Administration Review, Vol. 73 No. 2, pp. 291-300.
Kunc, M. and O'Brien, F.A. (2017), “Exploring the development of a methodology for scenario use: combining scenario and resource mapping approaches”, Technological Forecasting and Social Change, Vol. 124, pp. 150-159.
Laboratorio Mes and Agenas (2021), “La comunicazione della vaccinazione anti COVID19 sui siti web delle Regioni e Province Autonome italiane”, available at: https://www.agenas.gov.it/comunicazione/primo-piano/1891-comunicazione-vaccinazione-anti-covid-19-siti-web-regioni-pa (accessed February 2022).
Lagreid, P. and Rykkja, L.H. (2015), “Organizing for ‘wicked problems’–analyzing coordination arrangements in two policy areas: internal security and the welfare administration”, International Journal of Public Sector Management, Vol. 28 No. 6, pp. 475-493.
Landrum, L.B. and Baker, S.L. (2004), “Managing complex systems: performance management in public health”, Journal of Public Health Management and Practice, Vol. 10 No. 1, pp. 13-18.
Lebas, M.J. (1995), “Performance measurement and performance management”, International Journal of Production Economics, Vol. 41 Nos 1-3, pp. 23-35.
Lega, F., Prenestini, A. and Spurgeon, P. (2013), “Is management essential to improving the performance and sustainability of health care systems and organizations? A systematic review and a roadmap for future studies”, Value in Health, Vol. 16 No. 1, pp. S46-S51.
Lenzi, J., Noto, G., Corazza, I., Lepiksone, J. and Fantini, M.P. (2020), “Measuring the quality of care in small countries: the empirical analysis of 30-day mortality following acute myocardial infarction and ischaemic stroke in Latvia”, Health Policy, Vol. 124 No. 7, pp. 695-700.
Loeffler, E. and Bovaird, T. (2019), “Co-commissioning of public services and outcomes in the UK: bringing co-production into the strategic commissioning cycle”, Public Money and Management, Vol. 39 No. 4, pp. 241-252.
Massey, A. (2022), “Silent Spring: can we fix wicked problems?”, Public Money and Management, Vol. 42 No. 2, pp. 53-54.
Meadows, D.H. (1980), “The unavoidable a priori”, in Elements of the System Dynamics Method, Pegasus Communications, Waltham, MA.
Meadows, D.H., Meadows, D.L. and Randers, J. (1972), The Limits to Growth, Yale University Press, New Haven, CT.
Moore, M.H. (1995), Creating Public Value: Strategic Management in Government, Harvard University Press, Cambridge, MA.
Nabavi, E., Daniell, K.A. and Najafi, H. (2017), “Boundary matters: the potential of system dynamics to support sustainability?”, Journal of Cleaner Production, Vol. 140, pp. 312-323.
Noto, G. (2022), “A pragmatic and systemic approach to advance research in health policy and management”, International Journal of Health Policy and Planning, Vol. 12, p. 7690, doi: 10.34172/ijhpm.2022.7690.
Noto, G. and Bianchi, C. (2015), “Dealing with multilevel governance and wicked problems in urban transportation systems: the case of Palermo municipality”, Systems, Vol. 3 No. 3, pp. 62-80.
Noto, G. and Cosenz, F. (2021), “Introducing a strategic perspective in lean thinking applications through system dynamics modelling: the dynamic value stream map”, Business Process Management Journal, Vol. 27 No. 1, pp. 306-327.
Noto, G., Lo Verso, A.C. and Barresi, G. (2021), “What is the performance in public hospitals? A longitudinal analysis of performance plans through topic modeling”, BMC Health Services Research, Vol. 21 No. 1, pp. 1-11.
Noto, G., Coletta, L. and Vainieri, M. (2022), “Measuring the performance of collaborative governance in food safety management: an Italian case study”, Public Money and Management, Vol. 42 No. 8, pp. 627-636.
Nuti, S., Noto, G., Vola, F. and Vainieri, M. (2018), “Let's play the patients music: a new generation of performance measurement systems in healthcare”, Management Decision, Vol. 56 No. 10, pp. 2252-2272.
Ouchi, W.G. (1979), “A conceptual framework for the design of organizational control mechanisms”, Management Science, Vol. 25 No. 9, pp. 833-848.
Papadopoulos, C.E. and Yeung, H. (2001), “Uncertainty estimation and Monte Carlo simulation method”, Flow Measurement and Instrumentation, Vol. 12 No. 4, pp. 291-298.
Rajala, T., Laihonen, H. and Haapala, P. (2018), “Why is dialogue on performance challenging in the public sector?”, Measuring Business Excellence, Vol. 22 No. 2, pp. 117-129.
Reckhow, K.H. (1994), “Importance of scientific uncertainty in decision making”, Environmental Management, Vol. 18 No. 2, pp. 161-166.
Riley, W.J., Parsons, H.M., Duffy, G.L., Moran, J.W. and Henry, B. (2010), “Realizing transformational change through quality improvement in public health”, Journal of Public Health Management Practice, Vol. 16 No. 1, pp. 72-78.
Riley, W.J., Elizabeth, J., Lownik, M., Scutchfield, F.D., Mays, G.P., Corso, L.C. and Beitsch, L.M. (2012), “Public health department accreditation: setting the research agenda”, American Journal of Preventive Medicine, Vol. 42 No. 3, pp. 263-271.
Rittel, W.J. and Webber, M.M. (1973), “Dilemmas in a general theory of planning”, Policy Sciences, Vol. 4 No. 2, pp. 155-169.
Roberts, N. (2000), “Wicked problems and network approaches to resolution”, International Public Management Review, Vol. 1 No. 1, pp. 1-19.
Robinson, V.A., Hunter, D. and Shortt, S.E.D. (2003), “Accountability in public health Units”, Canadian Journal of Public Health, Vol. 94 No. 5, pp. 391-396.
Sahin, O., Salim, H., Suprun, E., Richards, R., MacAskill, S., Heilgeist, S., Rutherford, S., Stewart, R.A. and Beal, C.D. (2020), “Developing a preliminary causal loop diagram for understanding the wicked complexity of the COVID-19 pandemic”, Systems, Vol. 8 No. 2, p. 20.
Scapens, R.W. (2004), “Doing case study research”, in The Real Life Guide to Accounting Research, Elsevier, pp. 257-279.
Schwartz, R. and Deber, R. (2016), “The performance measurement–management divide in public health”, Health Policy, Vol. 120 No. 3, pp. 273-280.
Schwartz, R. and Pais, G. (2009), “Challenges and approaches to evaluating comprehensive complex tobacco control strategies”, The Canadian Journal of Program Evaluation, Vol. 24 No. 3, p. 1.
Scutchfield, F.D., Knight, E.A., Kelly, A.V., Bhandari, M.W. and Vasilescu, I.P. (2004), “Local public health agency capacity and its relationship to public health system performance”, Journal of Public Health Management and Practice, Vol. 10 No. 3, pp. 204-215.
Steen, T. and Brandsen, T. (2020), “Co-Production during and after the COVID‐19 pandemic: will it last?”, Public Administration Review, Vol. 80 No. 5, pp. 851-855.
Sterman, J.D. (2000), Business Dynamics, McGraw-Hill, Cambridge, MA.
Sterman, J.D. (2002), “All models are wrong: reflections on becoming a systems scientist”, System Dynamics Review, Vol. 18 No. 4, pp. 501-531.
Sterman, J.D. (2014), “Interactive web‐based simulations for strategy and sustainability: the MIT Sloan LearningEdge management flight simulators, Part I”, System Dynamics Review, Vol. 30 Nos 1-2, pp. 89-121.
Torres, J.P., Kunc, M. and O'Brien, F. (2017), “Supporting strategy using system dynamics”, European Journal of Operational Research, Vol. 260 No. 3, pp. 1081-1094.
Turning Point Performance Management Collaborative (2003), From Silos to Systems: Using Performance Management to Improve the Public's Health, Public Health Foundation, WA DC.
Turrini, A., Cristofoli, D., Frosini, F. and Nasi, G. (2010), “Networking literature about determinants of network effectiveness”, Public Administration, Vol. 88 No. 2, pp. 528-550.
Vainieri, M., Noto, G., Ferrè, F. and Rosella, L.C. (2020), “A performance management system in healthcare for all seasons?”, International Journal of Environmental Research and Public Health, Vol. 17 No. 15, p. 5590.
Vennix, J.A.M (1999), “Group model–building: tackling messy problems”, System Dynamics Review, Vol. 15 No. 4, pp. 379-401.
Weber, J.A. (1999), “A response to public administration’s lack of a general theory of uncertainty: a theoretical vision of uncertainty”, Public Administration Quarterly, Vol. 23 No. 1, pp. 18-45.
Weber, E.P. and Khademian, A.M. (2008), “Wicked problems, knowledge challenges, and collaborative capacity builders in network settings”, Public Administration Review, Vol. 68 No. 2, pp. 334-349.
Weir, E., d'Entremont, N., Stalker, S., Kurji, K. and Robinson, V. (2009), “Applying the balanced scorecard to local public health performance measurement: deliberations and decisions”, BMC Public Health, Vol. 9 No. 1, pp. 1-7.
Welter, C. and Kim, S. (2018), “Effectuation under risk and uncertainty: a simulation model”, Journal of Business Venturing, Vol. 33 No. 1, pp. 100-116.
Acknowledgements
The authors gratefully acknowledge Massimo Annicchiarico, general director of the Lazio regional health department, and the key informants who participated in the research activities. The author would also like to thank the editors and the anonymous reviewers.
Corresponding author
About the authors
Guido Noto, Ph.D., is assistant professor of Business and Public Administration at the Department of Economics, University of Messina, Italy. He holds an International PhD in “Model Based Public Planning, Policy Design, and Management” at the University of Palermo. Prior to his current position, Guido Noto was post-doc at the Management and Health Laboratory of Sant’Anna School of Advanced Studies in Pisa, Italy. His research interests are mainly related to Performance Management in the public sector.
Anna Prenestini, Ph.D., is associate professor of Business Administration and Healthcare Management at the Department of Economics, Management, and Quantitative Methods (DEMM) and Research Fellow at the Center of Research and Advanced Education in Health Administration (CRC Head), University of Milan, Italy. She is also member of the Board of Director of the Milano School of Management. Her research interests are mainly related to strategic planning and control, performance management, organizational cultures, organizational design, and operations management in healthcare organizations.
Federico Cosenz, Ph.D., is associate professor of Business and Public Management at the University of Palermo, Italy. He holds an International PhD in “Model Based Public Planning, Policy Design, and Management”. He has published on several academic and professional journals. His main research interests focus on Business Model Design, Strategy, Performance Management Systems, Simulation for Business Planning and Strategic Organizational Learning.
Gustavo Barresi, Ph.D., is full professor of Management Accounting and Control at the Department of Economics, University of Messina, Italy. He holds a PhD in Business Management at the University of Catania. His research interests are mainly related to the design and implementation of planning and control systems in healthcare organizations.