Performance measurement and critical success factors: a case study of a national health service

Abstract

Purpose

The process of performance measurement provides support to company management to achieve the objectives established in strategic planning. Through the definition of critical success factors and related key performance indicators, performance measurement verifies the gap between planned objectives and the results achieved, informing the responsible bodies to enable them to evaluate performance and, if necessary, implement improvement actions. Although many types of companies adopt performance measurement, this process is challenging when applied to national health services. This paper aims to identify the evolution of performance measurement and the critical success factors of national health services.

Design/methodology/approach

The authors conducted an explorative case study of a leading national health service to delineate the evolutionary path of performance measurement and the main critical success factors.

Findings

The results indicate a significant increase in the maturity of performance measurement of a national health service that has been motivated by international reforms and national regulations. This research highlights performance measurement features such as a balanced set of metrics, targets, and incentives linked to strategic objectives and regular and frequent performance reviews. Furthermore, it identifies the performance measurement model of a leading national health service.

Originality/value

The evolution of performance measurement and numerous critical success factors of national health services are described; the critical success factors cover a wide range of financial to operational aspects such as patient safety, organizational appropriateness, and clinical appropriateness.

Citation

Sardi, A., Sorano, E., Tradori, V. and Ceruzzi, P. (2024), "Performance measurement and critical success factors: a case study of a national health service", International Journal of Productivity and Performance Management, Vol. 73 No. 11, pp. 270-293. https://doi.org/10.1108/IJPPM-05-2023-0238

1. Introduction

The scientific literature presents numerous studies on performance measurement and control management in various business sectors (Bedford, 2015; Gond et al., 2012; Kollberg and Elg, 2011; Otley, 2016; Pekkola et al., 2016; Tessier and Otley, 2012). These disciplines have been studied from many points of view; for instance, they have been investigated in the public and private sectors and in relation to many contingency factors (Ates et al., 2013; Bititci et al., 2012, 2016; Garengo and Sharma, 2014; Jardioui et al., 2019; Pekkola et al., 2016; Tessier and Otley, 2012). These topics have evolved from different literature streams: Performance measurement emerges from organizational and management control theories (e.g. Neely, 1995), while management control emerges from management accounting literature (e.g. Otley, 1999). Both disciplines highlight their key roles in monitoring and improving business performance and resource management and in promoting the cost-effectiveness of business administration, primarily in terms of effectiveness and efficiency (Neely et al., 1995; Otley, 1999), and, more recently, in terms of business quality, sustainability, compliance, and social impact (Betto et al., 2022; Cinaroglu and Baser, 2018; Khalid et al., 2019; Layman et al., 2023). Both performance measurement and control management are evolving toward a full integration of performance management, that is, how organizations use these measures to improve their business (Bititci et al., 2012; Tessier and Otley, 2012). Although these disciplines converge on the same key elements, they have evolved from related but parallel fields (Bititci, 2015). This paper focuses on the performance measurement literature because the foundations of performance measurement lie in the theories of organizational and management control (Smith and Bititci, 2017).

It can be difficult to apply performance measurement in the public sector (Betto et al., 2022; Conaty, 2012; Garengo and Sardi, 2021; Goh et al., 2015; Moullin et al., 2007; Yi et al., 2019). For instance, in the public healthcare sector, performance measurement must include multiple critical success factors (CSFs) such as cost reduction, appropriateness of healthcare, patient safety, and quality of treatments (Adinolfi and Borgonovi, 2017; Francesconi et al., 2018; Inamdar et al., 2002). As stated by Kaplan and Porter (2011, p. 47), “The biggest problem with health care isn’t with insurance or politics. It’s that we’re measuring the wrong things the wrong way” (Kaplan and Porter, 2011). The use of performance measurement in the public healthcare sector has undergone numerous modifications in recent decades, and these changes were engendered by international reforms, such as new public management reforms (Hood, 1991) and new public governance (Osborne, 2010), as well as national legislation (Sorano et al., 2023). The current configuration of performance measurement, therefore, is the result of a long evolutionary process driven by numerous factors (Vagnoni, 2004). Although the literature outlines the key role of performance measurement for monitoring the sustainability of national health services (NHSs), it rarely presents empirical and theoretical studies on how performance measurement is evolving in NHSs and what CSFs should be monitored from the performance measurement of an NHS to control its sustainability (Betto et al., 2022).

The fundamental assumption of this study is that the design of the performance measurement for an NHS is imposed mainly by the regulations of a given nation and promoted by international and political scenarios. That is one of the main failures of the use of main performance measurement models such as the balanced scorecard and performance prism (Betto et al., 2022). This article aims to fill this literature and practical gap that is the definition of a model useful for measuring performance of national health services. Having said that, this article aims to detect the evolution of performance measurement and CSFs of an NHS, as indicated by the current national legislation, in order to define a performance measurement model for NHS. Through an explorative case study developed in an NHS, the following research questions are addressed in this paper:

RQ1.

How is performance measurement evolving in national health services?

RQ2.

What are the critical success factors of national health services?

The article is organized into several sections. The second section presents the literature background, and the third section explains the method adopted for the research. The fourth section highlights the case study, and the fifth section discusses the findings. Finally, the main contributions, implications, and limitations of this research are outlined in the sixth section, and future research opportunities on this challenging topic are presented.

2. Literature background

At the beginning of the 1900s, the Anglo–Saxon school disseminated the theory of scientific management through scholars such as Taylor, Newman, and Koontz; it aimed to achieve business efficiency measured in terms of production costs (Taylor, 1910). Starting in the second half of the last century, management process was investigated in the scientific literature, with a specific focus on the organizational process. According to Anthony (1965), control management is the process by which companies seek to ensure that productive resources are obtained and used effectively and efficiently in pursuit of corporate objectives in both the short and long terms (Anthony, 1965). Anthony developed a systematic performance measurement characterized by three main steps—strategic planning, directional control, and operational control—and continuous monitoring ensures the connection among these steps. Some studies subsequently suggested that companies also closely monitor the environment, market, customers, and competition, which allows them to gain a competitive advantage over their rivals (Ansoff, 1976).

According to Deming (1982), management also had to evaluate innovative plans, protect investments, ensure future dividends, and provide more jobs through improved products and services. Deming (1982) promoted quality management and continuous improvement (Anderson et al., 1994), stating that such a process favors the ability to remove waste, reduce errors, and avoid delays. Reducing costs through process improvement and, at the same time, improving productivity and product effectiveness allows a company to benefit from positive impacts on product and service quality (Deming, 1982). Deming supported the principle that each organization is built on a system of interrelated processes and people that comprise the system’s components (Deming, 1993). To respond to new business needs, scholars proposed innovative holistic and multi-perspective performance measurement models, such as the balanced scorecard and performance prism (Kaplan and Norton, 1992; Neely and Adams, 2001).

From this management control literature, scholars have recognized performance measurement as a process that includes the integrated planning of the activity and the comparison between objectives and results (Bititci et al., 1997; Neely et al., 1995) regarding additional perspectives, such as financial, customer, internal, and learning and growth perspectives (Kaplan and Norton, 1992, 2004). Performance measurement systems monitor qualitative and quantitative objectives that allow the measurement of the effectiveness and efficiency of a strategic action (Bititci, 2015). Recently, the literature has integrated performance measurement and performance management (Smith and Bititci, 2017): The former refers to the process of setting goals; developing a set of performance measures; and collecting, analyzing, reporting, interpreting, reviewing, and acting on performance data (Bititci, 2015); while the latter refers to the process that defines the means of using the measures for managing an organization’s performance (Smith and Bititci, 2017). Therefore, the literature has moved from the concept of a performance measurement system to performance measurement and management systems. These can be considered as holistic, dynamic, and balanced systems that are integrated into the management control system (Bianchi et al., 2017; Bititci et al., 2012; Kaplan and Norton, 1992). This system allows for the monitoring of an organization’s strategy and the managing of its performance (Bititci and Muir, 1997; Ittner and Larcker, 2003; Kaplan and Norton, 1992; Neely and Adams, 2001).

Health companies use the balanced scorecard as their major performance measurement and management model to monitor key performance indicators (Betto et al., 2022; Broccardo, 2015; Gonzalez-Sanchez et al., 2018; Gurd and Gao, 2008; Naranjo-Gil et al., 2016). It is a holistic set of financial and non-financial measures that provide information from various perspectives. Kaplan and Norton used the concept of CSFs to translate corporate strategy. CSFs represent the key elements for an organization to achieve its mission; they address the question of what companies must do to become successful (Kaplan and Norton, 1992). Rockart (1979, p. 85) stated that

Critical success factors are, for any business, the limited number of areas in which results, if they are satisfactory, will ensure successful competitive performance for the organization. They are the few key areas where “things must go right” for the business to flourish. If results in these areas are not adequate, the organization’s efforts for the period will be less than desired. As a result, the critical success factors are areas of activity that should receive constant and careful attention from management. The current status of performance in each area should be continually measured, and that information should be made available. (Rockart, 1979)

The identification of CSFs is more challenging in NHSs, as this area is characterized by, for instance, high diversity, multiple stakeholders, process complexity, and health professional autonomy (Adinolfi and Borgonovi, 2017). Furthermore, public companies differ from private firms (Garengo and Sardi, 2021). Public companies are characterized by features such as intangibility (i.e. difficulty in defining and measuring the service), heterogeneity (i.e. dependence from numerous stakeholders), and inseparability (i.e. the provision of the service equals and is contextual to its consumption) (Fryer et al., 2007).

Performance measurement involves the verification of the objectives and allows administrative and healthcare managers to optimize the relationship between costs and results and ensure the timely application of corrective interventions (Broccardo, 2015; Gonzalez-Sanchez et al., 2018; Naranjo-Gil et al., 2016). On the one hand, a public health company must provide appropriate healthcare services to everyone; on the other hand, it must guarantee the cost-effectiveness, efficiency, and appropriateness of the organizational, administrative, and assistance components in compliance with the laws and the rights of citizens. The performance measurement of public health companies is essential for cost containment and the adequate satisfaction of user needs.

Although performance measurement is mature in almost all business sectors (Sardi et al., 2021; Smith and Bititci, 2017), performance measurement of public health companies remains a significant challenge in both theoretical and practical aspects (Betto et al., 2022). Public organizations have implemented the performance measurement model adapted from the private sector, however, the specific needs of the public sector are different from those of the private sector, and the literature evolution is still too generic to answer these needs (Sorano et al., 2023). Furthermore, the rules and regulations governing each public system influence the methods used for performance measurement and the definition of some CSFs (DiMascio and Natalini, 2013; Sardi et al., 2020).

This article aims to identify the development of performance measurement and CSFs of NHSs through an explorative case study of an NHS. To detect the evolutionary path of performance measurement, the research adopts the features of the conceptual framework proposed by Smith and Bititci (2017). According to Smith and Bititci (2017), the maturity of performance measurement depends on the full achievement of characteristics including a balanced set of metrics, a high degree of awareness of the causal relationship, strategic measures deployed to lower levels, targets and incentives linked to strategic objectives, managers with the appropriate span of accountability and control, and a short interval review. This conceptual framework was validated and cited in numerous international studies that discuss the interplay between the performance measurement and performance management processes to inspire employee engagement and produce better organizational performance (Betto et al., 2022; Castelo and Gomes, 2023; Demartini and Taticchi, 2022; Naeem and Garengo, 2022; Wang et al., 2016).

3. Methodology

An explorative case-study approach was adopted in the empirical investigation (Yin, 2018). Numerous studies have employed this approach to examine complex situations within their real and dynamic contexts (Easterby-Smith et al., 2021; Eisenhardt, 1989). It favors the investigation of a phenomenon in depth and allows a better understanding of it (Yin, 2018). This type of case study is conducted when the topic under investigation is highly complex or, as suggested by Yin (2018), when there is scant literature or theory on the research topic. Furthermore, an explorative case study allows significant flexibility for researchers to adapt the study methods as they delve deeper into the subject (Pan and Scarbrough, 1999). The case study adopts a holistic approach, considering multiple aspects and perspectives related to the phenomenon under investigation. This research follows three main steps of explorative case studies: research topic definition, sample selection, and data collection and data analysis (Yin, 2018).

First, the research topics are defined, and this research explores performance measurement and CSF evolution of an NHS. As defined by Bititci (2015, p. 17), performance measurement is “the process of collecting, analyzing, and reporting information regarding the performance of an action,” while the concept of CSFs has been described by Kaplan and Norton (1992, p. 73) as the business areas that “force managers to focus on the handful of most critical measures.” Therefore, an NHS, as defined by the England National Health Service, is “a complex system, which can sometimes make it difficult to understand – especially working out who is responsible for what. It’s made up of a wide range of different organisations with different roles, responsibilities, and specialities. These organisations provide a variety of services and support to patients and careers” (England NHS, 2022).

Second, the sample is defined. The Italian NHS was selected for analysis because it is recognized as one of the major NHSs (Rizzi et al., 2021). Italian public (€131bn) and private (€40bn) health spending amounted to €170bn in 2022, equal to 9.5% of the country’s gross domestic product, and Italian public health spending alone is equal to 6.9%. The Italian NHS is one of the better in the number of typologies of health services offered, expenses, and life expectations (Italian National Institute of Statistics, 2023). For instance, it offers approximately 5,700 health services including three levels of essential assistance (LEAs): collective prevention and public health, district assistance, and hospital assistance (Ministry of Health, 2024). Furthermore, as reported by the official website of the Italian Ministry of Health,

The National Health Service makes the right to health accessible to all citizens, without discrimination based on income, gender, or age. The NHS provides high-quality healthcare to all citizens for hospital care, emergency care, and primary care provided by general practitioners and paediatricians. In addition, the NHS covers a wide range of pharmaceuticals and all the hospital and diagnostic services essential for health. (Ministry of Health, 2023)

For instance, “The NHS is a leader in Europe in terms of free medicines; it has the largest number of prescription drugs dispensed by a National Health Service in Europe: In Italy, all [of] the drugs for the treatment of serious and chronic diseases, including last-generation innovative medicines, are offered free of charge.” According to the Annual Report of the Pharmaceuticals Observatory Year 2021, 70% of national pharmaceutical expenditures are covered by Italy’s NHS. Total pharmaceutical spending reached €29.9bn in 2021 (€508 per capita), of which €20.9bn was covered by the NHS and €8.9bn by patients (Cergas Bocconi Centre, 2022).

Third, data collection is described. To define the performance measurement and CSF evolution of the Italian NHS, data were gathered from regulations published on official institutional websites and through semi-structured interviews (Yin, 2018). The main objective of data collection was to gauge the regulation of performance measurement and to highlight the CSFs. Data collection was conducted from March 2022 to November 2023. The following information was gathered:

1. impacts of the main NHS regulations on performance measurement (see Table 2),

2. CSFs related to these regulations (see Table 2), and

3. performance measurement assessment (see Table 3).

Data collection from documents is a common and valuable method in research—especially in fields such as business, management, and accounting—to systematically collect existing documents and extract relevant data (Yin, 2018). To assist in answering the research questions, data were initially collected from the following national legislation sources: the constitution, legislative decrees, laws, ministerial decrees, et cetera. The following official government websites were used to find these sources:

1. https://www.gazzettaufficiale.it/(last accessed on November 5, 2023),

2. https://www.salute.gov.it/(last accessed on November 5, 2023),

3. https://www.agenas.gov.it/(last accessed on November 5, 2023), and

4. https://www.istat.it/it/(last accessed on November 5, 2023).

Data collection from semi-structured interviews involves conducting interviews in which the researcher has a set of predetermined questions but also has the flexibility to explore additional topics or probe more deeply based on the participant’s responses (Yin, 2018). Semi-structured interviews were adopted to produce additional data on the design of performance measurement and the CSFs. The questions were: What are the main national health service regulations? What are the impacts of these regulations on performance measurement?

Four researchers conducted 26 semi-structured interviews involving 24 managers and directors of public health companies included in the Italian NHS, a manager of a regional health service, and a manager of NHS with at least 25 years of work experience. Researchers interviewed employees in the following job positions: chief executive officer of health companies, chief financial officer of health companies, chief healthcare officer of health companies, chief risk officer of health companies, chief facility officer of health companies, control management manager of health companies, a control manager of a regional health service, and a control manager of NHS. Data collection produced the following documents (Table 1).

Participants in the semi-structured interviews assessed the maturity of performance measurement of the NHS through the conceptual framework proposed by Smith and Bititci (2017). Using a scale from 0 (no features indicated) to 5 (features fully portrayed), managers and directors assessed performance measurement in different years: 1992 (historical evaluation, the first foundation year of the actual Italian NHS) and 2023 (the actual evaluation). The interviewees provided the following assessment as suggested by Smith and Bititci (2017):

Finally, data analysis is described. Data analysis is a critical step in the research process, involving the examination, interpretation, and transformation of raw data into meaningful insights (Yin, 2018). Before analyzing the data, the researchers cleaned and prepared the raw data for analysis, removing outliers and inconsistent values. Then, the researchers conducted a rigorous data analysis to provide meaningful insights and contribute to the overall understanding of the research topic.

The case study was examined via a pre-understanding of the context and a within-case study (Eisenhardt and Graebner, 2007; Yin, 2018). The first analysis was the pre-understanding of the context; it aimed to highlight the main information about the case study, including the numbers and typologies of health services provided by the NHS, the number of accredited public and private facilities, and the main information about management control. The second analysis was the within-case study, which is an in-depth exploration of a unique case as a single entity (Mills et al., 2010). The within-case study, based on the qualitative analysis, favored the understanding of the evolutionary process of the NHS and the impact of the main regulations on the performance measurement and CSFs. Then, the research assessed the evolutionary path of performance measurement characteristics by adopting the conceptual framework suggested by Smith and Bititci (2017).

As suggested by numerous scholars, data triangulation was adopted using multiple sources or methods to collect and analyze data. This process improved the credibility and reliability of the findings by cross-verifying information in different ways (Easterby-Smith et al., 2021; Eisenhardt, 1989; Yin, 2018). This research used two types of triangulation. First, it adopted data source triangulation using multiple sources of data to gain a comprehensive understanding of the performance measurement evolution, combining information from documents provided from official websites, regulations, and semi-structured interviews with employees of both public health companies and the NHS. Second, four researchers collected and analyzed data independently. This triangulation avoided the personal biases or preconceptions of a single researcher and did not influence the findings.

4. Results

The first analysis highlights the context of the explorative case study. It describes the main information on the Italian NHS, such as the financing of NHS, the numbers and typologies of health services provided by the NHS, and the number of accredited public and private facilities (see Table 2). It illustrates a highly complex and challenging organization for designing, implementing, and using performance measurement.

The actual NHS was established by Law 833/1978, which defined the principles covering the rights to health, generality, equity, gratuity, and free access to the NHS. The management was entrusted to the local health units that were without legal personality and were designed by the municipalities with high political representation. The lack of institutional and managerial structures and poor control of public spending were serious issues that led to corporatization (Legislative Decree 502/1992) and that transformed the local health authorities into local health companies with public juridical personality and entrepreneurial autonomy. According to this decree, each region is now responsible for health planning, and each health company adopts management control to monitor the effectiveness and efficiency of healthcare activities as well as guarantee the quality and management of financial resources. This decree increased their responsibility after the constitutional reform of Article 117 in 2001.

The NHS is structured on three levels: The first concerns the central government, the second the 20 regional governments, and the third the health companies. The standard national health requirement of the NHS is defined each year by a specific law. This standard national health requirement represents the overall level of resources of the NHS to which the State contributes; it is determined in accordance with the overall macroeconomic framework and with public finance constraints and the obligations assumed by Italy at the European Union level, consistently with the requirement deriving from the determination of the LEAs provided in efficient conditions and appropriateness.

The standard national health requirement is regulated by Legislative Decree 56/2000 and is financed mainly by the general taxation of the regions and the NHS bodies’ own revenues, such as tickets and revenues deriving from the intramoenic activity of their employees. In 2022, the standard national healthcare requirement was estimated at approximately €126bn for a population of approximately 59 million people, of whom 23.8% were over 65 years of age. It has about 620,000 employees. These resources finance the LEAs, which are the benefits and services that the NHS is required to provide to all citizens, free of charge or upon payment of a copay. The three essential levels of care include approximately 5,700 health services offered at approximately 27,000 accredited public and private facilities.

The three major LEAs identified by the DPC of January 12, 2017, are (https://www.salute.gov.it/):

1. Collective prevention and public health, financed with 5% of the national health fund, which includes the following prevention activities aimed at communities and individuals.

2. District assistance, financed with 44% of the national health fund, focused on health and social–health activities and services spread across the territory.

3. Hospital care, financed with 51% of the National Health Fund.

A reorganization of the hospital and territorial network is current underway to achieve a balance between the role of the hospital and that of territorial services in healthcare, adapting to the tight financial constraints.

Approximately 51% of the organizational and economic resources in the healthcare sector are directed to hospital-level care. The number of ordinary beds, the hospitalization rate, and the average length of stay are indicators that allow an overall assessment of the level of resources used, both in terms of availability of the hospital offer and the possibility of treating a patient in hospital. Ministerial Decree of April 2, 2015, n. 70, defined for the regions, as reference parameters, a maximum hospitalization rate equal to 160 hospitalizations per thousand inhabitants and a supply of beds not exceeding 3.7 per thousand inhabitants. In 2021 in Italy, the number of beds under ordinary conditions amounted to 209,568, 3.5 per thousand inhabitants, and the average hospital stay for ordinary hospitalizations was 8.5 days.

Residential and semi-residential care facilities play an important role in ensuring an adequate response to the healthcare demand of non-self-sufficient people or those with serious health problems while keeping them out of the hospital. In 2021, the number of beds in healthcare facilities for residential care was 266,000, 44.8 beds per 10,000 inhabitants, while those numbers for semi-residential healthcare facilities were 56,000, 9.6 beds per 10,000 thousand inhabitants.

Territorial assistance is at the center of a profound reform project aimed at modernizing the system of extra-hospital care to make it more accessible and closer to the needs of citizens. The Decree of May 23, 2022, n. 77, dictates the principles of the reorganization of the territorial assistance system through the drafting of a regulation that defines the development, models, and standards that inspire the new system. The decree establishes the pillars of primary care and, in particular, community houses that are designed to respond to the health and social health needs of the reference population. It also promotes the strengthening of home care and the integration between health and social care through the development of multi-professional teams to care for the patient. Furthermore, the decree establishes the creation of digitalized services to encourage home care through telemedicine and telemonitoring tools and the integration of the professional network that operates in the area and in the hospital. While waiting for the reform process to be perfected, this analysis documents the number of general practitioners and pediatricians of free choice affiliated with the NHS, who constitute the fulcrum around which local healthcare currently revolves. These are healthcare professionals who have the role of assessing the citizens’ healthcare needs, assisting them, and, in the most serious cases, entrusting them to the care of specialist doctors or directing them to other NHS healthcare facilities for diagnostic tests.

Finally, this analysis highlights that each region is responsible for healthcare planning and the management control and quality assessment of health services, and it proposes a guideline for the development of management control in a healthcare company. According to Legislative Decree 502/1992, health companies adopt management control to monitor the effectiveness and efficiency of healthcare activities and to guarantee the quality and management of financial resources. Each healthcare company must define the main elements useful for the design, implementation, and use of management control.

The second analysis presents the evolutionary process of the Italian NHS based on the regulations’ impact on the performance measurement and the CSF factors that emerged (see Table 3).

According to Legislative Decree 502/1992, the regions are responsible for the strategic organization of services and activities intended for the protection of health, the financing criteria of hospital companies, and the technical direction, promotion, and support activities of local health companies in relation to management control and evaluation of the quality of healthcare services. The regions regulate the organizational and functioning methods of the companies within their competence, providing, among other services, the methods of supervision and control over the local health units. The regions verify compliance with the provisions regarding minimum requirements and classification of the providing structures, with particular regard to the requirements relating to the quality control activities of services, and conduct scheduled interventions to evaluate the quality of care. This legislative decree requires the adoption of analytical accounting with the aim of supporting management control activities. Therefore, to design management control, health companies must follow the central and regional guidelines for the controls.

The internal controls for the regions and the health companies are multiple. Internal control for public administrations involves directing, evaluating, and correcting organizational activities concerning the assigned objectives (Legislative Decree 29/1993). Efficiency becomes a CSF of the public administration due to its responsibility for any failure to achieve objectives. Through comparative evaluations of costs and returns, the achievement of objectives, cost-effectiveness of management, impartiality, and efficient performance of the administrative action are verified, according to the control parameters that are defined annually. According to Legislative Decree 286/1999, four controls are required: internal audit, administrative, and accounting control; management control; executive evaluation; and strategic control. These controls ensure the functioning of the administrative organization. Each public administration defines the main aspects of management control as indicated in Table 2 (Article 1 Legislative Decree 286/1999).

The management control system uses the integrated information–statistical system for economic–financial data relating to the accounting reporting of the single administration, personnel management, institutional activities, administrative expenses, and analytical accounting systems and procedures. Legislative Decree 286/1999 identified the procedure and responsibility for the evaluation of managerial performance. In particular, the general director has control and organizational powers and has a twofold task: respect the congruity between the services provided and the available resources and guarantee the quality of these services. To fulfill the tasks, the general director should be aware of all company costs. Such cognizance should be supported by the economic–financial general accounting system, particularly two management control tools: (1) analytical accounting by cost centers, which favors the analysis and the attribution of the cost structure to each organizational unit, and (2) the management system for responsibility centers, which allows for the allocation of the budget set by management among the specific areas of responsibility within the company. To limit the continuous increase in public spending, various rules have been introduced. An important provision was Law 311/2004, which required respect for the economic and financial balance of the regional health service during regional planning. Regions and, therefore, health companies must adopt analytical accounting by cost and responsibility centers.

Consequently, DPCM 29/2001 increased the performance measurement framework by defining the LEAs and the approximately 5,700 health services guaranteed to citizens with public resources. The LEA Committee verifies the provision of LEAs in conditions of appropriateness and efficiency in the use of resources as well as the congruity between the services to be provided and the resources made available by the NHS. The Ministerial Decree of March 12, 2019, introduced the new guarantee system, which is designed for monitoring and evaluating health activities. The purpose of the system is to evaluate the efficiency, organizational appropriateness, clinical appropriateness, and patient safety of the LEAs. The system provides for the monitoring and evaluation of the level of diagnostic–therapeutic–assistance pathways for specific categories of needs or health conditions, quality and humanization of care, and geographic and social equity. In addition, the system promotes consistency between national monitoring and verification activities and evaluation systems intra-regionally, also favoring audit activities and consequent interventions adopted by the regions identified for the improvement of the provision of the LEAs. The guarantee system also includes verification, review, and monitoring methodologies. A total of 88 indicators identified in the Ministerial Decree of March 12, 2019, are distributed by macro-areas:

1. 16 for collective prevention and public health,

2. 33 for district assistance,

3. 24 for hospital care,

4. 4 context indicators for estimating healthcare needs,

5. 1 social equity indicator, and

6. 10 indicators for monitoring and evaluating diagnostic–therapeutic care pathways.

Within the guarantee system, 22 indicators have been identified for evaluation of the provision of LEAs (https://www.salute.gov.it/). These indicators are divided into three macro areas: collective prevention in public health, district assistance, and hospital care. Legislative Decree 150/2009 reforms the regulation of the employment relationship of public administration employees by intervening in the field of collective bargaining, evaluation of public administration structures and personnel, valorization of merit, promotion of equal opportunities, and management of public and disciplinary responsibility. The decree improves the organization of work and the qualitative and economic standards of functions and services by encouraging the quality of work performance, the selectivity and competitiveness in career progression, the recognition of merits and demerits, and the selectivity and valorization of skills and results for the purposes of managerial positions. Furthermore, the decree aims to strengthen the powers and responsibility of the management, increase the efficiency of public work, improve the productivity and transparency of the work of public administrations, and guarantee their legality.

Legislative Decree 68/2011 provides for the autonomy of entry of the regions and the determination of costs and standard needs in the healthcare sector. It also verifies the provision of LEAs in conditions of efficiency and appropriateness. Law 118/2011 harmonizes the accounting systems and balance sheet layouts of the regions, local authorities, and their bodies. Additionally, it verifies the provision of LEAs in conditions of efficiency and appropriateness in line with the management report. Legislative Decree 123/2011 rationalizes and reorganizes all of the regulatory provisions. Moreover, it reforms administrative and accounting controls and strengthens the analysis and evaluation of expenditures. It also closely focuses on the audits of the boards of auditors and trade unions at public entities and bodies. The main points addressed by Legislative Decree 123/2011 are as follows:

1. the identification of the parties involved in administrative and accounting controls,

2. the issuance of principles for the analysis and evaluation of expenditures as a type of monitoring to be conducted within the framework of the analysis and evaluation nuclei established by Article 39 of Law 196/2009, and

3. the identification of the accounting documents to control.

The publication of Legislative Decree 118/2011 outlined a more precise regulatory framework regarding management control and performance evaluation. The decree established the organizational and management requirements of health companies and defined the methods of performance evaluation and management control. Furthermore, Ministerial Decree 70/2015 established the criteria for drafting the management plan of health companies and defined the methods for monitoring activities and results. It also delineated the structural and technological qualitative and quantitative standards of the facilities dedicated to hospital assistance.

Law 190/2012 regulates the prevention of corruption and illegality in public administration. Subsequently, Legislative Decree 33/2013, the so-called transparency decree, reorganizes the existing legislation, thereby providing a unitary regulation of administrative transparency. The transparency of public organizations is defined as the total accessibility of data and documents held by public administrations to protect the rights of citizens, promote stakeholder participation, and allow for control over the pursuit of institutional functions and the use of public resources.

The Ministerial Decree of June 21, 2016, introduced the National Outcomes Program, which provides national comparative assessments of health service companies in terms of efficacy, safety, efficiency, and quality of care. The planning, management, and definition are undertaken by the National Agency for Regional Health Services. The National Outcomes Program supports clinical and organizational auditing programs to improve the effectiveness and equity of the NHS.

Law 24/2017 focuses on the safety of care of an assisted person and on the subject of professional liability of healthcare professionals, seeking to improve healthcare risk management and patient safety.

According to the Ministerial Decree of May 24, 2019, health companies must produce new models of the budget, quarterly and final income statements, and the final balance sheet. The new models favor standardization and, consequently, allow for comparison of the results through analytical accounting.

In the event of failure to achieve the objectives of the regional health service, Law 311/2004 provides for the development of operational programs for the reorganization, requalification, and strengthening of the regional health service, defined as recovery plans. The recovery plans must contain the measures to rebalance the provision profile of the LEAs to make them compliant with the national programming, the standards of the LEAs, and the measures to guarantee the balance of the healthcare budget. If the periodic monitoring of the recovery plan highlights negative results in terms of economic and welfare (e.g. serious economic and financial deficits and failure or poor provision of LEAs), the Council of Ministers initiates the procedure for placing the region under commissionership. The commissioner then adopts all of the measures indicated by the plan as well as any further regulatory, administrative, organizational, and management acts or provisions related to the complete implementation of the plan. Furthermore, with the appointment of the commissioner, various sanctioning measures are activated, including the automatic increase in the rates of some taxes.

According to Law 28 of December 2015, n. 208, comma 524, each region must identify public health companies that present one or both of the following conditions:

1. a difference between the costs recognized by the final income statement recognition model and the revenues determined as remuneration for the activity or greater than 7% of the aforementioned revenues, or, in absolute value, equal to at least €7 million;

2. failure to comply with the parameters relating to the volume, quality, and outcomes of care. For each hospital institution, these measures are determined for each clinical area, making reference to hospital discharges in a specific year.

The identified entities must present to the region, within 90 days of the issuance of the identification provision, a recovery plan lasting no more than three years, containing the measures aimed at achieving economic–financial and equity balance and improving the quality of care or the adaptation of the offer, to improve the negative results.

The third analysis describes the evolution of performance measurement from chiefs and managers employed in the NHS (Figure 1). The assessment outlines an important increase in the maturity of performance measurement driven by the legislation highlighted above. In 1992, the public health companies highlighted a low level of all of the features described by Smith and Bititci (2017), while in 2023, they described a medium–high maturity level of performance measurement. The evaluation highlights an excellent balanced set of metrics, targets, and incentives linked to strategic objectives and regular and frequent performance reviews. The lower score is related to poor accessibility of measures due to the lack of integration of the various information technology systems.

5. Discussions

From the 1980s, new public management reforms and public governance required the performance measurement of public organizations based on criteria such as efficiency and efficacy (Hood, 1995; Modell, 2001; Osborne, 2010; Worrall et al., 2010). The new public governance shifted the focus from internal and economic perspectives toward the integration of customer and learning and growth standpoints. As suggested by the new public governance (Osborne, 2010), the NHS should improve managerial tools in such a way that they function within the contexts of uncertainty and scarcity of resources.

In line with the suggestions of international reforms, the findings of this research indicate that the performance measurement of the Italian NHS is moving from a system based exclusively on a financial perspective to a system based on financial and internal process perspectives related to the quality of LEAs. CSFs of the internal process perspective include aspects such as organizational appropriateness, clinical appropriateness, and patient safety. As required by the literature, the performance measurement identified may be recognized as a multi-perspective performance measurement system (Bianchi et al., 2017; Bititci et al., 2012; Kaplan and Norton, 1992; Sardi and Sorano, 2019). In this system, numerous CSFs and various managerial tools are considered, such as analytical accounting, budgeting, and risk management. The CSFs identified are efficiency, effectiveness, functionality, cost, quality, returns, cost-effectiveness, impartiality, output, employee performance, transparency, expenditure, patient safety, organizational appropriateness, clinical appropriateness, and geographic and social equity. The list of CSFs represents an integrated performance measurement on various aspects; however, according to the balanced scorecard design for private organizations, the customer and learning and growth perspectives are underdeveloped in comparison to a holistic and integrated system. For instance, the list of CSFs should integrate metrics such as user satisfaction and employee training.

This research outlines an important increase in the maturity of performance measurement over the past three decades. This evaluation highlights an excellent balanced set of metrics, targets, and incentives linked to strategic objectives and regular and frequent performance reviews. In line with Smith and Bititci’s (2017) research that established clear relationships among performance measurement, management practices, employee engagement, and performance, the change of performance measurement toward full maturity should affect performance management, employee engagement, and organizational performance. However, the low score of the accessibility measure may decelerate the regular and frequent performance reviews and the short interval control and, consequently, may decelerate the impact on performance management.

To summarize the results deriving from the three analyses—the definition of the context, the analysis of the legislation, and the assessment of the maturity of the performance measurement—the research highlighted a very complex NHS model based on three levels.

The first level is represented by the central government, particularly the Ministry of Health, which exercises the following functions: protection of human health, coordination of the national health system, veterinary health, protection of health in the workplace, and hygiene and food safety. In particular, it addresses general guidelines, coordination, and monitoring of regional technical health activities in agreement with the Ministry of Economy and Finance for all profiles relating to the State’s contribution to the financing of the NHS and to regional recovery plans. Furthermore, it monitors the quality of regional healthcare activities concerning the LEAs provided.

The second level is related to the regional health service. The regions plan and manage healthcare in full autonomy within their territorial jurisdiction, making use of healthcare companies. The regional health service organizes and coordinates a set of functions, resources, structures, and activities that contribute to responding to health needs, promote the state of well-being of the individual and community, and operate in an integrated and uniform manner across the regional territory. It operates across the regional territory through the health companies. Each region is responsible for health planning as well as the management control and quality assessment of health services by proposing a guideline for the development of management control in a health company.

The third level relates to health companies that are part of the NHS; they are companies with a public legal personality, equipped with organizational, managerial, technical, administrative, patrimonial, and accounting autonomy as well as centers of attribution of entrepreneurial autonomy. Companies must act on criteria of effectiveness, efficiency, and cost-effectiveness. The local health authorities are autonomous bodies led by a general director, a health director, and an administrative director; any complaints from citizens must be addressed to them. The general manager is responsible for the overall management of the individual company. In particular, through the establishment of a specific internal control service, the general manager is tasked with verifying, through comparative evaluations of costs, returns, and results, the correct and economic management of the resources allocated and received as well as the impartiality and good performance of administrative action. The board of auditors verifies the administration of the company from an economic point of view, supervises compliance with the law and compliance with the budget and accounting, and reports to the region at least quarterly. Additionally, based on the results of the checks conducted, the board of auditors immediately reports the facts if there is a well-founded suspicion of serious irregularities.

Moreover, regarding the internal controls of each public body and the audit controls, if the regional health service does not achieve its objectives, it must follow operational programs of reorganization, redevelopment, and strengthening of services. This process aims to rebalance the LEAs with the healthcare budget. If the periodic monitoring still produces negative results, the Council of Ministers initiates the procedure for placing the region under commissionership. Furthermore, each region identifies public health companies that have negative income statements and poor results in terms of volume, quality, and result of care.

According to this research, the performance measurement model of Italian NHS includes the financial and internal process perspectives that include the main CSFs identified during the study. The NHS should observe the economic–financial and equity balance and volume, quality, and outcomes of healthcare activities, as indicated by the new guarantee system (Decree of March, 12, 2019). In particular, within the new guarantee system, 22 indicators have been identified for evaluation of the provision of LEAs. These indicators are divided into three macro areas: collective prevention in public health, district assistance, and hospital care. The total of indicators, 24 measures, corresponds to the range suggested by Marr (2012).

As highlighted by the research, Italian law introduced the innovation required by international reforms; this innovation has been described as a process of modernization driven by law (Cepiku and Savignon, 2012; Iacovino et al., 2017; Ongaro and Valotti, 2008). The reorganization of the NHS has introduced elements of the private sector, such as performance measurement and management models (e.g. balanced scorecard and analytical accounting). However, the regulatory framework should also prioritize customer or stakeholder satisfaction to respond to the requirements of the new public governance and the European Union plan.

6. Conclusions

The findings of this research describe a significant increase in the maturity of performance measurement of an NHS that has been motivated by international reforms and national regulations over the past three decades. This research highlights performance measurement features such as a balanced set of metrics, targets, and incentives linked to strategic objectives and regular and frequent performance reviews. The results illustrate numerous CSFs that cover financial and internal business process perspectives, and the regulations of the past decade focus on aspects of patient safety, organizational appropriateness, and clinical appropriateness. In contrast, the customer and learning and growth perspectives are still poorly addressed by regulations, particularly regarding user satisfaction and employee training.

This article provides several theoretical and practical contributions. First, it outlines the evolutionary path of performance measurement and defines the main CSFs of an NHS driven by international and national reforms. The article also rationalizes the main regulations for the development of performance measurement of an NHS. Overall, it defines the performance measurement model of an NHS; it is based on two prospective financial and internal processes perspectives and includes numerous critical success factors that are translated into 24 indicators.

The theoretical implications provide scholars with an opportunity to analyze the performance measurement model adopted by an NHS for improvement or comparison to the other health companies of other NHSs. The practical implications offer practitioners and managers an opportunity to know and eventually apply performance measurement model adopted by a leading NHS. The social implications provide society with an opportunity to know the monitoring of an NHS.

Only one NHS is investigated in this study, which may indicate a research limitation. However, the explorative case-study method favors the deep knowledge of a single case related to a leading NHS, as indicated by Yin (2018). Thereby, this study offers suggestions for new practical and theoretical research.

Future theoretical research may investigate the degree of democratic and participative performance management and, consequently, the entire conceptual framework proposed by Smith and Bititci (2017) in other NHSs to test and validate the positive interplay between performance measurement, performance management, employee engagement, and organizational performance. Additionally, future theoretical research may investigate the factors that influence performance measurement practices such as management information systems, business models, leadership styles, and organizational culture. Future studies, thus, may understand how contingency factors could influence the evolutionary path of performance measurement and CSFs (Garengo and Bititci, 2007).

As for future practical research, it may investigate the implementation level of the features of performance management, according to national legislation. Furthermore, performance measurement needs of health companies may be investigated to define the performance measurement and management of public companies included in an NHS, thereby obtaining a model based on legislation and managerial needs, as suggested by Sorano et al. (2023).

Figures

Figure 1

The evolutionary maturity of performance measurement

Table 1

Main documents collected

Regulation*	Description
Art. 32 Constitution 1948	It recognizes health as a fundamental human right
Law 833/1978	It establishes the NHS and the financial management of health services by the State
Leg. Decree 502/1992	It reorganizes the health discipline, leading to corporatization, market orientation, and the distribution of responsibilities to the regions
Leg. Decree 286/1999	It strengthens the tools for monitoring and evaluating the costs, yields, and results of the activities of public administrations
DPCM 29/2001	It defines the levels of essential assistance
Art. 117 Constitution 2001	It reforms the competencies of the State and regions also in health matters
Law 311/2004	It defines the economic and financial balance
Leg. Decree 150/2009	It optimizes the productivity of public work and the transparency of public administrations
Leg. Decree 68/2011	It provides the autonomy of entry of the regions as well as the determination of costs and standard needs in the healthcare sector
Law 118/2011	It harmonizes the accounting systems and balance sheet layouts of the regions, local authorities, and their bodies
Leg. Decree 123/2011	It reforms administrative and accounting control and strengthens the analysis and evaluation of expenditures
Leg. Decree 33/2013	It organizes the discipline concerning the publicity, transparency, and dissemination of information by public administrations
DM 70/2015	It regulates qualitative, structural, technological, and quantitative standards relating to hospital care. It requires clinical risk management
Law 208/2015	It arranges the formation of annual and multi-year State budgets; it also revolutionizes health risk management
DM 21 June 2016	It determines a new assurance system for healthcare monitoring
DPCM 12/2017	It updates the levels of essential assistance relating to collective assistance and public health, district assistance, and hospital assistance
Law 24/2017	It governs the safety of care and the healthcare professional liability. It requires a risk management system
DM 12 March 2019	It determines a new assurance system for healthcare monitoring
DM 24 May 2019	It indicates the new economic models: income statement, balance sheet, cost report of levels of essential assistance, and facility report

Note(s): * Its subsequent modifications – ** DM: Ministerial Decree – *** DPCM: Decree of the president of the council of ministers

Source(s): Author’s own creation

Table 2

Pre-understanding the context

NHS	Description
Main information*	National health fund		€126bn
	Population		58.997.201
	Employees		About 620,000
	Health services provided by the NHS		About 5,700
	Accredited public and private facilities		About 27,000
	Healthcare institutions (1,051 units) under ordinary regime in the NHS	209,568 Beds	3.5 Beds Per 1,000 inhabitants
		5,461,902 Hospitalizations	92.4 Hospitalization rate
		46,527,824 Hospital days	8.5 Hospitalization day average
	Residential healthcare and rehabilitation facilities	265,776 Beds	44.8 Beds per 10,000 inhabitants
	Semi-residential healthcare and rehabilitation facilities	56,937 Beds	9.6 Beds per 10,000 inhabitants
Purpose	The Italian Republic safeguards health as a fundamental right of the individual and as a collective interest, and guarantees free medical care to the poor (Article 32, Italian Constitution)
Principles	The Republic protects health as a fundamental right of the individual and the interest of the community through service national healthcare
	The protection of physical and mental health must be done with respect of the dignity and freedom of the human person. The NHS is composed of a complex of intended functions, structures, services, and activities to promote, maintain, and recover the physical and psychic health of the entire population without distinction of individual or social conditions and in ways that ensure the equality of citizens in relation to the service. The implementation of the NHS is the responsibility of the State, the regions, and local territorial bodies, guaranteeing the participation of citizens. Connection is ensured in the NHS’ coordination with the activities and interventions of all other bodies, centers, institutions, and services that conduct social activities in the sector that affect the state of health of the individuals and the community. Voluntary associations can contribute to the purposes and institutions of the NHS in ways and forms established by this law. (Art. 1, Law 833/1978)
Objectives	The formation of modern health-consciousness based on adequate health education of citizens and communities
	The prevention of illnesses and accidents in every area of life and work
	The diagnosis and treatment of morbid events whatever their causes, phenomenology, and duration
	The rehabilitation of the somatic and psychic states of invalidity and incapacity
	The promotion and protection of the health and hygiene of the natural living and working environment
	The hygiene of food, drinks, products, and leftovers of animal origin for implications relating to the health of human beings as well as the prevention and health defense of animal farms and the control of their feeding (supplemented and medicated)
	A discipline of experimentation, production, and introduction in the marketing and distribution of medicines and information research on them aimed at ensuring the effectiveness, non-harmfulness, and cost-effectiveness of the product; [and]
	Professional and permanent training as well as scientific and cultural updating of service personnel national health
	The Italian NHS within the scope of its competencies pursues …. (Art. 2, Law 833/1978)
Management control	Each region is responsible for health planning as well as the management control and quality assessment of health services; it proposes a guideline for the development of management control in a healthcare company. According to Legislative Decree 502/1992, health companies adopt management control to monitor the effectiveness and efficiency of healthcare activities and to guarantee the quality and management of financial resources. Each health company must define the main aspects useful for the design, implementation, and use of management control, such as: the unit or units responsible for the design and management of performance control;
	the organizational units that measure the effectiveness, efficiency, and economy
	the procedures for determining the management objectives and the responsible subjects
	the set of products and purposes of the administrative action concerning the administration or organizational units
	the methods for recording and allocating costs among the organizational units and for identifying the objectives for which the costs are incurred
	the specific indicators for measuring effectiveness, efficiency, and economy; and the frequency of the collection of information (Article 1 Legislative Decree 286/1999).

Note(s): * Italian National Institute of Statistics (2023)

Source(s): Author’s own creation

Table 3

Impact of the regulations on the performance measurement and CSFs

Regulation*	Impact on performance measurement	Critical success factor
Leg. Decree 502/1992	Determine NHS principles concerning management control and quality assessment of health services, determined by the regions Establish the evaluation criteria of the general managers’ objectives concerning NHS efficiency, effectiveness, and functionality Implement economic–financial accounting and analytical accounting for cost centers and responsibilities for the comparative analysis of costs, yields, and results Aim for the criteria of effectiveness, efficiency, cost-effectiveness, and compliance with budget constraints through the balance of costs and revenues Verify and guarantee healthcare quality by checking the cost, quality, and quantity of services through functional organizational models and information flows	Efficiency – Effectiveness – Functionality – Cost – Service Quality
Leg. Decree 29/1993	Control the assigned objectives according to criteria such as efficiency, impartiality, and performance, and the party or subject responsible for any failure should be identified	Efficiency – Costs – Returns – Cost Effectiveness – Impartiality
Leg. Decree 286/1999	Define the procedures for detecting and allocating costs among the organizational units, pinpointing the costs relating to the objectives	Quality – Efficiency – Effectiveness – Cost Effectiveness
DPCM 29/2001	Define the activities, services, and benefits guaranteed to citizens with the public resources made available by the NHS	Output
Law 311/2004	Guarantee the economic-financial balance of the regional health service Health companies must adopt analytical accounting by cost and responsibility centers	Cost Effectiveness
Leg. Decree 150/2009	Measure and evaluate the performance to improve the quality of services in public health facilities	Quality – Employee Performance
Leg. Decree 68/2011	Verify the provision of the LEAs in conditions of efficiency and appropriateness	Efficiency – Appropriateness
Leg. Decree 123/2011	Control the administrative and accounting acts that have a financial impact on the budgets of the State and other public organizations	Expenditure
Law 190/2012	Regulate the prevention and suppression of corruption and illegality in public administration	Legality
Leg. Decree 33/2013	Determine the publication of key corporate information	Transparency
DM 70/2015	Indicate hospitalization volumes and outcomes, quality standards, and organizational, technological, and structural standards	Qualitative Standards – Quantitative Standards
Law 208/2015	Verify results relating to the volume, quality, and outcomes of care, preserving the logic of the economic equilibrium	Patient Safety
DM** June 21, 2016	Provide a national comparative assessment of health service companies in terms of efficacy, safety, efficiency, and quality of care	Efficacy – Safety – Efficiency – Quality
DPCM*** 12/2017	Define the activities, services, and benefits guaranteed to citizens with the public resources made available by the NHS	Output
Law 24/2017	Rule the safety of care and the healthcare professional liability. This task requires a risk management system	Patient Safety – Healthcare Risk
DM March 12, 2019	Identify an indicator system that associates each level of essential assistance with the relevant attributes of the service processes	Efficiency – Efficacy – Organizational and Clinical Appropriateness Patient Safety – Quality – Geographic and Social Equity

Note(s): * Its subsequent modifications – ** DM: Ministerial Decree – *** DPCM: Decree of the president of the council of ministers

Source(s): Author’s own creation