Abstract
Purpose
In this article, we outline our views on the appropriateness and utility of Return on Investment (ROI) for the evaluation of the value of healthcare quality improvement (QI) programmes.
Design/methodology/approach
Our recent research explored the ROI concept and became the genesis of our viewpoint. We reflect on our findings from an extensive research project on the concept of ROI, involving a multidisciplinary global systematic literature review, a qualitative and Delphi study with mental healthcare leaders from the United Kingdom National Health Service. Research participants included board members, clinical directors and QI leaders. Our findings led to our conclusions and interpretation of ROI against the broad QI governance. We discuss our views against the predominant governance frameworks and wider literature.
Findings
ROI is in-line with top-down control governance frameworks based in politics and economics. However, there is evidence that to be of better utility, a tool for the assessment of the value of QI benefits must include comprehensive benefits that reflect broad monetary and non-monetary benefits. This is in-line with bottom-up and collaborative governance approaches. ROI has several challenges that may limit it as a QI governance tool. This is supported by wider literature on ROI, QI as well as modern governance theories and models. As such, we question whether ROI is the appropriate tool for QI governance. A more pragmatic governance framework that accommodates various healthcare objectives is advised.
Practical implications
This article highlights some of the challenges in adopting ROI as a QI governance tool. We signal a need for the exploration of a suitable QI governance approach. Particularly, are healthcare leaders to be perceived as “agents”, “stewards” or both. The evidence from our research and wider literature indicates that both are crucial. Better QI governance through an appropriate value assessment tool could improve clarity on QI value, and thus investment allocation decision-making. Constructive discussion about the utility and appropriateness of ROI in the evaluation of healthcare QI programmes may help safeguard investment in effective and efficient health systems.
Originality/value
The article raises awareness of QI governance and encourages discussions about the challenges of using ROI as a tool for healthcare QI governance.
Keywords
Citation
Thusini, S., Soukup, T. and Henderson, C. (2024), "Is return on investment the appropriate tool for healthcare quality improvement governance?", International Journal of Health Governance, Vol. 29 No. 3, pp. 296-308. https://doi.org/10.1108/IJHG-06-2024-0067
Publisher
:Emerald Publishing Limited
Copyright © 2024, Emerald Publishing Limited
Introduction
Healthcare funders worldwide spend millions on the provision of safe and quality care through quality improvement (QI) programmes. There are many ways to improve the quality of healthcare services. QI is a specific methodology used to investigate and identify problems, as well as find solutions to localised problems (small projects) or systemic problems (large-scale QI). Large-scale QI uses systematic methods to improve structures, processes and outcomes of services (Benn et al., 2009). Regardless of the type of improvement approach taken, good governance is a crucial corner-stone that supports responsible use of public funds. “Good governance” in the quality and safety of health services includes prudent investment of funds to deliver optimal health outcomes efficiently and effectively (The National Leadership Council (NLC), 2012). This is done through a number of tools, e.g. performance monitoring.
Recently, there has been an interest in the act of governing for QI (Lorelei et al., 2017) or QI governance (Chua et al., 2023). As part of this, Return on Investment (ROI) may be adopted as a governance tool. ROI originates from accounting as a metric for predicting the profitability of financial investments. Later, ROI was adopted in economic analysis practice where it is used to predict or evaluate value for money by assessing financial benefits against programme implementation costs (The National Institute for Health and Care Excellence (NICE), 2011). This indicated a shift from evaluating ex-post to predictive investment. That is, programmes that do not promise financial benefits may not be funded to begin with (Leatherman et al., 2003).
However, predicting or evaluating QI value in a complex health system can be challenging. On an upcoming publication, we highlight the inherent complexity, uncertainty and ambiguity of healthcare institutions. Such factors make healthcare QI governance complex, requiring robust tools to help manage complexity. For effective implementation of such tools, ensuring that what “we think” aligns with what “we do” is critical. Therefore, to support effective application of ROI, a study of its meaning within the QI context was essential.
To that effect, we have recently completed an extensive study of the concept of “return-on-investment” where we explored perceptions about ROI from large-scale QI programmes (Thusini et al., 2022, 2023). Specifically, we explored the metaphorical use of ROI as opposed to ROI as a metric, hence our reference to QI benefits as “QI-ROI” or “return(s)-on-investment”. Our research findings led us to conclusions that became the genesis of this viewpoint. Below, we outline the approach we took in formulating our view.
Approach
This viewpoint is shaped by (1) our research, (2) wider literature on ROI, QI and governance. Our research took a sequential confirmatory-exploratory-explanatory mixed-methods design where each study served to confirm previous knowledge, as well as explore and explain new insights through a process called “building” (Fetters et al., 2013). We commenced with an integrative systematic literature review (Thusini et al., 2022). Here, we explored views on QI benefits in both public and private healthcare globally. We used this data to create an initial conceptual framework for ROI from QI. We then performed a qualitative study (Thusini et al., 2023), where we interviewed UK mental healthcare leaders to refine our ROI framework. Lastly, we performed a consensus study using the Delphi method where we explored benefit legitimacy and priority to further refine our conceptual framework (paper forthcoming).
The systematic review covered different disciplines and levels (primary to tertiary care) of both public and private healthcare globally. The qualitative and Delphi studies were performed with leaders of the publicly-funded United Kingdom (UK) National Health Service (NHS) mental healthcare. Participants included largely board members and QI leaders. Mental healthcare was the source of the question about the concept of ROI in the QI context, and hence our main research setting. However, we contend that our findings apply to similar organisations where acute and long-term patients are cared for from community to hospital care.
Notably, the private sector may wish to cater to special utilities beyond those sought in the public sector. Largely, we found that modern healthcare in both sectors invariably seek similar ideals, e.g. patient satisfaction and raising revenue. The goals and priorities may depend on certain factors, e.g. an organisation’s developmental stage. Thus, organisations may prioritise different benefits at specific times due to specific strategic needs. Additionally, our views are more specific to governance of large-scale QI programmes. Further, although QI represents a particular discourse in the improvement practice, it is yet unclear whether governing for QI is any different to that of other approaches. Some approaches may be more amenable to the traditional ROI methodology. Nonetheless, we postulate that our research and views apply to any programme with similar goals, strategies, size and investments as large QI. We defined “large-scale” as involving at least two organisational departments (Thusini et al., 2022).
Below, we bring together the components that reflect the findings that shaped our viewpoint (our research and wider literature). In effect, we integrate the findings produced as we explored the ROI concept through our series evaluations, with the “findings” from our reflection on our conclusions and views against wider literature. We then provide an outline of healthcare governance including principles, purposes and theories. Next, we synthesise these sources, discuss our views and their implications. Lastly recommend a way forward.
Findings
Our series evaluations indicated that both monetary and non-monetary benefits form significant components of the conceptualisation of ROI from QI. In the systematic review, we deduced four main themes of benefits: organisational performance (patient and financial benefits), organisational development (e.g. culture), external benefits (e.g. socio-economic benefits), unintended benefits (e.g. revealing concealed issues and lessons). In the qualitative study, leaders indicated that any benefit within these themes that contributes to an organisation’s strategic vision is valued and viewed as ROI. In the Delphi, leaders indicated that patient benefits, organisational development and external benefits were more in line with their ROI concept than financial benefits due to the nature of healthcare functions and values.
We then evaluated our research findings against broader literature from QI, ROI and healthcare governance. During our research, we found few literature works addressing ROI from QI. Nonetheless, our findings support views of other QI authors (e.g. Swensen et al., 2013; Solid, 2020). Solid (2020) stated that in QI, some benefits are tangible, positive ROIs have been reported and guidance on how to monetise benefits exists. With “high value ticket” or “low hanging fruit” programmes, benefits are more obvious (Solid, 2020; Willson, 2015), such as in care overuse or misuse (Swensen et al., 2013). However, some benefits are intangible. That is, they may be impossible or challenging to measure, monetise and attribute.
Where intangible benefits are more valued, e.g. improving service experience, traditional ROI may be counter-productive. Solid (2020, pp. 59–60) explained this phenomenon; programmes with lower rates of return often require the most skill to appropriately frame associated benefits. As such, interventions that produce the most benefit may not produce the highest ROI (Solid, 2020). Thus, focussing on measuring value (through ROI) may reduce the ability to meet stated quality goals “because ensuring that value can be reliably measured and quantified can make it difficult to measure quality, and vice versa” (Solid, 2020).
Based on the various benefits conceptualised as QI-ROI in our research, we agree that the above is a matter of benefit attribution rather than QI contributions. Further, a perceived lack of QI-ROI may reflect an accepted definition of returns rather than the actual absence of value. If QI-ROI is accepted to encompass all valued benefits as found in our research, and intangible benefits are sometimes more important (Solid, 2020), it would be paradoxical to state that “interventions that produce the most benefit may not produce the highest ROI” (Solid, 2020, p. 59). Defining QI-ROI this way risks negative implications for QI investment and governance.
Our research also supports ongoing concerns about ROI in the evaluation of programmes in both for profit and no-profit health systems (Fischer and Duncan, 2020; Masters et al., 2017; van der Goes et al., 2019). In their iron triangle ROI model van der Goes et al. (2019) noted that traditional ROI neglects valued benefits like access to care. Similarly, in other service-based industries like healthcare, the ROI appropriateness and utility has long been questioned (De Meuse et al., 2009; Dearden, 1969; Ozminkowski et al., 2016). Here, the main concerns are also ROI’s ability to capture the true value of programmes. This realisation of ongoing concerns over ROI led us to reflect on our findings against the premises of healthcare governance.
Healthcare governance overview
In 2012, the UK National Leadership Council (NLC) introduced governance principles in their document; “The Healthy NHS Board: principles for good governance”. They encouraged boards across the system to follow their guidance in tackling the challenges of improving quality (NLC, 2012). This incorporated (1) Quality assurance and clinical governance (2) financial stewardship, (3) risk management, (4) legality, (5) decision-making, (6) probity (or integrity) and (7) corporate trustee. This list hints to the roots of healthcare governance in corporate governance. Today, a number of influential corporate and civil organisations provide some definitions and descriptions that sign-post governance rules, functions, means or ends.
Largely, governance is seen a framework used to direct, control and manage an organisation and its employees (Conmy, 2024). Its frameworks instruct on who has authority and accountability for effective running of an organisation. Governance provides a structure for creating an organisation’s objectives, as well as how they can be achieved and monitored (Organisation for Economic Co-operation and Development (OECD), 2015). This then support clear decision-making, strategic planning, risk management and accountability through performance management (Conmy, 2024). To this extent, governance functions largely include conformance (external accountability), performance (vision, mission, strategy) and fiduciary obligations (monitoring and compliance) (Chambers et al., 2012).
Today, board members are also expected to foster positive cultures for themselves and their organisations (NLC, 2012). “Good” governance must support boards to be good employers by encouraging high standards, transparency and engagement (The Chartered Governance Institute UK & Ireland (CGI), 2024). This may enable management teams and boards to run organisations successfully, legally, ethically and sustainably (The Chartered Governance Institute UK & Ireland (CGI), 2024). These modern views on governance call for more inclusive bottom-up approaches that accommodate varied goals, values, purpose and assumptions. Inevitably, theories sought to explain these approaches.
Governance theories
The concept of governance stems from politics, then later economics. In this context, top-down regulation, oversight and incentives are used to encourage leaders to comply with governance mechanisms. This reflects the assumptions by the Principal-Agent (P-A) Theory (Jensen and Meckling, 2019). The P-A assumes that unless controlled, leaders (agents) are at risk of promoting their own goals at the expense of policy-makers (principals). In effect, many P-A relationships exist across different levels of healthcare funding and provision. For example, in the context of QI, the board are the principals and QI leaders, the agents. The P-A Theory is the most visible part of governance as associated frameworks and models enforce conformity and compliance for organisations to attain and maintain economic and political legitimacy.
Although board governance may be activated and enforced by politicians and policy-makers, it cannot be effectively enacted without the co-operation and participation of leaders. This understanding followed the engagement of social scientists in governance matters. Leaders must consider the fundamental health system goals such as improved health, responsiveness and fair financing (The World Health Organisation (WHO), 2000). As such, they make decisions based on priorities within their organisations and systems. This transformed understanding allows governance to be viewed through theories like stewardship, resource dependency, stakeholder and institutional theories.
The Stakeholder Theory asserts that organisations are made of various stakeholders that impact and are impacted by an organisation (Laplume et al., 2008). Systems theory adds that organisations are part of a system within which they must gain social legitimacy (Friedman and Allen, 2011). Complexity Theory highlights that these and other factors make organisations complex with unpredictable processes and outcomes (Braithwaite et al., 2018). This complexity involves humans, but also technology (Actor-Network Theory) (Cresswell et al., 2010) and requires organisations to be flexible and adaptable (Gell-Mannn, 1994). In view of this, intrinsically driven leaders may wish to exercise take ownership, autonomy and employ discretion in their strategic behaviour (Stewardship Theory) (Donaldson and Davis, 1991).
Organisations form institutions. Institutions are the norms and routines that guide accepted social behaviour in a context (Scott, 2005). This guides institutional rules of engagement with any stakeholder or reform. Institutional Theory has been evolving over time to engage more with governance matters, as seen with the eminence of “new governance” and “institutional economics” (Abimbola et al., 2017). Thus, Institutional Theory is significant for QI governance. For example, according to the Resource Dependency Theory, institutions must perform and abide by external rules to attain or retain QI resources (Pfeffer and Salancik, 2015). However, performance monitoring is enacted within internal rules and norms that dictate internal logic. These rules and norms can also be limiting and myopic (DiMaggio and Powell, 1993).
Institutions make “institutionally rational” choices in search of value, but these are bound or contingent to what is deemed rational or logical in context. Bounded Rationality and Contingency theories (Tarter and Hoy, 1998) acknowledge assumptions by systems, stakeholder, stewardship, complexity theories. As such, they support institutional rationality. Alternatively, Resource Dependency and P-A theories explain political and economic rationality. Unlike bounded rationality, this view on rationality can be rigid. Also related to ROI, is Rational Choice Theory where cost-benefit analysis guides decision-making (Ostrom, 1991). This assumes that individuals have all information required to make decisions, which may not be the case.
One perspective is deficient. In 2012, the UK National Leadership Council (NLC) asserted that governance models are not mutually exclusive. The NLC identified common models as the agency, stakeholder and stewardship models. In addition, was the policy and generative models. The policy model is similar to the agency model in that chief executives are seen as the “operators” who act to satisfy the “owner representatives” objectives. In this instance, policymakers act as the representatives of the public, the “owners”. Here, policies limit management freedoms to ensure prudence. Alternatively, the generative model emphasises broader bottom-up knowledge production through data and stakeholder engagement.
The need to govern with a broader view later encouraged governance approaches like “collaborative governance” (Ansell and Gash, 2008), “participatory governance” (Chhotray et al., 2009), “network governance” (Davies, 2011) and “interactive governance” (Torfing, 2012). Recently, the idea of governing with others has led to a shift away from competition to a more collaborative paradigm. For example, the integrated care sought in the UK health system (NHS, 2021), and the US originated value-based care principles (Teisberg et al., 2020). Value-based care may seek value at the unit or individual level, but integration highlights that value for patients or organisations may be best produced and governed at a system level. Given the overlapping thinking on governance, some scholars advocate for coherence through theory.
Chambers et al. (2012) argued for a unified theory. Unifying theories is contentious as some nuanced explanations may be lost (Czarniawska, 2008). However, broader streamlined understandings may be produced to guide core governance principles. Alternatively, Bevir and Rhodes (2001) argued for a “de-centred governance theory” that acknowledges agency, participation, interpretation and democracy. In this theory, external factors are seen to influence governance only through the ways in which they are understood by internal actors (Bevir and Rhodes, 2001). This also indicated a paradigm shift to one supportive of healthcare organisations as they seek to balance their multiple governance responsibilities.
Synthesis
Altogether, these principle, purposes and theories of healthcare governance acknowledge that healthcare is influenced by a number of factors including values, needs and ideologies of multiple stakeholders. This supports Chambers et al. (2012) argument for a unified theory and (Bevir and Rhodes, 2001) idea of “de-centred” theory. The above also indicate that QI governance must be more pragmatic and less transactional. To this extent, our research indicated that healthcare leaders in both public and private healthcare prioritise stewardship for population health. Today, this includes accounting for efficient allocation and use of resources. In our research, this was demonstrated by the justification of economic discourses, albeit apprehension over monetisation (Thusini et al., 2022, 2023).
The WHO has long promoted stewardship governance (The World Health Organisation (WHO), 2000). They see a health system as responsible for several stewardship functions such as strategy and policy development, effective system design, collaboration, governance, regulation and intelligence (Veillard et al., 2011; The World Health Organisation (WHO), 2000). This is based on ideals such as truth, justice, fairness and democracy (Veillard et al., 2011). Similarly, the United Nations (UN) saw governance characteristics as including participation, consensus, transparency and equity (Banerji, 2015). In their research, Lorelei et al. (2017) also identified similar attributes of board governance, e.g. stakeholder engagement. These attributes are indicative of modern healthcare institutions.
Stewardship demonstrates servant, ethical, transformational and collaborative leaderships (Tatum et al., 2003). However, stewardship with its democratic and discretionary properties raises concerns of laissez-faire leadership (Tatum et al., 2003). But according to Chan et al. (2019), the stewardship view of governance supports strategy formulation for quality, sets a clear vision, activities and responsibilities for execution. To this effect, our research indicated that healthcare leaders envision QI-ROI as broad strategic value. Further, our research indicated that although in disagreement with the ROI philosophy, leaders tend to be pragmatic and compromise or comply with its use (Thusini et al., 2023). Thus, we believe redefining value in-line with public value Bozeman and Su (2015) is not synonymous with laissez-faire leadership. Instead, it recognises the many goals that leaders must balance. As illustrated in Figure 1, we see QI governance as a pragmatic exercise to support various ideals in service of public value.
Discussion
In this article, we sought to argue our viewpoint that questions the appropriateness and utility of ROI as a QI governance tool. Viewed against our research and wider literature, the ROI assumptions based on transactional principals and agents are problematic in the QI context. Thus, we argue that the use of ROI as a primary tool for QI and overall QI governance needs a review. Although experts have attempted to alley anxieties regarding ROI (Brousselle et al., 2016; Phillips and Phillips, 2008), some continue to raise concerns (Fischer and Duncan, 2020; Krlev et al., 2013; Leggat, 2007; Ozminkowski et al., 2016). Nonetheless, evidence has shown that although in disagreement with the ROI philosophy, leaders tend to be pragmatic and compromise or comply with its use (Thusini et al., 2023). Complying may mean symbolic adoption (Botchkarev and Andru, 2011; Brousselle et al., 2016).
Currently, there appears to be no legitimate room for intangible benefits within the ROI methodology. Some argue that ROI should not attempt to accommodate all value (Botchkarev and Andru, 2011; Solid, 2020). In Botchkarev and Andru (2011), predicted that intangible benefits are unlikely to become official components of organisations’ financial statements in the future. That appears to be the case today. Thus, if ROI becomes the most legitimised way to account for QI value, it can create challenges. Due to limitations, ROI is not seen as a stand-alone tool and is often supplemented with other data (The National Institute for Health and Care Excellence (NICE), 2011). Health economists such as Appleby (2005) do advocate for comprehensive data to truthfully assess the value of QI. In service of this, compromises in scientific rigour may also be necessary.
Compromise indicates the need for unified theories and pragmatism. For some, this has meant creating ROI imitations or variations that recognise broader benefits, e.g. (Phillips and Phillips, 2008). The existing ROI imitations ultimately seek monetisation, a practice proven to be problematic (Krlev et al., 2013) and prone to technical errors (Botchkarev and Andru, 2011). Alternatively, the multi-decision-criteria-analysis (MCDA) and cost-consequence analysis are recommended to accommodate non-monetary benefits (The National Institute for Health and Care Excellence (NICE), 2011). However, these are rarely used. MCDA mimics a practice used in commercial industries to weight different value on the same scale. This way, intangible benefits can be recognised for their perceived value. Intangible benefits are increasingly seen as crucial to bottom lines (Ozminkowski et al., 2016).
QI programmes can produce numerous benefits for various stakeholders across organisations and systems. Some benefits are through capability and capacity building, as well as lessons learnt. Therefore, financial benefits may not be immediate thus challenging to attribute to improvement efforts. ROI traditionally minimises broader benefits for health systems and organisations, thereby limiting perceptions of QI value amongst upstream investors (e.g. politicians). In politics, economics and even management, quantified evidence is often key. In this respect, we see a need for a more suitable QI governance approach that highlights QI contributions to healthcare. This includes championing qualitative evidence of value alongside quantitative measures. Thus, a more pragmatic approach may support effective QI governance that considers fundamental health system goals as defined by The World Health Organisation (WHO) (2000).
In healthcare, ROI has been described as a tool for assessing value-for-money alongside cost-benefit analysis (CBA) and cost-effectiveness analysis (CEA) (The National Institute for Health and Care Excellence (NICE), 2011). Debates about value and some of these methods are centuries old. The debates largely relate to philosophers and theorists viewing value as objective or subjective, individual or collective, instrumental or symbolic etc. Crucially value was not always seen as a single monetised ratio, and things can be of value in and of themselves (Grassl, 2017). As such, Chambers et al. (2012) argued that governance cannot be measured solely through financial metrics like ROI. Thus, ROI criticism may have brought the debates about the utility of economic metrics in healthcare full-circle.
ROI descends from CBA; ROI focusses on monetary benefits from an investor’s perspective while CBA takes a societal view. Scholars have expanded ROI to include societal perspectives in a version called Social ROI (Krlev et al., 2013). Decades ago, CBA criticisms led to CEA where non-monetary benefits are presented in natural units like life years gained. Ultimately, economic evaluations are monetary focussed, a legitimate but limited governance function. The arguments are of-course more complex than our brief summary (Bridges, 2005; Drummond et al., 2015). It is worth noting that the utility of ROI as a QI governance tool is also complicated by the differing views on the concept of quality itself.
Historically, patient safety and clinical outcomes were the leading motivators for defining and seeking healthcare quality. Later, the focus shifted to cost-saving as healthcare grappled with rising costs. Quality attributes often include efficiency, effectiveness, timely care access and positive patient experience (Donabedian, 1988; Lohr, 1990). Managers may focus on efficiency and effectiveness, practitioners may define quality in terms of evidence-based care, whilst a service user may focus on timely access to care, a positive experience and receiving a desired health outcome. As such, hundreds of quality definitions exist, some catalogued to reflect different perspectives, e.g. Structures-Processes-Outcomes framework (Donabedian, 1988). This framework indicates that healthcare quality is a systems attribute that may require a systematic solution like QI. As such, improvement benefits may be broad and comprehensive.
Challenges in translating commercial ideas into healthcare are not unique. Like any tool, the ROI methodology has technical and philosophical challenges that need attention. In the knowledge that not all QI is or can be cost-saving, and saving costs is not the only reason to implement QI (Solid, 2020; Stusser and Dickey, 2013; van der Goes et al., 2019), an alternative way of looking at QI value is needed. Whether or not QI will be cost-saving may be predetermined by the responsiveness of the perceived quality issue, e.g. reducing over-use (cost-saving), under-use (cost-consuming), misuse (could be either). Nonetheless, value and quality are closely related, good quality is of value and value is seen as ROI. In our view, the broad view on ROI from QI seems absolutely aligned with the modern versions of governance.
Implications
This article highlights some of the challenges in adopting ROI as a QI governance tool. We signal a need for the exploration of a suitable QI governance approach. Particularly, are healthcare leaders to perceived as “agents”, “stewards” or both. The evidence from our research and wider literature indicates that both are crucial. Better QI governance through an appropriate value assessment tool could improve clarity on QI value, and thus investment allocation decision-making. Constructive discussion about the utility and appropriateness of ROI in the evaluation of healthcare QI programmes may help safeguard investment in effective and efficient health systems.
Conclusion
Ultimately, what is rational must be decided by leaders within their context rather than through a transactional “principal-agent” framework. A collective effort by funders, providers and service-users to find innovative ways to account for QI value is required. As the ROI discourse continues to be promoted, it is crucial that an understanding of the underlying assumptions must also continue to be explored. Caution must be exercised in taking rigid policy positions on QI and overall QI governance to minimise negative unintended consequences resulting from misaligned values. Inappropriate adoption and use of ROI may hinder conversations and innovation, thereby be counterproductive.
Potential future directions
Granted, intangible benefits can be unstable and can differ within and between individuals, rendering fund allocation decision-making unstable (Eabrasu, 2011). Nonetheless, if ROI misrepresents QI value, it may not be the appropriate tool for its assessment given the many benefits valued in QI (Thusini et al., 2023). Thus, perhaps the crucial next step is the determination of the role that should be played by ROI in the assessment of QI value. Further study should consider engagement of relevant stakeholders, e.g. commissioners as fund-holders (in UK) and public as payers and recipients of care and staff who are at the frontline.
A comprehensive study may offer more robust clarity on the utility of traditional ROI. This may then give way to constructive conversations about the exploration of alternatives to ROI. This may also provide more clarity about the role of healthcare leaders in QI governance. Unless effectively challenged, the improper use of ROI as the primary method to assess QI value is set to continue. ROI use in the midst of misaligned values, needs or abilities, risks negative outcomes (Oliver, 1991) that would be detrimental to health system improvement.
Figures
Patient and public involvement
Patients and the public were not required for this work.
Ethics and dissemination: Ethical approval was not required for this work.
References
Abimbola, S., Negin, J., Martiniuk, A.L. and Jan, S. (2017), “Institutional analysis of health system governance”, Health Policy and Planning, Vol. 32 No. 9, pp. 1337-1344, doi: 10.1093/heapol/czx083.
Ansell, C. and Gash, A. (2008), “Collaborative governance in theory and practice”, Journal of Public Administration Research and Theory, Vol. 18 No. 4, pp. 543-571, doi: 10.1093/jopart/mum032.
Appleby, J. (2005), “The quest for quality in the NHS: still searching?”, BMJ (Clinical Research ed.), Vol. 331 No. 7508, pp. 63-64, doi: 10.1136/bmj.331.7508.63.
Banerji, A. (2015), “Global and national leadership in good governance”, available at: https://www.un.org/en/chronicle/article/global-and-national-leadership-good-governance (accessed 12 June 2024).
Benn, J., Burnett, S., Par, A., Pinto, A., Isk, S. and Vincent, C. (2009), “Studying large-scale programmes to improve patient safety in whole care systems: challenges for research”, Social Science Medicine, Vol. 69 No. 12, pp. 1767-1776, doi: 10.1016/j.socscimed.2009.09.051.
Bevir, M. and Rhodes, R.A. (2001), “A decentered theory of governance: rational choice, institutionalism, and interpretation”, Journal des Economistes et des Etudes Humaines, Vol. 12.
Botchkarev, A. and Andru, P. (2011), “A return on investment as a metric for evaluating information systems: taxonomy and application”, Interdisciplinary Journal of Information, Knowledge, and Management, Vol. 6, pp. 245-269, doi: 10.28945/1535.
Bozeman, B. and Su, X. (2015), “Public service motivation concepts and theory: a critique”, Public Administration Review, Vol. 75 No. 5, pp. 700-710, doi: 10.1111/puar.12248.
Braithwaite, J., Churruca, K., Long, J.C., Ellis, L.A. and Herkes, J. (2018), “When complexity science meets implementation science: a theoretical and empirical analysis of systems change”, BMC Medicine, Vol. 16 No. 1, pp. 1-14, doi: 10.1186/s12916-018-1057-z.
Bridges, J.F.P. (2005), “Future challenges for the economic evaluation of healthcare”, PharmacoEconomics, Vol. 23 No. 4, pp. 317-321, doi: 10.2165/00019053-200523040-00002.
Brousselle, A., Benmarhnia, T. and Benhadj, L. (2016), “What are the benefits and risks of using return on investment to defend public health programs?”, Preventive Medicine Reports, Vol. 3, pp. 135-138, doi: 10.1016/j.pmedr.2015.11.015.
Chambers, N., Benson, L., Boyd, A. and Girling, J. (2012), “Assessing governance theory and practice in health-care organizations: a survey of UK hospices”, Health Services Management Research, Vol. 25 No. 2, pp. 87-96, doi: 10.1258/hsmr.2012.012015.
Chan, B.T., Veillard, J.H., Cowling, K., Klazinga, N.S., Brown, A.D. and Leatherman, S. (2019), “Stewardship of quality of care in health systems: core functions, common pitfalls, and potential solutions”, Public Administration and Development, Vol. 39 No. 1, pp. 34-46, doi: 10.1002/pad.1835.
Chhotray, V., Stoker, G., Chhotray, V. and Stoker, G. (2009), “Participatory governance”, Governance Theory and Practice: A Cross-Disciplinary Approach, pp. 165-190, doi: 10.1057/9780230583344_8.
Chua, K.-C., Henderson, C., Grey, B., Holland, M. and Sevdalis, N. (2023), “Evaluating quality improvement at scale: a pilot study on routine reporting for executive board governance in a UK National Health Service organisation”, Evaluation and Program Planning, Vol. 97, 102222, doi: 10.1016/j.evalprogplan.2022.102222.
Conmy, S. (2024), “What is corporate governance?”, The Corporate Governance Institute, available at: https://www.thecorporategovernanceinstitute.com/insights/lexicon/what-is-corporate-governance/ (accessed 12 June 2024).
Cresswell, K.M., Worth, A. and Sheikh, A. (2010), “Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare”, Bmc Medical Informatics and Decision Making, Vol. 10 No. 1, pp. 1-11, doi: 10.1186/1472-6947-10-67.
Czarniawska, B. (2008), “How to misuse institutions and get away with it: some reflections on institutional theory (ies)”, in The Sage Handbook of Organizational Institutionalism, pp. 769-782.
Davies, J.S. (2011), Challenging Governance Theory: from Networks to Hegemony, Policy Press, New York.
De Meuse, K.P., Dai, G. and Lee, R.J. (2009), “Evaluating the effectiveness of executive coaching: beyond ROI?”, Coaching: An International Journal of Theory, Research and Practice, Vol. 2 No. 2, pp. 117-134, doi: 10.1080/17521880902882413.
Dearden, J. (1969), “Case against roi control”, The Havard Business Review, US, available at: https://hbr.org/1969/05/the-case-against-roi-control (accessed 12 June 2024).
DiMaggio, P.J. and Powell, W.W. (1983), “The iron cage revisited: institutional isomorphism and collective rationality in organizational fields”, American Sociological Review, Vol. 48 No. 2, pp. 147-160, doi: 10.2307/2095101.
Donabedian, A. (1988), “The quality of care. How can it be assessed?”, JAMA, Vol. 260 No. 12, pp. 1743-1748, doi: 10.1001/jama.260.12.1743.
Donaldson, L. and Davis, J.H. (1991), “Stewardship theory or agency theory: CEO governance and shareholder returns”, Australian Journal of Management, Vol. 16 No. 1, pp. 49-64, doi: 10.1177/031289629101600103.
Drummond, M.F., Sculpher, M.J., Claxton, K., Stoddart, G.L. and Torrance, G.W. (2015), Methods for the Economic Evaluation of Health Care Programmes, Oxford University Press, Oxford.
Eabrasu, M. (2011), “A praxeological assessment of subjective value”, Quarterly Journal of Austrian Economics, Vol. 14 No. 2, pp. 216-241.
Fetters, M.D., Curry, L.A. and Creswell, J.W. (2013), “Achieving integration in mixed methods designs—principles and practice”, Health Services Research, Vol. 48 No. 6pt2, pp. 2134-2156.
Fischer, H.R. and Duncan, S.D. (2020), “The business case for quality improvement”, Journal of Perinatology, Vol. 40 No. 6, pp. 972-979, doi: 10.1038/s41372-020-0660-y.
Friedman, B.D. and Allen, K.N. (2011), “Systems theory”, Theory and Practice in Clinical Social Work, Vol. 2 No. 3, pp. 3-20, doi: 10.4135/9781483398266.n1.
Gell-Mann, M. (1994), “Complex adaptive systems”, Santa Fe Institute Studies In The Sciences Of Complexity-Proceedings Volume-19, Addison-Wesley, p. 17.
Grassl, W. (2017), “Toward a unified theory of value: from Austrian economics to Austrian philosophy”, Axiomathes, Vol. 27 No. 5, pp. 531-559, doi: 10.1007/s10516-017-9348-0.
Jensen, M.C. and Meckling, W.H. (2019), “Theory of the firm: managerial behavior, agency costs and ownership structure”, in Corporate Governance, pp. 77-132.
Krlev, G., Münscher, R. and Mülbert, K. (2013), “Social Return on Investment (SROI): state-of-the-art and perspectives-a meta-analysis of practice in Social Return on Investment (SROI) studies published 2002-2012”.
Laplume, A.O., Sonpar, K. and Litz, R.A. (2008), “Stakeholder theory: reviewing a theory that moves us”, Journal of Management, Vol. 34 No. 6, pp. 1152-1189, doi: 10.1177/0149206308324322.
Leatherman, S., Berwick, D., Iles, D., Lewin, L.S., Davidoff, F., Nolan, T. and Bisognano, M. (2003), “The business case for quality: case studies and an analysis”, Health Affairs (Project Hope), Vol. 22 No. 2, pp. 17-30, doi: 10.1377/hlthaff.22.2.17.
Leggat, S.G. (2007), “Information management: the limitations of ROI”, Australian Health Review, Vol. 31 No. 4, p. 488, doi: 10.1071/ah070488.
Lohr, K.N. (1990), Strategy Report; Medicare: A Strategy for Quality Assurance.
Lorelei, J., Linda, P., Glenn, R., Susan, B., Janet, E.A. and Naomi, J.F. (2017), “How do hospital boards govern for quality improvement? A mixed methods study of 15 organisations in England”, BMJ Quality and Safety, Vol. 26 No. 12, pp. 978-986, doi: 10.1136/bmjqs-2016-006433.
Masters, R., Anwar, E., Collins, B., Cookson, R. and Capewell, S. (2017), “Return on investment of public health interventions: a systematic review”, Journal of Epidemiology and Community Health, Vol. 71 No. 8, pp. 827-834, doi: 10.1136/jech-2016-208141.
National Health Service (NHS) (2021), Integrated Care Systems: Design Framework, NHS, London, available at: https://www.england.nhs.uk/wp-content/uploads/2021/06/B0642-ics-design-framework-june-2021.pdf (accessed 12 June 2024).
Oliver, C. (1991), “Strategic responses to institutional processes”, Academy of Management Review, Vol. 16 No. 1, pp. 145-179, doi: 10.2307/258610.
Organisation for Economic Co-operation and Development (OECD) (2015), “OECD principles of corporate governance”, available at: https://legalinstruments.oecd.org/en/ (accessed 12 June 2024).
Ostrom, E. (1991), “Rational choice theory and institutional analysis: toward complementarity”, American Political Science Review, Vol. 85 No. 1, pp. 237-243, doi: 10.2307/1962888.
Ozminkowski, R.J., Serxner, S., Marlo, K., Kichlu, R., Ratelis, E. and Van de Meulebroecke, J. (2016), “Beyond ROI: using value of investment to measure employee health and wellness”, Population Health Management, Vol. 19 No. 4, pp. 227-229, doi: 10.1089/pop.2015.0160.
Pfeffer, J. and Salancik, G. (2015), “External control of organizations—resource dependence perspective”, Organizational Behavior, Vol. 2, pp. 373-388.
Phillips, J.J. and Phillips, P.P. (2008), “Distinguishing ROI myths from reality”, Performance Improvement, Vol. 47 No. 6, pp. 12-17, doi: 10.1002/pfi.20003.
Scott, W.R. (2005), “Institutional theory: contributing to a theoretical research program”, Great Minds in Management: The Process of Theory Development, Vol. 37 No. 2, pp. 460-484, doi: 10.1093/oso/9780199276813.003.0022.
Solid, C.A. (2020), Return on Investment for Healthcare Quality Improvement, Springer, New York.
Stusser, R.J. and Dickey, R.A. (2013), “Quality and cost improvement of healthcare via complementary measurement and diagnosis of patient general health outcome using electronic health record data: research rationale and design”, Journal of Medical Systems, Vol. 37 No. 6, 9977, doi: 10.1007/s10916-013-9977-9.
Swensen, S.J., Dilling, J.A., Mc Carty, P.M., Bolton, J.W. and Harper, Jr, C.M. (2013), “The business case for health-care quality improvement”, Journal of Patient Safety, Vol. 9 No. 1, pp. 44-52, doi: 10.1097/pts.0b013e3182753e33.
Tarter, C.J. and Hoy, W.K. (1998), “Toward a contingency theory of decision making”, Journal of Educational Administration, Vol. 36 No. 3, pp. 212-228, doi: 10.1108/09578239810214687.
Tatum, B.C., Eberlin, R., Kottraba, C. and Bradberry, T. (2003), “Leadership, decision making, and organizational justice”, Management Decision, Vol. 41 No. 10, pp. 1006-1016, doi: 10.1108/00251740310509535.
Teisberg, E., Wallace, S. and O'Hara, S. (2020), “Defining and implementing value-based health care: a strategic framework”, Academic Medicine: Journal of the Association of American Medical Colleges, Vol. 95 No. 5, pp. 682-685, doi: 10.1097/ACM.0000000000003122.
The Chartered Governance Institute UK & Ireland (CGI) (2024), “Corporate governance and the chartered governance Institute UK & Ireland”, available at: https://www.cgi.org.uk/ (accessed 12 June 2024).
The National Leadership Council (NLC) (2012), The Healthy NHS Board Principles for Good Governance.
The National Institute for Health and Care Excellence (NICE) (2011), Supporting Investment in Public Health: Review of Methods for Assessing Cost Effectiveness, Cost Impact and Return on Investment.
The World Health Organisation (WHO) (2000), Report. Health Systems: Improving Performance, Online, Switzerland, available at: https://www.who.int/publications/i/item/924156198X (accessed 12 June 2024).
Thusini, S., Milenova, M., Nahabedian, N., Grey, B., Soukup, T., Chua, K.-C. and Henderson, C. (2022), “The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review”, BMC Health Services Research, Vol. 22 No. 1, p. 1492, doi: 10.1186/s12913-022-08832-3.
Thusini, S., Soukup, T., Chua, K.-C. and Henderson, C. (2023), “How is return on investment from quality improvement programmes conceptualised by mental healthcare leaders and why: a qualitative study”, BMC Health Services Research, Vol. 23 No. 1, p. 1009, doi: 10.1186/s12913-023-09911-9.
Torfing, J. (2012), Interactive Governance: Advancing the Paradigm, Oxford University Press, Oxford.
van der Goes, D.N., Edwardson, N., Rayamajhee, V., Hollis, C. and Hunter, D. (2019), “An iron triangle ROI model for health care”, ClinicoEconomics and Outcomes Research: CEOR, Vol. 11, pp. 335-348, doi: 10.2147/CEOR.S130623.
Veillard, J.H.M., Brown, A.D., Barış, E., Permanand, G. and Klazinga, N.S. (2011), “Health system stewardship of National Health Ministries in the WHO European region: concepts, functions and assessment framework”, Health Policy, Vol. 103 Nos 2-3, pp. 191-199, doi: 10.1016/j.healthpol.2011.09.002.
Willson, A. (2015), “The problem with eliminating ‘low-value care’”, BMJ Quality and Safety, Vol. 24 No. 10, pp. 611-614, doi: 10.1136/bmjqs-2015-004518.
Acknowledgements
Funding: This work is supported by the Economic and Social Research Council (grant number: ES/P000703/1). Infrastructure support for this research for TS was provided by the NIHR London MedTech and In Vitro Diagnostic Co-operative.
Conflicting interests: TS received funding from Cancer Alliance and NHS England for training cancer multidisciplinary teams (MDTs) in assessment and quality improvement methods in the United Kingdom. TS received consultancy fees from Roche Diagnostics, Parsek and Salutare. The other authors declare that they have no competing interests.
Authors’ contributions: As part of her Ph.D. project, ST performed the studies discussed in paper under the supervision of co-authors TS and CH. ST wrote the manuscript and compiled all the figures in this manuscript. All authors advised, reviewed and approved the development of this manuscript and figures.