Unpacking the context of value for money assessment in global markets: a procurement option framework for public-private partnerships

Abstract

Purpose

The unprecedented SARS-CoV-2 (COVID-19) pandemic has further constrained the budgets of governments worldwide for delivering their much-needed infrastructure. Consequently, public-private partnerships (PPPs), with the private sector's investment and ingenuity, would appear to be an increasingly popular alternative. Value for money (VfM) has become the major criterion for evaluating PPPs against the traditional public sector procurement and, however, is plagued with controversy. Hence, it is important that governments compare and contrast their practice with similar and disparate bodies to engender best practice. This paper, therefore, aims to understand governments' assessment context and provide a cross-continental comparison of their VfM assessment.

Design/methodology/approach

Faced with different domestic contexts (e.g. aging infrastructure, population growth, and competing demands on finance), governments tend to place different emphases when undertaking the VfM assessment. In line with the theory of boundary spanning, a cross-continental comparison is conducted between three of the most noticeable PPP markets (i.e. the United Kingdom, Australia and China) about their VfM assessment. The institutional level is interpreted by a social, economic and political framework, and the methodological level is elucidated through a qualitative and quantitative VfM assessment.

Findings

There are individual institutional characteristics that have shaped the way each country assesses VfM. For the methodological level, we identify that: (1) these global markets use a public sector comparator as the benchmark in VfM assessment; (2) ambiguous qualitative assessment is conducted only against PPPs to strengthen their policy development; (3) Australia's priority is in service provision whereas that of the UK and China is project finance and production; and (4) all markets are seeking an amelioration of existing controversial VfM assessments so that purported VfM relates to project lifecycles. As such, an option framework is proposed to make headway towards a sensible selection of infrastructure procurement approaches in the post COVID-19 era.

Originality/value

This study addresses a current void of enhancing the decision-making process for using PPPs within today's changing environment and then opens up an avenue for future empirical research to examine the option framework and ensuing VfM decisions. Practically, it presents a holistic VfM landscape for public sector procurers that aim to engage with PPPs for their infrastructure interventions.

Citation

Zhao, J., Thurairajah, N., Greenwood, D., Liu, H. and Yuan, J. (2023), "Unpacking the context of value for money assessment in global markets: a procurement option framework for public-private partnerships", Engineering, Construction and Architectural Management, Vol. 30 No. 8, pp. 3583-3601. https://doi.org/10.1108/ECAM-10-2021-0963

1. Introduction

The global economy has been encountering severe turmoil since the spread of SARS-CoV-2 (COVID-19) in 2020. A baseline forecast by The World Bank (2020) estimated a 5.2% contraction in global GDP in 2020. Faced with a recession, the United Kingdom (UK) government, in common with others, aimed to invest £100 billion in 2021–22 to resuscitate the country's economy (HM Treasury, 2020a). Of this stimulus package, £27 billion was earmarked for economic infrastructure (HM Treasury, 2020a), highlighting the role of infrastructure development in the recovery. Nevertheless, as illustrated by UK Parliament (2021), the UK has under-invested in infrastructure for decades. According to the financial services company–Legal and General (2020), the infrastructure investment gap between 2020 and 2030 is circa £1 trillion. One possible solution to balance this funding shortage against incremental demands appears to lie in public-private partnerships (PPPs) (Chowdhury et al., 2011; Ma et al., 2020). During COVID-19, Casady and Baxter (2021) postulate that delivering healthcare infrastructure through unsolicited PPPs would not only foster rapid response but also mitigate its aftermath. In this stance, the pandemic can be treated as an “opportunity” rather than a “threat” to unleash PPPs' potential to address the above dilemma. However, PPPs certainly are not a “panacea”, which was reflected in the exposed failures reported by Zhang and Tariq (2020). In addition, the debt of the UK, according to the Office for National Statistics (2021), has risen above the European Union (EU) average (i.e. 12.3% higher) during the pandemic, which means the public budget has to be carefully allocated after COVID-19. Therefore, we need to understand the macro and micro contexts that can accommodate PPPs.

The debate on the utility of PPPs is enduring (see, for example, Shaoul, 2005; Hodge et al., 2018; and Verweij and van Meerkerk, 2021). Proponents cite their abilities in easing governments' budget constraints (Chan et al., 2009), transferring risks to the private sector (Jin and Zhang, 2011), and curbing delays and cost overruns (Raisbeck et al., 2010). As a consequence, more than 700 projects of this nature (tallying around £56 billion capital investment) have been enacted in the UK (HM Treasury, 2021). However, according to Hodge and Greve (2017), solid evidence to support the rhetoric is extremely rare. If anything, most commentators are critical and argue to the contrary. Subsequently, the UK announced in 2018 that no new Private Finance 2 (PF2) projects would emerge due to their less-than-satisfactory performance (e.g. significant fiscal risk) (HM Treasury, 2021). Another example arose in China where Xiong et al. (2021b) contend that political opportunism has partially contributed to the failures of PPPs. Although this does not mean the end of PPP types of contracts, the value for money (VfM) assessment that justified their use has been undoubtably questioned. As a relatively simple way of comparing costs and benefits, VfM assessment has become an indispensable component in the public procurement process. Nevertheless, the methodology enshrined in VfM assessment has been criticised as being deeply flawed and un-rigorous (Shaoul, 2005; Zhao et al., 2022). In order to improve delivery of infrastructure in the post COVID-19 epoch, governments, especially those experiencing “failures” with PPPs, therefore need to learn from each other and be equipped with a robust instrument that can evaluate their VfM. This is supported by the theory of “boundary spanning”, where Marrone (2010) argues that organisations must increasingly coordinate across their boundaries and actively manage external relationships to achieve success. To this end, this article aims to address the following research questions:

RQ1.

How has VfM been assessed in global PPP markets? and

RQ2.

How can governments (specifically the UK government) capitalise on best practice in the post COVID-19 epoch?

In the existing literature, studies on VfM in a single country are not scarce. For instance, Ismail (2013) used survey results to propose a VfM assessment framework that integrates financial and non-financial aspects in a Malaysian context. Opara (2018) suggested improved information disclosure, transparency and risk quantification of VfM assessment in Canada. Acknowledging the need to engage with what Aldrich and Herker (1977) call external information processing, Grimsey and Lewis (2005) compared the views of academics and practitioners on VfM assessment and its practice in different countries. Subsequently, Morallos and Amekudzi (2008) reviewed the VfM model adopted by agencies in Australia, Canada, Europe, and Asia. Addressing the variances in different states in the US, Morallos et al. (2009) surveyed their VfM analyses for transport projects. While these studies have attempted to span the single-country boundary to a number of settings to draw lessons, they focus on the VfM assessment without considering the context underlying it; do not reflect the spectrum of changes within organisations, particularly in the most recent situations; and do not provide a possible solution to the procurement conundrum. Therefore, this paper provides a timely inquiry to make sense of the VfM assessment in the global market, which is particularly directed at policy in the UK, to inform the use of PPPs in the post COVID-19 epoch. A fresh approach, through these findings, would enable decision-makers to garner an understanding of how VfM assessment can be better utilised. We now review the concept of PPPs, VfM and their significance in infrastructure procurement before heading to the methodology.

2. Public-private partnerships (PPPs) and value for money (VfM)

2.1 PPP definitions

PPPs gained momentum in the UK in the 1990s where it took the form of Private Finance Initiative (PFI) and subsequently PF2. As an innovation to public procurement, PPPs have been adopted around the globe to deliver infrastructure projects and/or public services in the areas of transport, water, energy, education, etc. However, their common application does not result in a common definition of PPPs, as governments assume different priorities and intentions (Muleya et al., 2020). Cherkos and Jha (2021) report that emerging markets embrace PPPs mainly through economic and financial stimuli, compared with developed countries' pursuit of service quality. As a result, various approaches such as PFI, build-operate-transfer (BOT), concession and franchise, have been generated to accommodate multiple types of assets (e.g. new or existing), functions borne by private sectors, and payment sources such as users or governments (The World Bank, 2017). In the UK, HM Treasury (2021) defines PFI as “a long-term contract between a private party and a government entity where the private sector designs, builds, finances and operates a public asset and related services.” Australia's Department of Infrastructure and Regional Development (2008a) perceives PPPs as “a long-term contract between the public and private sectors where government pays the private sector to deliver infrastructure and related services on behalf, or in support, of government's broader service responsibilities”. China, on the other hand, seeks to build a long-term partnership where private entities design, build, operate and maintain the infrastructure while the government supervises its price and quality (Ministry of Finance–MoF, 2014). These definitions reinforce the perception that a “one-size-fits-all” approach to PPPs may be problematic. However, in line with Collier (1993), evaluating cross-experiences could facilitate the identification of problems and promote best practice in different settings.

2.2 The concept of VfM

Another element that is intertwined with PPPs is VfM. According to Almarri and Boussabaine (2017), the viability of PPPs is determined by VfM to demonstrate the additional value realisation through private participation in infrastructure (PPI). The use of VfM ranges from daily life (e.g. buying a phone) to professional trade (e.g. selecting a best practice procurement approach). Yet in the latter, the concept of VfM is not clear-cut because of variables such as stakeholders, measurement, attribution, and stability (McKevitt, 2015). One of the most cited definitions of VfM is that it “is the optimum combination of whole-of-life costs and quality (or fitness for purpose) of the product or service to meet the users' requirement” (Morallos and Amekudzi, 2008). Similarly, Almarri and Boussabaine (2017) argue that life-cycle cost efficiency and clear service outputs should be added to VfM. On the other hand, the “3 Es” (economy, efficiency and effectiveness) plus a recent fourth “equity” are commonly used as proxies for VfM (Jackson, 2012). Ismail (2013) affirms that VfM depends on realising technical innovation through competitive tendering. In other words, commentators consider VfM to be a function of multi-attributes. Nevertheless, Ng et al. (2012) and Cui et al. (2019) have identified that cost effectiveness is the most fundamental driver. This to some extent explains why cost is paid overriding attention in actual VfM assessment.

2.3 Why is VfM assessment important?

VfM assessment can be classified into ex ante assessment and ex-post assessment. The mainstream role of the former is to determine an optimal procurement route between different options at the initial decision-making stage. Typically, it is conducted by comparing the net present value of a PPP with that of a traditional public procurement option (Ismail, 2013). The ex-post VfM assessment is often entangled with performance measurement to target whether VfM has been realised via the selected method (Liu et al., 2018). Some organisations, such as the UK's National Audit Office (NAO), have institutionalised VfM assessment into the scrutiny of government spending, thereby aligning and comparing an ex-post with the ex ante VfM assessment (Heald, 2003). Given the lump-sum capital investment, unsuccessful infrastructure delivery will not only result in the financial vulnerability of stakeholders but loss of overall social welfare. As such, according to Shi et al. (2020), VfM assessment has attracted attention in academia and formed a major research area in PPP related studies. In practice, it has become a mandatory procedure in the procurement process of some countries (e.g. UK, Australia and China) if PPPs are being considered. A number of other countries, such as Belgium (Van Den Hurk, 2018), Malaysia (Ismail, 2013), Albania (Keci, 2019), and Vietnam (Hang, 2016) are also proposing and implementing their own VfM frameworks.

2.4 Problems with VfM assessment

The pervasive use of VfM assessment in project evaluation requires the methodology itself to be sound and reliable, otherwise the validity of the decision would be in doubt. However, current questions in the VfM debate include, inter alia: what is a suitable discount rate? And should the same discount rate be used for evaluating PPPs and traditional procurement? Jomo et al. (2016) confirm that discounting PPP costs at a higher discount rate renders a lower, more attractive net present equivalent, and thus may bring a disproportionate advantage to the PPP option. Another argument concerns the balance of risk allocation between the two main contractual parties (Jin and Zhang, 2011). There are cases where undue risks have bankrupted the PPP provider. For example, Ng and Loosemore (2007) report that Airport Link Company, the private consortium of the $920 million New Southern Railway project in Sydney, Australia entered into receivership due to the project's controversial risk allocation. In addition, with a contract valid up to 30 years, an exhaustive and accurate prediction of risks and their valuation is a persistent challenge (Kumar et al., 2018). More importantly, Grimsey and Lewis (2005) argue that VfM assessment relies heavily on a hypothetical cost construction of a public delivery, known as the “public sector comparator” (PSC), which evades an “apple-to-apple” comparison. Therefore, Opara (2018) concerns that VfM assessment is compromised as a bureaucratic tool to legitimate a pre-conceived mindset, i.e. that PPPs are better. Examples have been seen worldwide (including those in the UK, the EU, Australia and the US) that PPPs have, retrospectively, been shown to be more expensive than estimates of the same delivery using a traditional method (Hodge and Greve, 2007; Leigland, 2018). If headway is to be made against these problems, after decades of PPP development, it is necessary to conduct a comparative study of global markets to extract best practices, particularly when the post COVID-19 era calls for more prudent public expenditure.

3. Methodology

To recap, the work presented here aims to streamline the UK's VfM assessment practice by making sense of the “context” from a global lens. As Davidoff (2019) put it: “context plays an important role in both improvement science and implementation science; limited understanding of context therefore limits understanding of both the fundamental principles of improvement and the actions that put improvements into practice.” Essentially, the importance of context has been emphasised in infrastructure research, such as Hertogh et al. (2008), OMEGA centre (2012) and Love and Ika (2021). Noting the hierarchical levels of context identified by Biggermann and Buttle (2009), we framed the VfM context to the institutional (macro-level) and the methodological (micro-level) perspectives. In particular, the institutional level was interpreted by a social, economic and political framework, and the methodological level was elucidated through a qualitative and quantitative VfM assessment.

In addition to the internal (“-emic”) context, boundary spanning theory has called for the external (“-etic”) information processing to assist implementation and improvement (Marrone, 2010). To do so, Esser and Vliegenthart (2017) suggest that a comparative analysis would fit as a boundary spanner to gain a deep understanding of one's own system by comparing against the routine prevalent in other countries. While there are many countries implementing PPPs, three criteria were adopted to choose the sample, and they consider: (1) similar and different systems that can capture variances as well as consistencies as proffered by Lor (2010); (2) representativeness of PPP experience; and (3) data accessibility (Table 1). Accordingly, three countries – the UK, Australia, and China – were selected because: (1) they have different institutional characteristics that to some extent underlie their methodological approaches to VfM assessment (i.e. different systems); (2) VfM assessment is legal procedure in these three countries that has to be followed if PPPs are deemed viable (i.e. similar systems); and (3) the UK and Australia are widely considered mature PPP markets in terms of their complexity and volume of projects (Grasman et al., 2014). China's PPP market, since its official adoption in 2014, has grown to be the world's largest (currently c.£16 trillion - 28 times larger than the UK's) and Perera et al. (2019) have equated its maturity to that of the UK and Australia (i.e. representativeness). According to Seawright and Gerring (2008), sample selection is by no means an easy task and requires an agenda of study. Nevertheless, by following the criteria and the reasons explained above, we submit that this comparison shows useful variations on the dimensions (macro and micro; internal and external) of theoretical interest (Seawright and Gerring, 2008), and can act as a point of departure for a better VfM assessment. Figure 1 outlines the overall research framework.

4. Findings – a tale of three practices

The motivation for this study is driven by the demand for infrastructure provision in the post COVID-19 era and the abolition of PFI and PF2 in the UK. It should be noted that the “pause” that followed did not discredit PPPs per se. On the contrary, they are booming both in the mature and emerging economies, and the UK is establishing a new infrastructure bank to harness PPI to expedite the recovery from COVID-19 (HM Treasury, 2020a, p. 70). It is with this backdrop that we now try to make sense of the previous “failures” and propose an option framework for the future by presenting the results from the comparative analysis. By “the future” or “better PPP adoption”, we refer to the provision of equivalent services at a less cost or better services at the same cost (Dixon et al., 2005). That is, true VfM can materialise.

4.1 Institutional characteristics

Infrastructure, labelled as “economic arteries and veins”, is inextricably related with economy. As noted by The World Bank (2006), a nation's socio-political environment shapes its development process and vice versa. The purpose of this study is not to delve deeply into their functioning mechanisms, which are sophisticated and delicate, and happen to be beyond the scope of this study. Instead, we delimit it to the specific social (e.g. population boom and urbanisation), economic (e.g. infrastructure provision) and political (e.g. policy and governance structure to PPPs) framework (Table 2) that conditions PPPs and VfM assessment. This explorative perspective has shown that: (1) some institutional barriers need to be removed before PPPs adoption; (2) PPPs are an approach that enables a particular government to deliver its promises to society; (3) The use of PPI is a way to stimulate economy; (4) The social, political and economic backgrounds to some extent determine how PPPs are applied; and (5) VfM assessment becomes an instrument to legitimate PPPs and is necessary to monitor whether the best VfM is delivered.

4.1.1 UK context

In the UK, the institutional barriers to PFI were cleared following the 1992 election of a Conservative government. To cope with an economic crisis that involved high unemployment, high interest rates, and high public borrowing, there was a shift to “NPM” (see Hood, 1991 for details regarding NPM), and departure from the “Ryrie rules” that limited the raising risk capital from financial markets (see Heald and McLeod, 2002, p. 420). In 1997 the incoming Labour government adopted PFI to improve public services: commissioning, between 1997 and 2003, 34 PFI hospitals with an estimated cost of £21.76 billion and expanding PFI into other areas such as education. The first VfM assessment guideline (PSC) was introduced by the Private Finance Treasury Taskforce in 1999, and in 2003 VfM became the criterion, as opposed to the simple “off-balance sheet” attraction of earlier years. The number of UK PFI projects then remained stable (around 60 every year) until the 2008 financial crisis. Despite the introduction of the amended “PF2” strategy in 2012, by 2018 only one PFI project (i.e. Arc21 Residual Waste Infrastructure Procurement) was commissioned at which time PFI and PF2 were deemed inflexible and complex.

4.1.2 Australian context

Unlike the UK, Australia has a federal parliamentary system. The first formal adoption of PPPs occurred in 2000 when the Victoria State Government established “Partnerships Victoria”. Other state PPP units followed, including “Projects Queensland” (now “Queensland Treasury's Commercial Group”) and “New South Wales (NSW) PPPs” (now the “Infrastructure and Structured Finance Unit”). These are responsible for the procurement of PPPs in each jurisdiction and apply state-specific guidelines (Table 3) where the National PPP Policy and Guidelines (NPPG) allow. At the federal level, the Council of Australian Governments monitors, reviews, and refines the NPPG with the assistance of its “holder”, Infrastructure Australia. Table 3 shows the relevant guidelines alongside the uptake of PPPs by each unitary player (the three major states and Infrastructure Australia). Notably, the populations of NSW, Victoria and Queensland account for 77.85% of total Australian population, which accounts for the predominance of PPP projects in these states. With the reform of its Australian Public Service (Australian Government, 2019), PPPs continue to breathe and grow even amid the COVID-19 as can be seen in the Sydney Metro City and Southwest OTS2 PPP, the Footscray Hospital PPP, the Inland Rail PPP in NSW, Victoria and Queensland, respectively.

4.1.3 China context

In China, an aging population, extreme poverty for 100 million people, urbanisation, and environmental worries, have all provided a stimulus to innovate in infrastructure. This contextual backdrop coincides with the surging number of PPPs in the area of urban and city development, elderly care, environmental protection, and social housing. The first BOT project (i.e. Shajiao B power plant) in Shenzhen, China can be traced back to 1984 with foreign direct investment. However, the central government's enthusiasm for PPI in 2014 (see Cheng et al., 2016 for macroeconomic environment and policies that shaped PPPs in China pre-2014) casted a watershed in PPPs. This was attributed to the newly elected administration declaring, in 2013, the decisive role of the market in resource allocation and allowing the private sector to invest in infrastructure. There was a milestone policy by the MoF (2014) that considered PPPs as a way to transform economy, support urbanisation, convert the role of government in public service, and reform the finance and taxation system. So far, 10,120 PPP projects have been commissioned across China led by the municipal sector, transport, environment, and urban and city development. One significant characteristic embedded with this rapid uptake is the involvement of state-owned enterprises (SOEs) [1] due to their ample resources and extensive political and financial access (Xiong et al., 2021a). In addition, dozens of laws, regulations and policies have been administered mainly by its national-level Standing Committee of the National People's Congress, State Council, MoF, National Development and Reform Commission to promote, regulate and stabilise PPPs. However, the perception of PPP as merely a source of finance has led to some concerns, and the MoF (2019) has warned some local authorities against the excessive invisible public deficits that may result.

4.2 What do “VfM” and VfM assessment mean in the context?

As the pioneer of PPPs, the UK has been grappling longest with their assessment. Specifically, the UK has replaced the PSC model developed in 1999 with a three-level (programme level, project level, and procurement level) assessment in 2004 and 2006, withdrawn the quantitative assessment in 2012, and re-invigorated PSC in 2020 (HM Treasury, 2020b). In contrast, Australia maintains its 2008 version while China updated its 2015-practice in 2016. In addition to the UK's definition of VfM within these documents, Australia specifies “VfM is a combination of the service outcome to be delivered by the private sector, together with the degree of risk transfer and financial implications for government.” Although China does not have an explicit VfM definition, it emphasises the improvement of service quality and operation efficiency, or reduced project cost over the project lifecycle (MoF, 2014).

It should be noted that here VfM is considered in the context of a comparison between PPPs and traditional procurement. Other forms of procurement may fall into a wider evaluation. For example, Australia enacts a “procurement options analysis” that can evaluate PPPs against construct-only, design and construct, alliance contracting, etc. in areas such as objectives, policy context, agency capability, and market. For PPPs to qualify as a potential VfM alternative, each country has a shortlisting mechanism, shown in Table 4. Despite the $50 million restriction in Australia, small projects that present measurable risk transfer, whole-of-life costing, innovation, measurable outputs, asset utilisation, better integration, and competitive process may also qualify for PPPs. Compared with the conditions required in the UK, in Australia and China projects with certain characteristics (Table 4) can be identified. If the listed thresholds are met, a VfM assessment is then undertaken between PPPs and the traditional procurement approach.

4.3 When does VfM assessment take place?

The UK's three-stage VfM assessment happens during the annual budgeting round, outline business case (OBC) and post-OBC to financial close, respectively. In the latest Green Book (HM Treasury, 2020b), these stages have been restructured as the longlist and shortlist appraisal stages. Australia and China conduct assessments after the investment decision is made and before the request for proposal is launched. In addition, China requires a mid-term assessment (3–5 years after the project is in operation) to check if the initial VfM is attained. There are also differences in the order of quantitative assessment (i.e. PSC) and qualitative assessment. Australia and China proceed with the quantitative assessment followed by a qualitative assessment. This emphasises the importance of the qualitative assessment, particularly when the PSC is close to the bidders' lowest price. The UK, however, has shifted from an identical practice to the opposite procedure, where critical success factors and other qualitative issues are assessed first, followed by a PSC calculation. A potential problem with this approach could be that the earlier qualitative assessment is not well interpreted (Coulson, 2008) and repeats the suitability test that is used where projects amenable to PPPs are subjected to preliminary screening. This is exacerbated by the evidence that UK's PSC guidance is biased towards PPPs (Pollock et al., 2007). Similarly, China originally used a qualitative assessment certified by a group of experts, with the quantitative assessment being at the discretion of responsible agencies. The transformation to its current practice may again corroborate Coulson's (2008) concerns about qualitative VfM. The implication is that the UK should perhaps consider the general processes prevailing in Australia and China and thus avoid unnecessary repetition.

4.4 How is VfM assessed?

4.4.1 Quantitative VfM

As mentioned above, PSC represents the hypothetical cost of a traditional procurement approach which in turn exposes the cost difference between that and a PPP in order to demonstrate VfM. Currently, the components of PSC are not detailed in UK's Green Book 2020. Drawing on relevant literature and practices in Australia and China, a PSC can be said to comprise: a “raw” PSC (i.e. the construction and operation costs associated with delivering the output specifications over a period), competitive neutrality, transferred risk and retained risk. This benchmarking cost can be revisited when consulting private sectors to illuminate potential market capability before the formal tendering. In Australia, it is then compared against the PPP bidders' price to quantify VfM. In China, a PPP value, which incorporates the cost the government is required to bear in the PPP scenario, is calculated. As it is undertaken at the pre-tender stage, this PPP value is akin to a shadow bid value (Grimsey and Lewis, 2005, p. 353). In addition to the PSC comparison against a PPP, an additional comparison between the value of a PPP version of “do the minimum” and a normal PPP is required in the UK. Furthermore, the comparison can be widened to include “Business as Usual”, “do the minimum option”, “PPP”, and any other viable alternative if no outsourcing or insourcing change exists. This results in a cost-benefit analysis similar to the approach taken at investment decision stage.

The importance of selecting a discount rate which underpins the net present value calculation is recognised. China proposes a discount rate based on local governments' bond yields (e.g. a road project procured in 2019 in Fujian used 4.08%) for both the PSC and PPP. It also requires that, if there are multiple discount rates available, the minimum discount rate should be used. We understand this as an attempt to avoid the debate that a higher discount rate underestimates the value of a PPP. The use of a single discount rate also reveals the lack of a sensitivity analysis (which is common in the UK and Australia) to trial the impact of different discount rates on decision-making. Regarding Australia's social infrastructure, the PPP side discount rate is adjusted to reward the private sector for assuming the transferred risks. For example, a risk premium is added to the risk-free discount rate based on the percentage of risk sharing. Although this practice has its roots in the capital asset model, the presumption that governments can really transfer risks to the private sector can be disputed (Pollock and Price, 2004). For its economic infrastructure, the project rate and risk-free rate are used in a PSC and a PPP, respectively. In the UK, a “social time preference rate” of 3.5% is applied for all possible options at the shortlist stage. It shows the government prefers the present society to the future, which in turn fits the institutional characteristic that the UK's PFI is finance-oriented. This is reflected by the £199 billion that the UK government has to pay for existing PFI projects until the 2040s (NAO, 2018).

To enable better risk management, all three countries uniformly price risks that governments are exposed to in PSC. In the process, risks are identified, and their probabilities and impacts are combined. Point estimate and Monte Carlo simulation are recommended as techniques for risk quantification in the UK and Australia. The UK additionally suggests decision trees and real options for a follow-up decision as the project progresses. Instead of instructing these techniques, China promotes the use of scenario analysis (in cases where the impacts of risks can be measured but not their probabilities); a percentage method (when both impacts and probability are hard to estimate); and the “probability × impact” method (when both can be calculated). Risk valuation is ultimately split into retained risks and transferred risks to prepare for the risk sharing that exists in PPPs. In order to avoid the illusion that a large project can be created with a small amount of investment, the UK has included an “optimism bias” adjustment based on experience of public-funded infrastructure. However, it is not clear how this concern is addressed in PPPs. For example, can a lower “optimism bias” percentage be applied to a PPP bid since private sectors are considered to have greater expertise? Moreover, empirical data reveal that change of “scope” and “client requirement” lead to project cost inflation (Love et al., 2019). Similarly, transaction costs, which can be as high as 20% of the capital investment in PPPs are not clearly addressed. Such omissions can sow the seeds for an overestimation of a PSC and an underestimation of a PPP.

4.4.2 Qualitative VfM

In light of the extensive criticism of the UK's PSC practice (e.g. Shaoul, 2005; Pollock et al., 2007), the quantitative assessment became dormant in 2012. As previously mentioned, despite the resurgence of PSC in 2020, its components and how it is operated are elusive. However, a new form of qualitative assessment at the longlist stage can reveal the social value of a project intervention. Table 5 outlines the qualitative factors that are considered in each of the three countries.

Spackman (2002) and Sun et al. (2021) argue that financial constraint skews the ideology to PPP forms of procurement in the UK and China. Consequently, a large number of projects are made possible by leveraging up limited budgets to meet immediate infrastructure demands. The concomitant risk is an uplifting public debt level and the jeopardising of the long-term VfM (Ball et al., 2001). In practice, a red flag was waved by China's State-owned Assets Supervision and Administration Commission (2021) regarding local SOEs' debt risk. The UK and Australia have a similar affordability analysis to avoid using PPPs simply as a way of off-balance sheet funding. Currently, this affordability is set at around 10–15% of total investment in public services. However, in Australia's qualitative assessment, service is emphasised through combined consideration of project management and prescient design inclusion. The ensuing result is its better performance at least in terms of cost and time (Raisbeck et al., 2010). In summary, the qualitative assessment employed by each of the three governments reflects their policy orientation in a specific spectrum, but each is subject to methodological weaknesses.

The emphasis on “service” does not make the qualitative assessment in Australia faultless. Compared with the UK and China, not only is the number of factors considered confined but also their assessment is unclear. In the UK a series of simple questions (see Table 5) have to be answered by the procuring team to pass the evaluation. By contrast, China implements a relatively robust qualitative assessment. An even number (more than nine) of experts in the fields of finance, accounting, regional development, construction, etc. are summoned to rate the weighting and score of each factor using criteria set by the local PPP unit. A total weighting of 20% is assigned to the “supplementary factors” that are not outlined in Table 5 to accommodate the project characteristics. The threshold between “fail” and “pass” is 60. However, a weighted average of over 80 can waive the need for a PSC, instigating pressure on the panel's independence and professionalism. Issues that are common to all three countries are that: (1) factors are appraised purely against the PPP option (and not against its traditional procurement alternative); and (2) the criteria are generic and not sector-sensitive. For qualitative issues to play their part in VfM calculations there should be a carefully considered and rigorously designed qualitative assessment to minimise bias and subjectivity.

5. The option framework

The UK has been confronted with the controversial use of PFI in that sometimes services are compromised (Ahmad et al., 2021), costs more expensive (40%) (NAO, 2018), and best VfM not achieved (Heald, 2003). The institutional characteristics and VfM assessment have provided an understanding of why this is the case by comparing the UK with Australia and China. It further corroborates the inherent political nature of PPPs as argued by Hodge and Greve (2017). However, even when the institutional barriers are removed to advocate PPPs, our findings identify that: (1) VfM is increasingly lauded to rationalise PPPs in the scheme of things; and (2) lessons can be learnt from the global market to improve the VfM assessment. We concur with Wu et al. (2016) that as governments will be held ultimately accountable for public expenditure, a robust VfM assessment is required to defend the move to PPPs. In Figure 2, therefore, we propose an option framework as the catalyst for action albeit its conceptual nature. Its aim is to stimulate an enhanced practice and to accommodate the institutional characteristics (as we by no means advocate a “one-solution-fits-all” approach). Notably, this framework is designed in the UK context. However, it can be adapted to fit other national settings.

In the face of what Pollock et al. (2002) call the “sleight of hand” in justifying PPPs, a government-wide definition of VfM which integrates government-side considerations (e.g. cost savings) and taxpayer-side benefits is urgently needed. This compound definition is supported by the global market's consensus that cost is not the sole determinant of VfM. The emergent prototype (i.e. a standard VfM definition) then sets the tone for VfM assessment and particularly how qualitative assessment is employed. The implications are that quantitative assessment and qualitative assessment (which often uses a quantitative scoring system) are complementary and together yield a solid decision. The importance of the qualitative assessment becomes more relevant as Vickerman (2021) argues that COVID-19 has made the prevailing competitive model (i.e. low costs as in the quantitative assessment) infeasible. In fact, Butcher (2018) has suggested a transition from “on the market” to “on the track” competition (i.e. performance measurement as in the qualitative assessment) to sustain the UK's rail system.

To address the problem of process repetition identified in the UK, a screening test is proposed prior to the VfM assessment. In it, the affordability analysis is similar to existing examples, but the “exclusion terms and conditions” will shortlist projects for sector-specific VfM dimensions check. By doing so, this initial test appraises all available options rather than the previous PPP-only qualitative assessment, and includes the currently absent but important sector-specific circumstances (Roe and Craig, 2004). The necessity of this is emphasised by the fiasco of ICT contracts revealed by Whitfield (2007) which demonstrates that PPPs are not suitable for all areas. If PPPs are potentially suitable, they will be compared against the best possible public delivery, which would otherwise be the VfM option. In addition to the normal Green Book evaluation (e.g. “Business as Usual” and “do the minimum option”), we add private competition (as opposed to PSC) to ensure the best practice is selected from other types of procurement (e.g. design-build and alliancing). The rationale of competition in both sides lies in the fact that if PSC is there to demonstrate the VfM of PPPs, a “private sector comparator” should be formed to stimulate the public sector (Burger and Hawkesworth, 2011). This “public-private” and “private-public” competition is important as it can compensate for the limited competition between bidders. This arises, according to OECD (2014), from the limited tender participation due to the complexity of PPPs, leading to potential monopoly and thus the sacrifice of VfM. Therefore, the result (best practice) generated from the rigorous screening and suitability test will be able to deliver VfM and in turn justify the institutional characteristics that originally underlie the VfM assessment. Equally, as the framework is fixed and consistent, concerns raised by Shaoul et al. (2010) and NAO (2018) over the previous obscure process can be mitigated to encourage transparency.

6. Conclusions

PPPs have been globally adopted to deliver infrastructure and/or provide public services in lieu of the traditional approach to public sector procurement. PPPs are however, plagued with controversy as to whether the purported advantages materialise over project life-cycles. Failures of this nature have led to the suspension of PFI and PF2 in the UK, which inevitably maligned the already controversial VfM assessment that rationalises PPPs. Given the significant role infrastructure plays (including in recovering from COVID-19) and the current lack of detailed VfM assessment in the UK, it is imperative that best practices are extracted to safeguard the public purse when it prepares for future forms of PPPs. The intention of this paper is not to conclude on the superiority of one practice over another. On the contrary, it calls for a sober consideration of global practice and argues for a more rigorous calibration of the existing procurement approach.

In line with the theory of boundary spanning, the UK, Australia and China are selected to make sense of the way VfM assessment is underpinned by their individual institutional characteristics. In general, the institutional characteristics (Table 2) have shaped how VfM is assessed. Specifically, the UK and China converge on the financial stimulus that drives the use of PPPs while Australia is service-oriented. Contrary to the stereotype, China is shown to be exerting the power of the market on PPP infrastructure delivery. In terms of the concept of VfM, the UK focuses on quality and whole-of-life cost while Australia seeks service, risk transfer and cost, and China prioritises cost, service quality and operational efficiency. As a consequence, PSC serves as a reliable tool in Australia and China for comparing the net present value of two options. The record of PSC in the UK is a recurring theme of adoption, replacement, withdrawal and re-adoption. Yet, the current version remains vague on its components and how it operates. Other issues such as “optimism bias” and transaction costs are touched upon but are not clearly estimated especially in the case of the evaluation of PPPs. In light of the potential manipulation of PSC, the spotlight has shifted to qualitative assessment. Both Australia and China conduct such assessment after the PSC comparison, while the UK undertakes the opposite. The concrete steps take the form of questions in the UK and a weighted average in China capitalising on experts' experience. Australia, on the other hand, proposes a few qualitative factors without providing “how”. The findings further reveal that in spite of the “weaknesses”, China has a direct and simple way on both types of assessment whilst the UK is enigmatic on PSC and Australia falls short on qualitative assessment. The understanding of the institutional characteristics and VfM assessment then provide a foundation for the option framework (Figure 2) for improvement. By considering the UK context, under the auspices of a standard VfM definition, it combines a screening test, comprising shortlisting mechanisms, affordability analysis and the sector-specific VfM factors check, and a VfM assessment that consolidates public competition for PPPs and private competition for other types of procurement. The standardised and consistent approach to infrastructure procurement can solve the repetition and conflict inherent in the current evaluation tool and increase transparency.

The three-country comparison is a limitation of the study, as there are undoubtedly other national approaches to be considered and this could form the basis of further work. Nevertheless, examining the similarities and differences of the three selected countries is informative because it: (1) presents a holistic VfM landscape for the public sector that aims to engage with PPPs for their infrastructure interventions; and (2) develops an option framework for the recalibration of the existing procurement approach and provides a platform for future research to empirically examine the option framework and the ensuing VfM.

Figures

Figure 1

Research framework

Figure 2

An option framework

Table 1

Countries and their respective VfM assessment guideline selected for research

Countries	Guidelines	Documentary sources
The UK	Value for Money Assessment Guidance	HM Treasury (2004)
	Value for Money Assessment Guidance	HM Treasury (2006)
	Quantitative assessment: user guide	HM Treasury (2011)
	The Green Book	HM Treasury (2020b)
Australia	National PPP Guidelines Overview	Department of Infrastructure and Regional Development (2008a)
	Volume 4 Public Sector Comparator Guidance	Department of Infrastructure and Regional Development (2008b)
	Volume 5 Discount Rate Methodology	Department of Infrastructure and Regional Development (2013)
China	PPP Value for Money Assessment Guidance	MoF (2015)
	PPP Value for Money Assessment Guidance	MoF (2016)

Table 2

Institutional characteristics of VfM assessment in the UK, Australia and China

	The social	The economic	The political (governance)
The UK	1) High unemployment rate; high interest rate; housing crisis; 2) Demand for quality NHS and education; 3) Under-investment in infrastructure; 4) Protection for staff; 5) Ongoing need for better public services, opportunity and security	1) The early 1990s recession; 2) Fiscal responsibility and government guarantee; 3) 60% PFI are on the balance sheet; 4) The 2008 financial crisis and tighter regulations on banks	1) NPM, retirement of the “Ryrie rules” and the 1992 general election; 2) the 1997 general election; 3) Using PFI to meet the investment challenge in 2003; 4) Using PFI to strengthen long-term partnership; 5) Introducing equity finance and transparency
Australia	1) Population size; 2) Expectation for excellence in public service provision; 3) Reluctancy to more tax	1) High public debt; 2) The longest sustained increase in commodity prices and the terms of trade but generally healthy*	1) New Public Management (NPM); 2) Federal government; 3) Reform of Australian Public Service
China	1) Population aging; 2) Poverty; 3) Environmental issues	1) Economic downward pressure; 2) Insufficient domestic demand	1) New administration; 2) Law modifications (e.g. long-term budget plans and taxation)

Source(s): *From Gerard and Kearns (2011) The Australian Economy in the 2000s

Table 3

Federal and state governance on PPPs in Australia

Unit	Guidelines	Document year	Number of projects/Project value after 2000*
Infrastructure Australia	NPPG contains: (1) National PPP Policy Framework; (2) National PPP Guidelines Overview; (3) Volumes 1–7 on detailed technical instructions; (4) Roadmap for Applying the Commercial Principles	2008: Original release; 2015: Revised version	90/≈$109.13 billion
Partnerships Victoria	Partnerships Victoria Requirements	2009: Original release; 2010: Update on PSC; 2013: Revised version; 2016: Revised version	24/≈$29 billion
Queensland Treasury's Commercial Group	Queensland public private partnership supporting guidelines	2015	11/≈$24 billion
NSW Infrastructure and Structured Finance Unit	NSW Public Private Partnership Guidelines	2012: Original release; 2017: Revised version	26/≈$38 billion

Source(s): *Secured from Infrastructure Partnerships Australia in July 2021. Guidelines are sourced from Department of Infrastructure and Regional Development (2008a), Treasury and Finance (2016), Queensland Government (2015), and The NSW Treasury (2017)

Table 4

Projects that may be suitable for PPPs

Countries	Conditions
UK	Non-IT/ICT projects*; Capital investment over £20 million
Australia	Capital investment over $50 million (≈£27.5 million)
China	Projects characterised with flexible price adjustment, high degree of market openness, high capital expenditure and stable demand

Note(s): ^*105 ICT projects experienced major cost overruns (an average of 30.5%), delays and terminations (Whitfield, 2007)

Table 5

Qualitative factors in VfM assessment

Countries	Timing	Factors
UK	Before quantitative assessment	Measurable objectives and outputs; risk allocation and management; operational flexibility; equity, efficiency and accountability; innovation; contract duration and residual value; incentives and monitoring; The Market; timescale; skills and resources
Australia	In conjunction with or after quantitative assessment	Service delivery and operational requirements; interface/relationship and project management; design considerations
China	After quantitative assessment	Life-cycle integration; risk identification and allocation; performance and innovation; competitiveness; governments' capabilities; financeability; asset correlation in the bundled contract

Note

1. The role of SOEs is also detected in Queensland and NSW, Australia (Queensland Government, 2015; The NSW Treasury, 2017). In Queensland, the application of the PPP policy is not mandatory for Government Owned Corporations, indicating an exempt from VfM assessment.