The analytical capacity of budgetary administrations: the case of the Euro area

Abstract

Purpose

The objective of this study is to analyze the role that budgetary analytical capacity (BAC) plays for fiscal discipline in a sample of Eurozone countries in the postcrisis period.

Dessign/methodology/approach

Building on the policy capacity literature, an index for the BAC is constructed by including OECD budgetary data from three dimensions: reliability of projections, openness to legislative scrutiny and transparency. The proposed BAC index is validated by checking that larger values of the index are associated with the higher fiscal discipline scores across countries.

Findings

Controlling for the economic cycle, BAC index is positively associated with fiscal discipline. The association is stronger for the index as a whole than for the three separate dimensions.

Research limitations/implications

The study is done on the limited sample of countries, and it is not feasible to validate results over time.

Practical implications

Budgetary policymakers can improve fiscal discipline by enhancing the three pillars that support the BAC.

Social implications

Stronger BAC can help to improve the quality of public decision-making and overcome political opportunism.

Originality/value

This is the first study that introduces the concept of BAC, makes it operational and suggests its relevance for supporting fiscal discipline.

Citation

Kasperskaya, Y. and Xifré, R. (2020), "The analytical capacity of budgetary administrations: the case of the Euro area", Journal of Public Budgeting, Accounting & Financial Management, Vol. 32 No. 3, pp. 379-398. https://doi.org/10.1108/JPBAFM-07-2019-0113

1. Introduction

In the aftermath of the Great Recession, the state of public finances and budgetary surveillance have risen to the top of policy agendas and interest in the determinants of fiscal discipline has been revived. In the specific case of the EU, several countries have recently implemented severe budget cutbacks in their efforts to escape sovereign defaults (Kickert and Randma-Liiv, 2015).

In parallel with these events, budgeting is becoming an increasingly complex process. The amount of information that has to be considered has increased, and growing uncertainties of all sorts demand an immediate response from what is still the key public policy instrument – the public budget. Accomplishing budgeting properly has come to depend more and more on the analytical capacity of budget decision-makers; however, their exercise of these skills can be facilitated or hampered by features of the budgetary process. How comprehensive, up-to-date and independent are a country's macroeconomic and fiscal projections? What quantity and quality of information is provided to the legislature for it to be able to scrutinize the executive? How transparent is the whole budgeting process? These questions point to some of the dimensions of the budget process that can enhance or impede the exercise of the analytical skills of the central government and other fiscal decision-makers.

Supranational organizations and international standard setters appear to be aware of this complexity and of the need to reform budgetary frameworks so that they can better face these challenges. The EU Council, for example, issued Directive 2011/85 highlighting the importance of sound macroeconomic and fiscal forecasts, transparency mechanisms and independent budgetary monitoring and analysis. Similarly, the OECD calls for the promotion of “the integrity and quality of budgetary forecasts, fiscal plans and budgetary implementation” (OECD, 2015, p. 10) and the IMF, in its updated version of the Fiscal Transparency Code, stresses the importance of “comprehensive, timely and credible projections of the evolution of the public finances” as a principle of prudent fiscal forecasting and budgeting (IMF, 2014, p. 2).

Most studies of the determinants of fiscal discipline have focused to date on two explanatory factors: statutory budget rules and macropolitical economy determinants. The former covers, for instance, the rules of budget negotiation, voting and approval and the functioning of central budget authorities, while the latter is concerned, among other issues, with the type of political system, executive and legislative fragmentation and the timing of political cycles. However, to the best of our knowledge, there has been no attempt to analyze the impact that the analytical capacities of budget decision-makers can have on fiscal discipline. This is the goal of the present paper, which undertakes an examination of these analytical capacities in a subset of 15 Euro-area countries for the period 2012–2016.

Building on the expanding literature on policy capacity (Weber and Khademian, 2008; Howlett, 2009; Wellstead et al., 2011; Howlett et al., 2014; Newman et al., 2017), this paper first proposes a conceptualization of budgetary analytical capacities (henceforth BACs). We advance the hypothesis that three properties of the budgetary setup – reliability of projections, openness to scrutiny and transparency – facilitate the exercise of the BACs of the government, legislature and the wider public. In line with the literature documenting the positive role played by policy capacity in ensuring correct decision-making in the public sector in general, here we argue that improving BACs should guarantee better management of information and improved decision-making in the public budget domain in particular. With respect to the earlier literature on policy capacity, a notable contribution of the current study is the empirical examination it undertakes across countries, that is, it does not limit itself to a single-country case study, which has been the predominant approach in this field to date.

Here, we construct an original BAC index, by developing the raw variables included in the 2012 OECD Survey of Budget Practices and Procedures (OECD, 2012), and we subject it to internal and external consistency tests. We show that our BAC index is positively and significantly correlated with a measure of fiscal discipline that controls for the state of the economy, for the 15 Euro-area countries that are also members of the OECD, for the period 2012–2016.

To demonstrate the robustness of our argument, we examine the possibility that other factors aside from the BACs might explain the variation in fiscal discipline in our sample. First, a quantitative analysis of the whole sample shows that the cyclically adjusted measure of fiscal discipline does not correlate with standard political economy variables concerning the composition of government. This is consistent with the previous literature, which reports that the usual variables of political economy and fiscal institutionalism do not perform well in explaining fiscal discipline (Kickert and Randma-Liiv, 2017). Second, we report a qualitative discussion of two extreme country cases, the Netherlands and Spain. The two cases are illustrative because they represent a verification of BAC logic at the limit: the Netherlands has the highest BAC score and the second highest measure of fiscal discipline, while Spain has the lowest BAC score and the second lowest measure of fiscal discipline.

The rest of the paper is organized as follows. Section 2 examines the literature on the political economy and fiscal institutionalism as determinants of fiscal responsibility. Section 3 presents the theoretical framework for policy capacity. Section 4 develops the notion of BACs, discusses the three variables (reliability of projections, openness to scrutiny and transparency) in the context of the literature and explains why they are expected to exert a positive effect on fiscal discipline. Section 5 confirms empirically this positive association, and section 6 briefly discusses the cases of the Netherlands and Spain and suggests ideas for future research.

2. Literature review: the determinants of fiscal discipline

According to the vision afforded by the political economy, countries with presidential systems tend to achieve better fiscal outcomes than those achieved by parliamentary systems (Cheibub, 2006), while coalition governments tend to report higher deficits than those reported by single-party administrations (Persson and Tabellini, 2003; Alesina et al., 2006). Budget discipline appears to weaken with the increased fragmentation of budgetary processes, both of the legislature (typically measured by the number of legislating parties) and of the executive (proxied with the number of spending ministries) (Perotti and Kontopoulos, 2002; Bawn and Rosenbluth, 2006; Wehner, 2010a). Fiscal discipline can also be influenced by the governing party's political orientation, with “conventional wisdom” suggesting that leftist governments tend to generate greater deficits than their rightist counterparts (Cusack, 1997; Tellier, 2006; García-Sánchez et al., 2014; Bojar, 2017). However, this has been openly questioned theoretically (Cusack, 1999), and it is not backed up empirically (Imbeau et al., 2001; Eslava, 2011; Armingeon, 2012; Hübscher, 2016). Indeed, an analysis of the fiscal consolidation episodes of 13 EU countries in the aftermath of the global recession fails to find support for the claim that the strength of governments or their political orientation has had a significant effect on the size and composition of fiscal consolidation (Kickert and Randma-Liiv, 2017).

Studies in fiscal institutionalism examine the impact that formal procedures and rules of budget negotiation, approval and voting have on fiscal discipline (Poterba and Hagen, 1999). Specifically, a finance minister exercising a strong role in budget negotiations, a hierarchical or top-down budget procedure with a central budget authority imposing budget ceilings and strict control on spending and a legislature's restricted amendment powers are considered to be conducive to fiscal discipline (Alesina and Perotti, 1999; Volkerink and Haan, 2001; Wehner, 2010a). However, in European countries, it is argued that the centralization or “delegation” of the budgetary power in the hands of the finance minister is not the only way to uphold fiscal discipline. Another approach conducive to fiscal discipline is the “contractual” one, where coalition parties agree and commit to numerical targets during budget negotiations (Hallerberg et al., 2009).

Finally, fiscal rules and independent fiscal institutions (IFIs) are two potential levers for achieving fiscal sustainability. Yet, relying solely on fiscal rules quickly became discredited (Wyplosz, 2012), and in the EU this was evident when the two leading countries in the bloc (Germany and France) repeatedly breached the Stability and Growth Pact (Andrle et al., 2015). In contrast, the rationale in favor of IFIs appears to be more convincing: IFIs constrain politicians' fiscal discretion (and associated deficit biases) while allowing for timely policy responses to unforeseen events (Beetsma and Debrun, 2018). Ongoing research into the role of independent fiscal watchdogs shows they contribute to producing more accurate forecasts and engender more transparent debates about public finances by creating or raising a reputation cost for noncomplying politicians (Debrun and Kinda, 2017; Alt and Dreyer Lassen, 2018; Beetsma et al., 2018). However, it is also clear that IFIs are extremely heterogeneous and differ greatly in size, remit, human resources and the circumstances in which they were created (von Trapp and Nicol, 2018). As such, it seems likely that the impact of IFIs on budgetary discipline varies across countries. We return to this issue later.

All in all, this literature has identified several mechanisms that appear to generate fiscal discipline. However, they only work well under certain conditions, for certain periods of time and in certain countries. This is the norm in social science phenomena, which are typically the consequence of multiple causes and therefore require multiple explanations. Here, we propose an additional factor that is both compatible and complementary with the lessons learnt from this literature and which should, we believe, help account for fiscal discipline. In this paper, we hypothesize that various properties of the budgetary framework facilitate the exercise of analytical, or technical, capabilities. We argue that these capabilities ensure a better management of information, resulting in improved collective decision-making in the public sector. In this respect, our hypothesis is in line with studies that stress the interrelated nature of elements that are conducive to fiscal responsibility: “Fiscal discipline does not emanate from institutions void of intentional decision-makers (…) fiscal discipline is produced by fiscally tight decision-makers, endowed with institutional power to act upon their preferences, and equipped with adequate knowledge of the fiscal consequences that follow from actions” (Helland, 2000, p. 107; the emphasis is ours).

3. Theoretical background: policy capacity in public management

Analytical capacity (Howlett, 2009; Howlett et al., 2014) is generally understood as one element under the umbrella concept of policy capacity (PC), which has received growing attention in the public management literature where it is treated both as an end in itself and as a means for governing better (Newman et al., 2017). For the purposes of the discussion that follows, it is worth considering two conceptualizations that the literature makes of PC.

First, PC is understood as an “enhancer” of the public sector insofar as it improves knowledge management and facilitates learning in complex environments, such as those typified by the public administration (Parsons, 2004; Weber and Khademian, 2008), and helps avoid the repetition of policy failures (Howlett et al., 2015). It means that governments with higher PCs are more efficient and better able to implement a given policy. As part of this broad definition of PC, some studies emphasize the importance of “pure” analytical PC – that is, the government's ability to appropriately access, interpret and transfer the necessary data and information for decision-making (Rasmussen, 1999; Oliphant and Howlett, 2010; Wellstead et al., 2011).

Second, some authors refer to PC as an expression of a country's governance capacity (Knill and Lehmkuhl, 2002), understood here as the state's “autonomy and capability to create its own desired policy objectives” (Newman et al., 2017, p. 160), that is, a more holistic, proactive vision of PC. As such, it expands beyond the limits of the central government and incorporates the ability of all relevant fiscal decision-makers (including subcentral governments and the legislative organ) to make “informed choices” (Bakvis, 2000) about conflicting policies or public programs on behalf of citizens. Thus, higher capacity administrations make better use of the information to design and choose the right policies.

Ultimately, the differences between conceptualizations are more terminological than substantive (Newman et al., 2017) so that PC is best understood as the ability to improve the quality of public decision-making by supporting the adoption of effective policies (Goetz and Wollmann, 2001; Wellstead et al., 2011). Analytical PC, on the other hand, is more closely associated with evidence-based policy formulation and is one of the main tools employed by policy workers (Howlett, 2009; Howlett et al., 2014).

Most studies conclude that well-honed analytical PCs result in improved public decision-making and high-performing governments (Cohen, 1995; Bakvis, 2000; Harrow, 2001; Painter and Pierre, 2005; Tiernan, 2011). Given this background, this paper seeks to determine whether enhanced analytical capacity can have a positive impact on the budgetary setup and contribute to fiscal responsibility.

4. Budgetary analytical capacities

We define BACs as the set of procedures, mechanisms and institutions required to improve PC in the budgetary decision-making domain. Specifically, we focus on those capacities concerning the access, processing and transmission of budget information that contribute to good government performance. Thus, we propose measuring BACs in relation to three dimensions or principles of the budgetary setup: reliability of projections, openness to scrutiny and transparency.

Each of these dimensions plays an important role in the budget cycle in which the management of information is a critical function: making projections that determine the budget aggregates; the scrutiny conditions established for the legislative process prior to budget approval; and the effective communication of budget information to the wider public. Thus, each principle is of relevance for capturing the analytical capacity of the respective policy agents in the budgetary process: that is, central government, the legislature and the wider public (citizens, investors, supranational organizations, think tanks, etc.).

In terms of the primary data we use to construct the BAC index, 46 individual variables are drawn from the OECD survey (OECD, 2012) in relation to the three dimensions: 18 concerning reliability and 14 related to both scrutiny and transparency. These were then aggregated into seven high-level variables, that is, three for reliability (variables A1, A2, A3), three for scrutiny (variables B1, B2, B3) and one for transparency (variable C), all of which were assigned the same weight. The 46 variables and their organization into high-level variables are illustrated in Appendix 1.

To better appreciate the policy relevance of the BAC principles, we show how they are embodied in a variety of regulations and policy recommendations. Table 1 summarizes how the three principles capture both (1) the main themes in the literature on PC (see previous section) and (2) the interest of politicians, policymakers and practitioners.

4.1 Reliability

In the case of the reliability of fiscal and macroeconomic projections, we identify three dimensions in which this can be enacted: time coverage and frequency of revision of projections; comprehensiveness (number of items included) of projections; and degree of independence of body primarily responsible for preparing projections.

Long-term coverage (periods of 10, 30 or 50 years) of macroeconomic fiscal projections coupled with frequent revisions (variable A1) is desirable since most developed countries face significant long-run uncertainties. Thus, long-term projections provide critical orientation for the major entitlement spending programs, including health, education and pensions, and they inform policy debate. The regular consideration of such factors as GDP growth rate, unemployment trends, short- and long-term interest rates on government debt and so on, when preparing the macroeconomic and fiscal scenarios (A2), provides the basis for more grounded and realistic inputs to budgetary analysis.

Finally, the independence of the institutions (A3) responsible for preparing the macroeconomic and long-term fiscal projections is critical for enhancing analytical capacity. However, this independence must be actual, and so, we only consider the independence criterion to be met when the body that is primarily responsible for the development of projections is independent. The 2013 Treaty on Stability, Coordination and Governance (the Fiscal Compact) required Euro-area countries to establish their own IFIs, a requirement that has been met. However, as mentioned in section 2, IFIs differ greatly in their remit. Indeed, most have only an indirect influence on fiscal discipline by producing or endorsing forecasts and providing opinions on budgets (Horvath, 2018). In contrast, some of the more longstanding IFIs – for example, the Dutch CPB – have a well-established tradition of leading the preparation of fiscal and macroeconomic projections.

In the case of the policy relevance of the reliability principle, EU Council Directive 2011/85/EU on Budgetary Frameworks requires that the public accounting system be “comprehensive” (Art. 3.1) and that fiscal planning be based on “realistic macroeconomic and budgetary forecasts using the most up-to-date information” (Art. 4.1). The adoption of a long planning horizon is required as a source of credibility for the whole budgetary framework (Art. 9.1). Finally, both the Directive and the aforementioned Fiscal Compact emphasize the necessary independence of the fiscal bodies to ensure the effective monitoring of countries' compliance with fiscal rules and that fiscal discipline is maintained.

4.2 Scrutiny

Openness to scrutiny means providing the legislature (parliament) with the necessary resources, information and time to perform an effective financial scrutiny of the budget. It is measured using three indicators: comprehensiveness of information available to the legislature (B1); whether the legislature can rely on a specialized budget research office to undertake budget analyses (B2); and the amount of time available to the legislature for deliberating and debating the budget (B3). These indicators are based on the standards for legislative financial scrutiny promoted by the IMF and the OECD (see Wehner, 2017, for an overview). According to which, the legislature has the important fiscal mission of holding governments accountable through parliamentary scrutiny. To be able to do so effectively, the legislature needs access to relevant budgetary information, including a comprehensive annual budget covering all government sectors, medium-term budgets, off-budget expenses and extra budgetary funds, among others (B1). Likewise, for scrutiny to be effective, the legislature needs sufficient time to examine the budget drafts and to deliberate (B2), three months before the start of the financial year being the minimum benchmark. Finally, a specialized budget research office attached to the legislature (B3) can strengthen its capacity by providing technical expertise on budgeting matters, something legislators often lack (Santiso and Varea, 2013).

Previous studies have examined the legal budgetary power of the legislature vis-à-vis that of the executive, including the former's options for making amendments. Their findings regarding the relationship between the formal (legislative) powers of the legislature and fiscal outcomes are somewhat conflicting (Alesina and Perotti, 1999; Lienert, 2005; Hallerberg et al., 2009; Wehner, 2010b). Germany provides a good example of a fiscally active and prudent legislature, the Bundestag typically introducing many amendments to executive budget proposals each year, very often reducing expenditures (Wehner, 2017, p. 8).

Amendment power (AP) is, in fact, more of an executive function than an analytical function of parliaments, which is what the BAC seeks to capture. A similar distinction can be made for the timing of events, which differs for the exercise of analytical functions and AP. Typically, parliaments will first exercise their analytical capacity and only then – on the basis of the diagnoses made – will they exercise, where applicable, their AP. In contrast to the other index variables, the relationship between this variable and analytical capacity is not conceptually clear. To the best of our knowledge, there is no theoretical support for the claim that more (or less) AP is likely to result in enhanced analytical capacity and, hence, better fiscal responsibility or discipline. Finally, the empirical relationship between AP and fiscal discipline is unclear in our sample. According to the OECD survey, of the 15 countries, seven record a maximum score in relation to formal AP. Among these seven, full AP is associated with both increases and reductions in total expenditure in 2012: four (Austria, Germany, Luxembourg, Slovak Republic) used their unrestricted AP to cut spending, while three (Belgium, Finland and Portugal) used their equally unrestricted AP to expand public spending.

Here, we focus not so much on how the legislature shares power with the executive but rather on what might be considered the predeterminants of the scrutiny principle: that is, what conditions (amount of information supplied, time provided to analyze it and availability of specialist support) sustain the legislative scrutiny capacity of the executive in the budgeting process. In this respect, our indicator of scrutiny resembles in part a legislature's organizational capacity (Wehner, 2006).

The role, conditions and impact of legislative scrutiny capacity on fiscal discipline is of obvious interest for policymakers and practitioners. In 2010, the IMF's Fiscal Affairs Department published guidelines emphasizing that national budgetary frameworks should be set up in such a way to facilitate such scrutiny (Lienert, 2010). Similarly, in 2013, the Association of Chartered Certified Accountants testified before the European Parliament that a lack of information and the legislature's deficient scrutiny capacity are both symptoms and causes that “[parliamentary] financial scrutiny may not be taken seriously enough or indeed seen as strategically important” (ACCA, 2011, p. 26). However, despite its interest, to the best of our knowledge, the impact of the legislature's BAC on a country's fiscal discipline has yet to be examined empirically.

4.3 Transparency

Finally, transparency – that is, how timely and fully budgets are disclosed to all stakeholders in the budgetary process, including citizens, supranational organizations, think tanks and investors – is captured by means of high-level variable C. This principle can be considered a necessary condition for the other two principles to hold. Budget transparency is essential in ensuring government accountability and scrutiny as it prevents politicians from acting opportunistically, generating fiscal deficits and overspending (Alesina and Perotti, 1999; Eslava, 2011). Several empirical studies document that a higher degree of budgetary transparency is conducive to lower levels of public debt and deficits (Alt and Lassen, 2006; Benito and Bastida, 2009; Wyplosz, 2012).

The policy relevance of transparency is made clear in the EU Budgetary Directive 2011/85, where it states that the principle should not only be seen as the making public of official forecasts, but also the requirement that Member States make explicit “the assumptions underpinning the preparation of macroeconomic and budgetary forecasts” (Art. 5).

5. Construction and quantitative analysis of the BAC index

The data employed herein are drawn from three sources. Budgetary data have been obtained from the 2012 OECD International database of Budget Practices and Procedures, a survey conducted periodically (previous editions correspond to 2003 and 2007) by its Budgeting and Public Expenditures Division. Survey data, which are publicly available, are both comprehensive and reputable and have been employed previously in cross-country budgeting studies (Alt and Lassen, 2006; Wehner, 2006; Benito and Bastida, 2009). Individual researchers are limited in their assessment of the consistency of responses, but the OECD conducts an exhaustive quality control of responses to ensure its reliability and veracity. However, the 2012 version was modified by the OECD and differs from the surveys conducted in 2003 and 2007, thus limiting comparability of responses. Indeed, the index we develop here cannot be replicated directly for editions prior to 2012.

Economic and fiscal data have been obtained from the IMF (WEO database, October 2019), while political economy data have been obtained from the Comparative Political Data Set (CPDS 1960–2017 database).

5.1 Construction of the index

Our sample includes a total of 15 Euro-area countries that are also members of the OECD for the period 2012–2016 (see Figure 1). There are two reasons for imposing this double restriction. First, Euro-area countries can be assumed to operate under relatively similar broad macroeconomic policy guidelines. They are deprived of an autonomous monetary policy and their fiscal and budgetary behavior is under EU surveillance, Council Directive 2011/85/EU being an eloquent illustration of this. Second, as the source of our budget procedure data is the OECD's survey (OECD, 2012), we can only select countries included within it. Survey responses were originally collected between November 2012 and February 2013 and verified in June 2013.

For each of the three dimensions of the BAC index, we construct an individual index: RELIAB, SCRUT and TRANSP, respectively. As explained earlier, each individual index incorporates several high-level variables computed from the OECD's raw survey data. Specifically, RELIAB = 1/3*(A1+A2+A3), SCRUT = 1/3*(B1+B2+B2) and TRANSP = C (see the Appendix for details). All the high-level variables and three dimensions have been normalized in the [0,1] scale (in line with Alesina et al., 1999). Table 2 reports the descriptive statistics of the three dimensions.

Finally, the BAC index is defined as the sum of the three individual indexes, that is, BAC = RELIAB + SCRUT + TRANSP, and is expressed on the [0,3] scale. Figure 1 shows the value of the BAC index for the 15 countries considered in our sample as well as the contributions of the three dimensions.

An analysis of the factors underpinning the BAC scores for each country is clearly unrealistic here, not least because of limitations of space and so we confine our remarks to just two issues: the time period and the method employed in constructing the index.

First, we examine the extent to which the period we study might be considered atypical. It could be argued that countries heavily affected by the 2008 crisis changed their budget procedures in its aftermath so as to improve their fiscal performance, while the rest of countries did not. If this were the case, direct comparisons across all the countries in our sample could not be made, as some would be reacting to an out-of-sample event. For instance, it is legitimate to question whether the score recorded by Portugal (a country badly hit by the crisis but with a relatively good BAC score in 2012) would be maintained if it had been computed with data from the 2007 OECD budget survey. A similar concern could apply to Ireland.

Unfortunately, BAC scores cannot be computed from 2007 OECD data because, as discussed, the majority of questions on the survey were changed for the 2012 edition. If we only consider the four questions that are common to both surveys, the ranking of three countries affected by the crisis (Portugal, Ireland and Spain) remains virtually the same, while only one country (Greece) was ranked more highly in 2012 than in 2007 [1].

Second, it might be argued that factor analysis could be employed to reduce the dimensionality of the 46 variables and, in this way, obtain a number of relevant factors. However, there are both methodological and conceptual limitations to using factor analysis in this setup. Methodologically speaking, conventional factor analysis is not appropriate for analyzing binary variables (Bartholomew et al., 2008) and here most of our 46 variables are binary. It might be further argued that factor analysis could be applied to our second-level continuous variables (A1, A2, A3, B1, B2, B3, C). However, in conceptual terms, we have adopted a different approach as we derive our composite indicator as a nested structural model. The construction of indicators with a nested structure typically requires a prior conceptual framework (see previous section) to guide the process (Nardo et al., 2005). While factor analysis applied to these seven variables could result in three factors, each factor would be constructed by different combinations of the seven second-level variables, which would hinder interpretation of the factors from a conceptual point of view. When a conceptual framework is employed to determine the structure of a structural indicator, its consistency can be tested externally by examining its relationship with the phenomenon of interest (Wehner, 2006; 2010b).

Here, to check for the consistency of the BAC index, we employ a two-step methodology. First, we analyze the internal consistency of the index by employing some of the most frequently used tests for indexes in this field. Second, we exploit the well-known negative relationship between GDP growth and the debt-to-GDP ratio (Checherita-Westphal and Rother, 2012; Panizza and Presbitero, 2014) to obtain an estimate of the countries' fiscal discipline. For the period 2012–2016, we regress the variation in the debt-to-GDP ratio against the average GDP growth rate and we obtain the residuals of the regression. These residuals serve as a measure of the fiscal discipline of the country compared to that of the other peer countries after controlling for the cyclical position of the economy. Then, we determine the correlation between this measure of fiscal discipline and the BAC index. To ensure this result is not driven by political variables, these residuals are also correlated with conventional political-governmental variables such the parliamentary majority of the government or the ideological orientation of the government.

5.2 Internal consistency

Constructing an index to undertake cross-national comparisons requires meeting a minimum set of consistency conditions (Wehner, 2010b). In our setup, we need to check (1) that the three individual indexes (RELIAB, SCRUT and TRANSP) do not significantly correlate with each other and (2) that the aggregate index is robust to different forms of aggregation other than that of the sum.

To check the first condition, we compute the bivariate correlation coefficients between the three variables. For the sake of completeness, we test the relationship by means of both the Pearson and Spearman correlation coefficients. The former is more appropriate for testing the linear relationship between two continuous variables while Spearman's rank-order correlation evaluates the relationship between two (not necessarily continuous) variables. The results are reported in Table 3. None of the correlation coefficients are significant.

In order to check the second condition, we construct an alternative index with the geometric mean, BAC´ = (C*S*T)^1/3. The correlation between BAC´ and BAC indexes is very high and significant at the 1% level (Pearson = 0.98; Spearman = 0.96), indicating that the BAC index is robust to alternative aggregation methods.

5.3 External consistency

We characterize a country's fiscal trajectory in terms of the variation in public debt as a percentage of GDP. This variable captures all liabilities that require payment by the sovereign state. This can be considered the key stock variable since it captures the state of public finances and is widely considered the most comprehensive measure of both fiscal performance and discipline (Alesina and Perotti, 1999; Ostry et al., 2015).

The fiscal position of an economy is strongly influenced by GDP growth, and in this respect, the relationship between GDP growth and the debt-to-GDP ratio has been widely examined. While the causal impact of debt on growth remains an open, controversial issue (Reinhart et al., 2012; Herndon et al., 2014), it is widely accepted that higher economic growth reduces debt (Panizza and Presbitero, 2014). In the context of the EU and the Euro-area countries, the relationship is likely to follow a negative linear pattern (Checherita-Westphal and Rother, 2012).

Building on this, we first perform an OLS regression with the change in the debt-to-GDP ratio between 2012 and 2016 as an independent variable (variable VARDEBT) and the average GDP growth rate over the same period (variable VARGDP) as an explanatory variable (model 1). Our goal is to retrieve the residuals of this regression (variable RESID1) as a measure of the countries' fiscal performance. To check the robustness of our approach, we also explore an auxiliary specification, which includes the level of the debt-to-GDP ratio in 2012 (variable DEBT12) as an additional explanatory variable (model 2) generating different residuals (RESID2). The rationale for the auxiliary specification is that it might be the case that the relationship between the variation in debt and GDP growth depends on the initial level of debt. The results of both estimations are reported in Table 4.

In our two specifications, the coefficients are highly significant and present the expected sign in line with theory. The fact that including the debt level as a check in model 2 results in a similar coefficient for the impact of growth on debt suggests that the estimation of the linear relationship in both specifications is relatively robust. Additionally, the residuals in both models do not correlate with the explanatory variables. Our linear models indicate that a 1 percentage point increase in average GDP growth between 2012 and 2016 is associated with a reduction of approximately 6.5–7 percentage points in the debt-to-GDP ratio over the same period. Both models are able to explain a relatively large part of the variation in our sample (73 and 79%, respectively).

For the sake of completeness, we examine the correlations of the BAC index with both residuals, RESID1 and RESID2. The residuals of both models are highly correlated (correlation coefficient of 0.87, which is significant at the 1% level). Both residuals are represented in Figure 2.

The residuals can be interpreted as follows. When a country presents a negative residual, this means its debt-to-GDP ratio has increased less than the value the linear model predicts for the whole sample and, therefore, it has had a fiscal performance above the norm. Conversely, when a country presents a positive residual, this means that its debt-to-GDP ratio has increased more than the value the linear model predicts and, therefore, it has had a fiscal performance below the norm. Negative residuals are in principle a manifestation of (relative) fiscal discipline, and positive residuals are a manifestation of (relative) fiscal indiscipline after controlling for changes in the conditions of the economy.

This control for the state of the economy is vital in our exercise as we need to correlate the BAC index with a measure of fiscal discipline that is not strongly affected by the working of the economy's automatic stabilizers, which have played a relevant role in the crisis and postcrisis periods (Bailey et al., 2014). As the BAC index captures the ability of the government and legislature to manage information adequately, what we seek is to correlate the BAC index with a measure of government efficiency that is independent of the effect the economic cycle has on welfare payments and tax revenues. For this reason, we employ the aforementioned method to filter out as much noise as possible from nonbudgetary factors before measuring the correlation between the budgetary setup and fiscal responsibility.

We turn now to the analysis of the correlation between the BAC index and its three dimensions and the residuals of both models. In principle, for the reasons outlined in section 4, we expect the relationship between these two sets of variables to be negative. Table 5 reports the results.

As predicted by theory, all correlation coefficients present a negative sign. For both models, the correlation between the residuals and the aggregate index of BAC presents a higher absolute value than for any of the index's three dimensions. Indeed, the correlations between the overall BAC index and the residuals of both models is highly statistically significant (beyond the 1% level) in both models: −0.67 in model 1 and −0.79 in model 2. In model 1, the individual dimensions are not significantly correlated with the residual, whereas in model 2, the reliability and transparency dimensions are significantly correlated with the residual. Kendall's Tau test confirms the significance of the aforementioned correlations. The analysis therefore suggests that the three dimensions are strong complements for capturing the analytical capacity of budgetary administrations.

Finally, to rule out the hypothesis that fiscal discipline is primarily driven by political economy determinants, we compute the correlation between three conventional political-governmental variables, on the one hand, and the two measures of fiscal discipline, on the other. We employ the following three variables from the Comparative Political Data Set: seat share, as a percentage of the total number of parliamentary seats, of the parties in government (variable GOV_SUP); cabinet posts held by social democratic and other left-wing parties as a percentage of the total number of cabinet posts (GOV_LEFT1); and cabinet posts held by right-wing parties as a percentage of the total number of cabinet posts (GOV_RIGHT1). All variables are weighted by the number of days in office in a given year and averaged over the time period under study (2012–2016). These variables are reported in Appendix 2. Table 6 reports the results of the analysis. Correlations between political variables and fiscal discipline measures are very low and none of them is statistically significant.

6. Discussion and conclusions

For reasons of space, it is not feasible to discuss in detail all the countries included in the analysis, and so, for the sake of illustrative purposes and clarity of exposition, we have opted to focus on two extreme cases.

The Netherlands stands out as the country with the highest BAC score and with the second lowest residuals, a sign of good fiscal discipline. At the other extreme, Spain is the country with the lowest BAC score and with the second largest residuals, indicating a low level of fiscal discipline relative to that of the other countries in the sample. At the two extremes, therefore, the results of the quantitative analysis are verified: that is, better analytical capacities are associated with higher fiscal discipline. However, we need to consider whether other factors, aside from the BACs, might explain the levels of fiscal discipline in the two countries.

Both countries are parliamentary constitutional monarchies, although Spain is constitutionally more complex than the Netherlands. However, if we look beyond this similarity, recent developments (2012–2016) in the two countries, as seen through the political economy lens, point to quite different sets of circumstances (for a richer description of their respective experiences, see Kickert and Ysa, 2014; Kickert and Randma-Liiv, 2015, 2018).

In the Netherlands, in April 2012, the Party for Freedom (PVV) withdrew its support for Mark Rutte's first cabinet (a minority government in office since October 2010) because of, among other reasons, public disagreements about the 2013 general budget. A political crisis ensued that led to elections; after which, it took 50 days to negotiate the formation of a new government, which remained in office until 2017. This cabinet, Rutte's second, was supported by the grand (or “purple”) coalition between Rutte's Conservative party (VVD) and the Labour party (PvdA). In this cabinet, key positions – including Deputy Prime Minister and Minister of Social Affairs and Employment and the Minister of Finance – were held by members of the Labour party. From the perspective of the political economy literature, a coalition government of heavy-weight, traditional parties of different ideologies in a parliamentary (i.e. nonpresidential) system does not, in principle, constitute the best scenario for achieving budget discipline (see Kickert and Randma-Liiv, 2018 for more details).

The political landscape in Spain in this period was apparently more stable. Amid severe criticisms for how the Socialist government (PSOE) had managed the economic crisis, the Conservative party (PP), led by Mariano Rajoy, won the general elections in November 2011, gaining absolute majorities in both Houses of Parliament. The new single-party government held office until January 2016 (staying on as acting government until June 2016) and was able to pass two high-profile fiscal reforms that had long been recommended by the EU, the IMF and the OECD. In April 2012, very shortly after coming into office, the Rajoy government passed the new law on budgetary stability and financial sustainability (Law 2/2012). This was an organic law (i.e. binding on all levels of government and public administrations in Spain), modeled according to the EU's Stability and Growth Pact. In November 2013, the first Spanish IFI was created, the AIReF (see Kickert and Ysa, 2014 for more details and Kasperskaya and Xifré, 2019 for an anlysis of the two fiscal reforms). Despite this favorable political economy environment, the level of fiscal discipline in Spain in the period 2012–2016 after adjusting for the cycle was among the lowest in the EU.

The differences between the two countries' IFIs are instructive in this regard. The Dutch IFI, the CPB, is widely considered a poster child for independent service, a comprehensive remit, an active role in drafting macroeconomic and fiscal projections and a capacity for scrutiny (Horvath, 2018). This is captured, in part, by our BAC index with the Netherlands being awarded the highest scores in the sample for both reliability and scrutiny. In contrast, the new Spanish IFI, the AIReF, despite having been created in line with international blueprints and despite the obvious commitment of its staff members to perform efficiently, has a much more modest remit. Indeed, it plays no primary role in drafting official projections, and initially, it even faced difficulties in obtaining the necessary information from the government (Kasperskaya and Xifré, 2019).

Based on this overview of recent developments in the two countries, it seems difficult to claim that the typical variables identified in the political economy and fiscal institutionalism literature are able, on their own, to account for the patterns of fiscal discipline presented by the Netherlands and Spain. It is in this respect that this paper suggests the need to include BACs among the elements that might explain fiscal discipline and, by doing so, enrich our understanding of the phenomenon.

Future research could usefully explore two issues at least. First, here, we have assumed for the sake of simplicity that the three elements comprising BAC (i.e. reliability, scrutiny and transparency) carry the same weight. It would be interesting to examine whether this assumption holds true in general or, alternatively, whether in certain circumstances it might be more appropriate to give greater weight to one or other of the dimensions when computing the index. Second, our measure of BAC is static for a set of countries. It would be interesting to develop a dynamic perspective of BAC for a given country to determine whether (as predicted by theory) fiscal discipline improves as the country's analytical capacity becomes more honed.

Figures

Figure 1

The value of the budgetary analytical capacity (BAC) index and its components

Figure 2

The residuals of the debt–GDP growth regression under both models

Table 3

Correlation between individual dimensions of the BAC index

	Pearson correlation				Spearman correlation
	RELIAB	SCRUT	TRANSP		RELIAB	SCRUT	TRANSP
RELIAB	1			RELIAB	1
SCRUT	−0.174 [0.534]	1		SCRUT	−0.052 [0.854]	1
TRANSP	0.246 [0.376]	0.118 [0.676]	1	TRANSP	0.345 [0.208]	0.218 [0.435]	1

Note(s): None of the correlations are significant at the 10% level. p-value in brackets

Table 4

Regression coefficients for the Debt–GDP growth

Model						R2
(1)	VARDEBT =	constant + 10.109*** (2.660)	VARgDP + −6.418*** (1.074)		RESID1	0.733
(2)	VARDEBT =	constant + 19.430*** (5.316)	VARgDP + −6.927*** (1.006)	DEBT12 + −0.106* (0.054)	RESID2	0.798

Note(s): *** and * denote significance at the 1 and 10% levels respectively. OLS estimates with standard errors in parentheses

Table 5

Correlation coefficients between budgetary variables and fiscal performance

	RESID1	RESID2
RELIAB	−0.383 [0.159]	−0.468* [0.079]
SCRUT	−0.427 [0.112]	−0.342 [0.213]
TRANSP	−0.416 [0.123]	−0.659*** [0.008]
BAC	−0.672*** [0.006]	−0.793*** [0.000]

Note(s): *** and * denote significance at the 1 and 10% levels respectively. p-value in brackets

Table 6

Correlation coefficients between political variables and fiscal performance

	RESID1	RESID2
GOV_SUP	−0.142 [0.613]	−0.205 [0.464]
GOV_LEFT1	0.320 [0.245]	0.290 [0.294]
GOV_RIGHT1	0.021 [0.940]	−0.153 [0.586]

Note(s): p-value in brackets

Table A1

Reliability - RELIAB

A. Comprehensiveness of the fiscal and macroeconomic projections. Includes three dimensions A1. Time coverage and frequency of revision of fiscal and macroeconomic projections A2. Comprehensiveness of long-term fiscal projections A3. Institutions in charge of developing projections From 0 to 1. Composite indicator (self-production), each of the dimensions is given equal weightings

A1 Includes time coverage and frequency of revision of fiscal and macroeconomic projections From 0 to 1 in proportion to the maximum points that can be attained (12)

A1.1 How many years do the long-term macroeconomic estimates cover? (Q.13) (1 = up to 5 years; 2 = between 6 and 10 years; 3 = between 11 and 30 years: 4 = between 31 and 50 years)

A1.2 How often are long-term fiscal projections generally revised/updated by this institution? (Q.14B) (1 = over 3 years; 2 = every 3 years; 3 = every 2 years; 4 = every year)

A1.3 How many years do the long-term fiscal projections span? (Q.14C) (1 = up to 5 years; 2 = between 6 and 10 years; 3 = between 11 and 30 years: 4 = between 31 and 50 years)

A2 Items regularly considered/included in the long-term fiscal projections underlying the budget? (Q.16) Economic growth rate (1), unemployment trends (2), short-term interest rates on government debt (3), long-term interest rates on government debt (4), exchange rate (5), fiscal gap projections (6), effects of significant policy reforms (e.g. pensions, health) (7), immigration/emigration flows (8), demographic changes (excluding migration) – aging, population growth (9), healthcare costs (10), intergenerational accounting (11), civil servants' pensions obligations (12), unfunded pension liabilities (13) From 0 to 1 in proportion to maximum points that can be attained (13)

A3 Institutions in charge of development of the macroeconomic estimates and long-term fiscal projections From 0 to 1 with respect to maximum points that can be attained (2)

A3.1 Body that is primarily responsible for the development of the macroeconomic estimates underlying the budget projections (Q.12) (0 = CBA or other governmental body; 1 = body that is independent from executive and legislature)

A 3.2. Body that is primarily responsible for the development of the long-term fiscal projections. (Q14A) (0 = CBA or other governmental body; 1 = body that is independent from executive and legislature)

Table A2

Scrutiny - SCRUT

B. Legislative scrutiny. Includes three dimensions B.1. Total budgetary information presented to legislature B.2. Existence of a specialized budget research office/unit attached to the legislature to conduct analyses of the budget B.3 Time available for the legislative debate of budget From 0 to 1. Composite indicator (designed by authors), each of the dimensions are given equal weightings

B.1 Total budgetary information presented to legislature (Q.61) Includes responses about the following types of budgetary information: Fiscal policy objectives for the medium term (1), macroeconomic assumptions (2), budget priorities (3), comprehensive annual financial plan encompassing all revenues and expenditures including off-budget expenditures and extra-budgetary funds (4), comprehensive annual financial plan encompassing all revenues and expenditures for all levels of Government (including regional and local) (5), comprehensive table of tax expenditures (exemptions, deductions and credits) (6), nonfinancial performance targets for programs and/or agencies (7), medium-term perspective on total revenue and expenditure (possibly in the form of a medium-term expenditure framework) (8), long-term perspective on total revenue and expenditure (ten or more years) (9), clearly defined appropriations to be voted by the legislature (10), linkage of appropriations to administrative units (e.g. ministry, agency) (11), text of legislation for policies proposed in the budget (12) From 0 to 1% of the 12 items available

B.2 Existence of a specialized budget research office/unit attached to the legislature to conduct analyses of the budget (Q.60a) 1 – yes, 0 – no

B.3 Time available for the legislative debate of budget (based on Q.6) From 0 to 1% of the maximum time available (4.5 months)

Table A3

Transparency – TRANSP

C. Transparency. Total budgetary information made publicly available Includes responses about the following types of budgetary information: methodology and assumptions for establishing fiscal projections used in the budget (1), sensitivity analyses of fiscal and/or macroeconomic models (2), independent reviews/analyses of macroeconomic and/or fiscal assumptions (3), budget Circular (4), prebudget report to the legislature (5), executive budget proposal submitted to the legislature (6), fiscal policy objectives for the medium term (7), comprehensive annual financial plan encompassing all revenues and expenditures including off-budget expenditures and extra-budgetary funds (8), comprehensive annual financial plan encompassing all revenues and expenditures for all levels of government (including regional and local) (9), medium-term perspective on total revenue and expenditure (possibly in the form of a medium-term expenditure framework (10), long term perspective on total revenue and expenditure (ten or more years) (11), citizens’ budget (12), citizens’ budget guide (explaining the budget process and actors involved) (13), budget approved by the legislature (14) From 0 to 1% of 14 items available (14)

Table A4

Political economy variables. Country averages for the period 2012–2016

	Government support	Left-wing parties in government	Right-wing parties in government
Austria	56.2	50.0	0.0
Belgium	60.8	21.3	50.0
Estonia	56.3	18.3	79.5
Finland	60.4	33.8	48.8
France	52.0	92.6	7.1
Germany	69.5	22.8	12.2
Greece	54.2	46.6	43.5
Ireland	63.3	29.0	0.0
Italy	41.4	47.1	5.9
Luxembourg	57.9	53.0	23.9
Netherlands	49.6	38.4	53.2
Portugal	52.8	12.6	57.2
Slovakia	55.1	88.7	7.7
Slovenia	55.7	58.4	38.2
Spain	52.7	0.0	0.0
Source(s): Comparative Political Data Set (CPDS 1960–2017 database)

Note

1. A separate note detailing these calculations is available from the authors upon request.

Appendix 1 Definition of the budget variables

Table 1

Relationship between the three budgetary analytical capacity (BAC) principles and academic conceptualizations of policy capacity. Examples of policy initiatives and recommendations that refer to the BAC principles

		Reliability	Scrutiny	Transparency
Conceptualizations of policy capacity (PC)	PC to manage knowledge and foster organizational learning (Parsons, 2004; Weber and Khademian, 2008; Wellstead et al., 2011; Howlett et al., 2014; Newman et al., 2017)	✓	✓	✓
	PC for effective policy formulation (Goetz and Wollmann, 2001) and to avoid repeating policy failures (Howlett et al., 2015)	✓	✓	✓
	PC to improve a country's governance capacity (Bakvis, 2000; Knill and Lehmkuhl, 2002; Wellstead et al., 2011)		✓	✓
	PC as central government's ability to access, transfer and process information (Rasmussen, 1999; Oliphant and Howlett, 2010; Newman et al., 2017)	✓
	Analytical PC (Howlett, 2009; Oliphant and Howlett, 2010; Wellstead et al., 2011; Howlett et al., 2014; Newman et al., 2017)	✓		✓
Agent in the budgetary framework to whose analytical capacity the BAC principle mainly refers		Government	Legislature	Wider public
Examples of policy initiatives and recommendations that refer to the BAC principles		EU Directive 2011/85; EU Fiscal Compact (2013)	IMF Manual (Lienert, 2010); Association of Chartered Certified Accountants (ACCA, 2011)	EU Directive 2011/85

Table 2

Descriptive statistics of the three budget principles

	Minimum	Maximum	Mean	Standard deviation
RELIAB	0.32	0.82	0.52	0.15
SCRUT	0.34	0.79	0.56	0.16
TRANSP	0.36	0.79	0.60	0.14

Appendix 2