Search results

The purpose of this paper is to explore, in literature and practice, the use of ex post impact evaluations within the public–private partnership (PPP) context and understand the major considerations for developing a PPP ex post impact evaluation method.

This paper applies exploratory research through expert interviews from Chile and Australia to describe: the relevance of this kind of evaluation, and the challenges of designing and conducting a PPP ex post evaluation.

The study confirms the lack of a formal method for evaluating the impact of a PPP project. Experts agree on the relevance of performing ex post evaluations to PPP projects and that in practice there is no formal procedure to follow. Among other challenges, experts discussed four general ex post evaluation aims: transparency and accountability, PPP legitimization, industry learning and government agency learning.

This study confirms the gap in knowledge and contributes to the developments of approaches to perform ex post impact evaluation of PPP projects. It also provides several suggestions that need to be addressed when attempting to evaluate PPPs beyond the financial and contractual parameters.

The topic is not fully addressed in the literature, and this study contributes to the initial discussion and development of this evaluation method, which is considered significant for the development of public infrastructure.

A study of the current practices for evaluating the ex-post performance of public-private partnership (PPP) school projects in Australia via literature review and qualitative case studies has found that no consistent approach exists for evaluating operational performance. A detailed critique of international PPP audits and practices has identified existing gaps in ex-post performance evaluation. Through a process of comparative analysis and industry confirmation, a performance analysis technique aligned with international practice has been developed that can be utilised by the educational departments across Australia to evaluate the ex-post performance of PPP projects (PPPs). The paper aims to discuss these issues.

This paper opted for qualitative archival analysis of case studies using pattern matching, explanation building, time series, and comparative analysis. The data used for document analysis included value reports, project summaries, and contract documents, as well as local and international audit guidelines.

This paper reviewed current practices, identified a range of processes, and reported the best practices. However, consideration of the approaches taken in the UK and Australia for evaluating operational performance indicates that current techniques lack consistency.

The developed ex-post performance measurement framework is limited to Australian PPP school projects and, at this stage, cannot be generalised to other social PPP projects.

The paper includes implications for the development of better performance evaluation practices and audits.

An enhanced framework for measuring operational performance will increase the accountability of taxpayers in the content of their utilisation by the government.

This paper presents an enhanced ex-post performance measurement framework for education departments across Australia.

This article explores the applicability of participatory action research (PAR) on two levels: on the one hand, as a participatory evaluation method for community engagement and community development; on the other hand, as a tool to link ex ante and ex-post evaluation that situated at various stages of the policy cycle.

Through a practice-based case study of the community engagement process of Valletta Design Cluster, this paper aims to illustrate how PAR can offer collaborative and continuous evaluation by facilitating social action through a practical, situative, context-bounded, responsive and transformative framework.

The study explores how PAR can contribute to cultural sustainability by linking community development with participatory evaluation, and it offers new perspectives on the applicability of PAR as a tool to link ex ante and ex-post evaluation, situated at various stages of the policy cycle.

Although based on a single case, the paper demonstrates that the method has the potential to be applied in various contexts, as it helps to foster local ownership and to develop future cultural strategies, thus providing a base for cultural sustainability.

The novelty of this study is to link evaluation – a majorly top-down and ex ante approach – with participatory planning. PAR-E offers a continuous participatory framework for the whole European Capitals of Culture (ECoC) cycle, as well as serving as a tool for empowerment and community development.

The authors explore the under-researched area of post-decision evaluation in PPPs (public–private partnerships), focusing upon how and whether Post-decision Project Evaluation (PdPE) is considered and provided for in United Kingdom (UK) public infrastructure projects.

The authors’ research design sought insights from overviewing UK PPP planning and more focused exploration of PPP operational practice. The authors combine the extensive analysis of planning documents for operational UK PPP projects with interviews of different stakeholders in PPP projects in one city. Mobilising an open critical perspective, documents were analysed using ethnographic content analysis (ECA) and interviews were analysed using thematic analysis consistent therewith. The authors theorise the absence and ambiguities of PdPE drawing on the sociology of ignorance.

The authors find a long-standing absence and lack of PdPE in PPP projects throughout planning and operational practice, reflecting a dynamic, multi-faceted ignorance. Concerning planning practice, the authors’ documentary analysis evidences a trend in PdPE from its absence in the early years (which may indicate some natural or genuine ignorance) to different levels or forms of weak inclusion later. Regarding this inclusion, the authors find strategic ignorance played a substantive role, involving “deliberate engineering” by both public sector and private partners. Interview findings indicate lack of clarity over PdPE and its under-development in PPP practice, deficiencies again suggestive of natural and strategic ignorance.

The authors draw from the sociology of ignorance vis-à-vis accounting's absence and ambiguity in the context of PPP, contributing to an under-researched area.

This study aims to develop an ex-post impact evaluation method for public-private partnership projects (PPP) that addresses one of its main difficulties: the multiplicity of perspectives and stakeholders that exist in these projects.

This study first developed a conceptual evaluation framework called ‘Project Success Evaluation Pyramid Model’ (PSEPM) that was designed using a Design Science approach. Second, an impact evaluation method was designed based on the PSEPM. This evaluation method was tested and refined using three PPP projects. This article presents the refined conceptual model PSEPM, the evaluation method and one of the three tests: the PPP toll road.

The impact evaluation method describes how to evaluate the impact of an existing mega project from the perspective of multiple stakeholders and isolate the impact of the procurement method. Its application in practice is demonstrated with a PPP toll road project in Chile, in which 11 stakeholders were identified. Issues such as city segregation, deficient business model, the relevance of infrastructure availability, and deficient service-fee strategy were identified in the project. The evaluation method provides a judgement about the use of the PPP procurement method and about the overall impact of the project, reconciling the stakeholder perspectives.

The use of this approach could give greater clarity and legitimacy to the discussion over the use of PPPs by evaluating existing projects beyond the use of traditional time and cost measurements. Its originality is the systematic recognition and identification of stakeholder motives, purposes, and judgements over a project.

Lists research evidence that organizations are often unaware of underutilized capacity resources and examines the implications of explicit unused capacity reporting which identifies the cost of unused capacity, pointing out that although this information is useful, it may increase evaluator outcome effects. Describes an experiment to test this and shows that where unused capacity is reported biased performance evaluation results; and individual decision makers may inappropriately reduce capacity or increase production to avoid negative evaluations. Considers how management accountants can mitigate this effect, recognizes the limitations of the study and calls for further research.

This paper reviews the normative literature of performance measurement within the context of Public-Private Partnerships (PPPs). The purpose of this paper is to examine the effectiveness of current ex post evaluations of PPPs and identify a feasible direction to comprehensively and effectively measure the performance of PPP infrastructure projects.

An in-depth literature review is conducted in this paper. The focus of the review is associated with the general performance measurement and performance measurements of PPPs.

This paper identifies that conventional ex post evaluation is not robust enough to measure the performance of PPP projects. Based on the characteristics of PPPs, the life-cycle (process-based) evaluation under performance measurement system is a promising approach to comprehensive and effective PPP performance measurement.

The outcomes of this paper can be used as a theoretical base for the development of PPP performance measurement framework.

Performance measurement is essential to business success, whether it is at the organisation or project level. Limitations on public funds have encouraged more and more governments across the world to use PPPs to procure economic and social infrastructure projects. Similar to traditional procurement, ex post evaluation is being widely used in PPP projects. However, PPPs are more complicated than other traditional procurement approaches. Exploration of literature suggests that limited research has been undertaken to examine if conventional ex post evaluation is sufficient to measure the performance of PPPs. This paper will bridge this significant knowledge gap.

The purpose of this paper is to understand the underlying logics applied by different project evaluation approaches and to propose an alternative research agenda.

This paper explores the project evaluation literature via conducting a qualitative research applying systematic literature review and thematic analysis.

The project evaluation literature has mainly concentrated on the objective aspects of project evaluation and overlooked the subjective aspects that reflect the temporal, dynamic, complex and subjective nature of today’s projects. The authors propose a meta-framework that helps project practitioners to select an appropriate project evaluation criterion for their projects by considering the strengths and limitations of their preferred project evaluation model as well as making project evaluators aware of the underlying logics associated to diverse project evaluation approaches.

This study suggests that new conceptual approaches to deal with some of the major challenges in the project evaluation field. Practice-based views, narrative analysis and actor-network theory are likely to be useful tools to better understand and cope with the projects’ uncertainty and complexity.

The findings of this research assist project management practitioners and particularly project evaluators to enhance their understanding of the subjectivity, complexity and dynamics of current projects. To increase the reflexivity and resilience of project evaluation practice, this study also proposes new directions to apply different criteria, sub-criteria and indicators to the evaluation practice.

The originality of this study relies on transcending the conventional objective and rational approaches prevailing in current project evaluation practices. It proposes a research agenda that pave the way to address the shortcomings of conventional project evaluation practice.

This study aims to reconceptualize performance evaluation styles and reveal their performance effects.

Based on a literature review, this study conceptualizes performance evaluation styles on two dimensions: priority of budgetary targets when setting performance criteria and use of accounting information for ex-post performance evaluation. This study discusses two concepts – budget rigidity and discretionary adjustments – to explain these two dimensions, and their optimal combination is then investigated by considering environmental uncertainty. The empirical analysis uses survey data from Japanese firms.

The results indicate that suitable combinations of budget rigidity and discretionary adjustments differ depending on environmental uncertainty. As expected, a combination of lower budget rigidity and higher discretionary adjustments is optimal in an uncertain environment. Contrary to expectations, a combination of higher budget rigidity and higher discretionary adjustments is optimal in a stable environment. Moreover, higher discretionary adjustments complement budgetary targets’ motivational effects, regardless of environmental uncertainty.

This study’s theoretical and empirical analysis suggests that it is difficult to understand the performance implications of performance evaluation styles without recognizing their multidimensionality and interdependencies. Moreover, the results demonstrate that discretionary adjustments in budget-based performance evaluations seem to act rationally in practice.

The purpose of this study is to characterize output and impact evaluation initiatives from the perspective of entrepreneurs and investors of social enterprises (SE) by highlighting their relevance, implementation processes and consequences.

This study adopted a qualitative approach through a multiple case study. This study also analyzed ten SE in five areas and three investors, using semi-structured interviews, codification and content analysis.

Evaluation is still at an initial stage in these businesses, and there is a gap between what the field-specific literature proposes and what firms do. Although entrepreneurs feel that evaluation is essential to demonstrate their value and confirm their purpose and impact, it is not a priority.

Due to the small number of Brazilian enterprises and investors, it is impossible to generalize the results.

The results are relevant to rethink strategies, processes and methodological tools that seek to portray, more realistically and reliably, the evaluation dimensions in SE.

This study highlights the risk regarding the legitimacy of social impact ventures since, without the due causal inferences, it is not easy to prove the impact they claim to achieve. This situation conducts to the weakening of the ecosystem leading to the emptying of the term impact.

The main research contributions address the methodologies, practices and evaluation routines of some Brazilian SE and their investors, presenting the characteristics, perceptions and limitations of the studied ventures.