Search results

Performance measurement and programme evaluation have been promoted as a central mechanism of recent Australian public sector (APS) reform. Outlines recent reforms in the APS and identifies links between evaluation and performance information. Identifies the major issue of credibility, when performance information is produced internally and not verified externally. A lack of performance systems and standards can create difficulties for both internal and external programme evaluations. Concludes that: reforms introduced to evaluate performance in the APS were promoted with high expectations which have only partially been fulfilled; the present system is internally focused with a narrow role for evaluation and a lack of credibility because of the independence issue; the present systems associated with the performance approach and its evaluation are not providing enough information to deal with the tough questions of the effectiveness of government programmes. Proposes that a middle ground between internal and external programme evaluation strategies be adopted. This allows the strengths of internal evaluation to be retained. At the same time, it allows the possibility of improving programme evaluation by adding external independent verification and an extended effectiveness role.

This chapter examines the nature and role of theory in criminal justice evaluation. A distinction between theories of and theories for evaluation is offered to clarify what is meant by ‘theory’ in the context of contemporary evaluation practice. Theories of evaluation provide a set of prescriptions and principles that can be used to guide the design, conduct and use of evaluation. Theories for evaluation include programme theory and the application of social science theory to understand how and why criminal justice interventions work to generate desired outcomes. The fundamental features of these three types of theory are discussed in detail, with a particular focus on demonstrating their combined value and utility for informing and improving the practice of criminal justice evaluation.

Management training and development is currently in vogue. There appears to be a growing belief in the benefits of investment in training and development. When a market is buoyant is the time to consider and anticipate the consequences of a future downturn in demand. Such a downturn in demand may demonstrate increasing pressure to “justify” investment in training and development. There is a long established academic body of knowledge on the subject of evaluating training and development. From research evidence and the authors′ experience, the sponsors and the providers of training and development pay scant attention to systematic evaluation of these activities and investments. It is the authors′ contention that when the market′s critical assessment of the value of training and development increases there will be an increasing interest in evaluation. An overview of the history of evaluation traditions is provided and the state of play is commented upon. It is noted that there is a shortfall between theory and practice. It is argued that evaluation is a worthwhile and important activity and ways through the evaluation literature maze and the underpinnings of the activity are demonstrated, especially to management. Similarly the literature on evaluation techniques is reviewed. Tables are provided which demonstrate areas of major activity and identify relatively uncharted waters. This monograph provides a resource whereby practitioners can choose techniques which are appropriate to the activity on which they are engaged. It highlights the process which should be undertaken to make that choice in order that needs of the major stakeholders in the exercise are fully met.

This study aims to explore ways of developing, implementing and validating a new framework and criteria for self-evaluation of programme curricula, with specific reference to the quality assurance certification process in a particular case.

The framework is developed using a case study research methodology and implemented based on criteria extraction through the triangulation of indicators achieved from internal goals and external directives.

The findings reveal that this is an improvement-led framework that can be adapted to other contexts during the quality assurance processes to facilitate periodical programme evaluations with a focus on the curriculum.

The proposed practical tool is developed for the programme evaluation process with a curricular focus during the quality assurance certification process of an interior architectural programme while enlightening the processes for the periodical self-evaluations of other institutions. The framework depends on both institution-specific internal and external directives and fulfilling curriculum-related criteria of the European Standards and Guidelines for Quality Assurance.

Within quality assurance processes, despite external quality assurance mechanisms, there is a shortage of self-evaluation tools for internal quality assurance procedures, which allow the dissociation of programme-specific qualities. The proposed framework is developed as an example of a self-tailored internal quality assurance tool and process for educational institutions to reveal their unique qualities.

This aim of this paper is to examine the existing gender digital inclusion evaluation guidance and proposes future research recommendations for their evaluation. Despite modern progress in towards gender equality and women’s empowerment movements, women’s access to, use of and benefits from digital technologies remain limited owing to economic, social and cultural obstacles. Addressing the existing gender digital divide is critical in the global efforts towards the United Nations’ Sustainable Development Goals (SDGs). In recent years, there has been a global increase of gender digital inclusion programmes for girls and women; these programmes serve as a mechanism to learn about gender-specific digital needs and inform future digital inclusion efforts. Evaluation reports of gender digital inclusion programmes can produce critical insights into girls’ and women’s learning needs and aspirations, including what works and what does not when engaging girls and women in information and communications technologies. While there are many accounts highlighting the importance of why gender digital inclusion programmes are important, there is limited knowledge on how to evaluate their impact.

The thematic analysis suggests three points to consider for the gender digital inclusion programmes evaluation: context-specific understanding of gender digital inclusion programmes; transparency and accountability of the evaluation process and its results; and tensions between evaluation targets and empowerment of evaluation participants.

The thematic analysis suggests three points of future focus for this evaluation process: context-specific understanding of gender digital inclusion programmes; transparency and accountability of the evaluation process and its results; and tensions between evaluation targets and empowerment of evaluation participants.

The authors propose recommendations for gender digital inclusion evaluation practice and areas for future research.

The purpose of this paper is to describe the evolving IFLA approach to impact evaluation through three of its international programmes: Freedom of Access to Information, Building Strong Library Associations (BSLA) and the International Advocacy Programme (IAP). This review positions these three programmes within the wider discourse of the international evaluation community.

Each of the three programmes is considered in turn to show what they were trying to achieve and how thinking about impact evaluation at IFLA is evolving.

This paper reports key evaluation findings for relevant phases of the BSLA and IAP programmes in general terms.

The views presented are those of the evaluation consultants who advised each of these programmes (and in the cases of BSLA and the IAP conducted the programme evaluations).

The processes described and the conclusions drawn should be of interest to anyone involved in international or national library evaluation, especially of public libraries, library associations and national libraries.

The paper suggests that more systematic impact evaluation of public libraries, library associations and national libraries is necessary to ensure their future survival.

The authors were uniquely placed to see and participate in IFLA impact evaluation discussions over the past decade.

The purpose of this chapter is to discuss theory-based evaluation of public–private partnership (PPP) projects/programmes and to develop an intervention logical framework. It aims to draw attention to the need to go beyond the measurement of project/programme results to address not only the question of whether or not the project/programme worked but also the how and why questions. The study follows an interpretative methodology. It analytically discloses the mechanics of theory-based evaluation in relation to a ‘PPP theory’ and describes a theory-based analytical framework that portrays an explicit path towards ultimate impacts so as to assess, in a more systematic and integrated way, the success or failure of a PPP. Theory-based evaluation is a promising evaluation approach that would fit into the complexities of PPP projects/programmes and would expand the available toolbox of evaluators. Proper use of theory-based evaluation in PPP interventions contributes to better policy formulation and project implementation, thus leading to improved socio-economic benefits derived from PPP projects and programmes. The main contribution of the study is that it develops a ‘PPP theory’ and a related logical intervention framework drawing on theory-based approaches. Although the framework is developed for a representative sector, transport, it can easily be applied to any other PPP intervention.

A case study methodology was applied as a major component of a mixed-methods approach to the evaluation of a mobile dementia education and support service in the Bega Valley Shire, New South Wales, Australia. In-depth interviews with people with dementia (PWD), their carers, programme staff, family members and service providers and document analysis including analysis of client case notes and client database were used.

The strengths of the case study approach included: (i) simultaneous evaluation of programme process and worth, (ii) eliciting the theory of change and addressing the problem of attribution, (iii) demonstrating the impact of the programme on earlier steps identified along the causal pathway (iv) understanding the complexity of confounding factors, (v) eliciting the critical role of the social, cultural and political context, (vi) understanding the importance of influences contributing to differences in programme impact for different participants and (vii) providing insight into how programme participants experience the value of the programme including unintended benefits.

The broader case of the collective experience of dementia and as part of this experience, the impact of a mobile programme of support and education, in a predominately rural area grew from the investigation of the programme experience of ‘individual cases’ of carers and PWD. Investigation of living conditions, relationships, service interactions through observation and increased depth of interviews with service providers and family members would have provided valuable perspectives and thicker description of the case for increased understanding of the case and strength of the evaluation.

There has been a surge in interest in public management development in the 1980s. Countries that had no management development programmes introduced new ones, while countries already having such programmes made far‐reaching changes to them. Initiatives have been costly precisely at a time when most governments have had to exercise restraint in their spending. Suggests that governments should have in place rigorous evaluation plans to assess if the programmes are successful. Reviews the evaluation efforts of several countries in public management development programmes. The study reveals that the evaluation record is spotty with the evaluation efforts of some countries, notably the United Kingdom, showing promise. In addition, points to several suggestions for governments to strengthen their capacity to assess the impact of their management development programmes. Concludes by arguing that governments tend to bias their evaluation of management development efforts and the results when they initially identify what ought to be evaluated and how.