Search results

1 – 10 of over 5000
Book part
Publication date: 6 December 2017

Mehmet Uzunkaya

The purpose of this chapter is to discuss theory-based evaluation of public–private partnership (PPP) projects/programmes and to develop an intervention logical framework. It aims…

Abstract

The purpose of this chapter is to discuss theory-based evaluation of public–private partnership (PPP) projects/programmes and to develop an intervention logical framework. It aims to draw attention to the need to go beyond the measurement of project/programme results to address not only the question of whether or not the project/programme worked but also the how and why questions. The study follows an interpretative methodology. It analytically discloses the mechanics of theory-based evaluation in relation to a ‘PPP theory’ and describes a theory-based analytical framework that portrays an explicit path towards ultimate impacts so as to assess, in a more systematic and integrated way, the success or failure of a PPP. Theory-based evaluation is a promising evaluation approach that would fit into the complexities of PPP projects/programmes and would expand the available toolbox of evaluators. Proper use of theory-based evaluation in PPP interventions contributes to better policy formulation and project implementation, thus leading to improved socio-economic benefits derived from PPP projects and programmes. The main contribution of the study is that it develops a ‘PPP theory’ and a related logical intervention framework drawing on theory-based approaches. Although the framework is developed for a representative sector, transport, it can easily be applied to any other PPP intervention.

Details

The Emerald Handbook of Public–Private Partnerships in Developing and Emerging Economies
Type: Book
ISBN: 978-1-78714-494-1

Keywords

Book part
Publication date: 29 September 2015

Murray Saunders, Cristina Sin and Steven Dempster

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish…

Abstract

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish higher education, with particular reference to the longitudinal evaluative research of support of teaching and learning (T&L) (the Quality Enhancement Framework or QEF). The chapter will discuss the features of the research process which are shaped by evaluation theory. We adopt a theoretical position on policy research which foregrounds the situated experience of policy as a core research focus. Policy is depicted as being underscored by an implicit theory of change which is used to structure and orientate the research focus. The design of the research is characterised by the involvement of potential users of the research output, with implications on the way in which findings are articulated, presented and ultimately used, along with aspects of the evaluative research design. The case study of the QEF will be contextualised, and the intersection between the design features and theoretical approaches, and the use and usability of research outputs, will be established.

Details

Theory and Method in Higher Education Research
Type: Book
ISBN: 978-1-78560-287-0

Article
Publication date: 16 June 2021

Alice Jones and Néstor Valero-Silva

English social housing providers are increasingly turning to social impact measurement to assess their social value. This paper aims to understand the contextual factors causing…

Abstract

Purpose

English social housing providers are increasingly turning to social impact measurement to assess their social value. This paper aims to understand the contextual factors causing this rise in the practice, specifically within this sector; the mechanisms that enable it to be effectively implemented within an individual organisation and the outcomes of successful implementation for individual organisations and more widely across the sector and beyond.

Design/methodology/approach

A realist theory-based approach is applied to the study of a small number of social housing organisations and leaders within the sector to explore the use of social impact measurement. The paper addresses three questions: Why is social impact measurement being adopted in this sector? How is it successfully implemented? And what happens (outcomes) when it is successfully implemented? Addressing these questions necessitates deeper insight into the contextual pressures that have brought to the fore social impact measurement within the sector and the beneficial outcomes the practice provides (or is anticipated to provide) to social housing providers. The methodological approach of Realist Evaluation (Pawson and Tilley, 1997, 2004) is used to structure and analyse the empirical data and findings into a programme theory for social impact measurement. Realist Evaluation provides a programme theory perspective, seeking to answer the question “what works, for whom and in what circumstances?”. In this research, the “whom” refers to English social housing providers and the circumstances are the contextual conditions experienced by the sector over the past decade. The programme theory aims to set out the links between the contextual drivers for social impact measurement, the mechanisms that bring about its implementation and the outcomes that occur as a result. Within this, greater detail on the implementation perspective is provided by developing an implementation theory using a Theory of Change approach (Connell et al., 1995; Fulbright-Anderson et al., 1998). The implementation theory is then embedded within the wider programme theory so as to bring the two elements together, thereby creating a refinement of the overall theory for social impact measurement. In turn, this paper demonstrates its importance (the outcomes that it can achieve for organisations and the sector) and how it can effectively be implemented to bring about those outcomes.

Findings

Social housing providers use social impact measurement both internally, to determine their organisational priorities and externally, to demonstrate their value to local and national governments and cross-sector partners then to shape and influence resource allocation. The practice itself is shown to be an open and active programme, rather than a fixed calculative practice.

Research limitations/implications

The intensive nature of the research means that only a limited number of cases were explored. Further research could test theories developed here against evidence collected from a wider range of cases, e.g. other types of providers or non-adopters.

Practical implications

The research makes a strong contribution to practice in the form of a re-conceptualisation of how social impact measurement can be shown to be effective, based on a deeper understanding of causal mechanisms, how they interact and the outcomes that result. This is of value to the sector as such information could help other organisations both to understand the value of social impact measurement and to provide practical guidance on how to implement it effectively.

Social implications

As the practice of impact measurement continues to develop, practitioners will need to be aware of any changes to these contextual factors and consider questions such as: is the context still supportive of impact measurement? Does the practice need to be adjusted to meet the needs of the current context? For instance, the recent tragedy at Grenfell Tower has led to a reconsideration of the role of social housing; a new Green Paper is currently being drafted (Ministry for Housing, Communities and Local Government, 2018). This may have a number of implications for social impact measurement, such as a rebalancing of emphasis on outcomes relating to environmental improvements, towards outcomes relating to the well-being of tenants.

Originality/value

Existing literature is largely limited to technical guides. This paper links theory-based evaluation to practice contributing to social housing practice.

Details

Qualitative Research in Accounting & Management, vol. 18 no. 3
Type: Research Article
ISSN: 1176-6093

Keywords

Book part
Publication date: 21 January 2022

Karen Laing and Liz Todd

The chapter will explore a collaborative theory of change approach that the authors used to evaluate three projects. The three projects worked with young people out of school in…

Abstract

The chapter will explore a collaborative theory of change approach that the authors used to evaluate three projects. The three projects worked with young people out of school in different ways to enable the young people to become agents of change in tackling the causes of alcohol misuse in their local Scottish communities. A theory of change approach provides a way of conceptualising programmes from inception, through to implementation and the evaluation of outcomes, in order to develop an understanding of how they work, for whom and in what circumstances. Using a collaborative model of this approach challenged prevailing notions of evaluation being the job of the evaluator and situated evaluation as a shared endeavour with the project staff. We outline the key attributes of such a collaborative model of theory of change and reflect on how this model can contribute to the evaluation of out-of-school activity.

Details

Repositioning Out-of-School Learning
Type: Book
ISBN: 978-1-78769-739-3

Keywords

Content available
Article
Publication date: 14 March 2016

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid…

Abstract

Purpose

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid change, complexity of context and inconsistent terminology. The purpose of this paper is to identify new evaluation approaches that will address these and reflect on the evaluation imperative for complex technology initiatives.

Approach

An initial investigation of traditional evaluative approaches used within the technology domain was broadened to investigate the evaluation practices within social and public policy domains. Realist evaluation, a branch of theory-based evaluation, was identified and reviewed in detail. The realist approach was then refined, proposing two additional necessary steps to support mapping the technical complexity of initiatives.

Findings

A refined illustrative example of a realist evaluation framework is presented, including two novel architectural edtech domain reference models to support mapping.

Practical implications

Recommendations include building individual evaluator capacity; adopting the realist framework; the use of architectural edtech domain reference models; phased evaluation to first build theories in technology “context” and then iteratively during complex implementation chains; and community contribution to a shared map of technical and organisational complexity.

Originality

This paper makes a novel contribution by arguing the imperative for a theory-based realist approach to help redefine evaluative thinking within the IT and complex system domain. It becomes an innovative proposal with the addition of two domain reference models that tailor the approach for edtech. Its widespread adoption will help build a shared evidence base that synthesizes and surfaces “what works, for whom, in which contexts and why”, benefiting educators, IT managers, funders, policymakers and future learners.

Details

Journal of Systems and Information Technology, vol. 18 no. 1
Type: Research Article
ISSN: 1328-7265

Keywords

Book part
Publication date: 30 December 2004

Deborah A. Boehm-Davis and Robert W. Holt

A strong, useful theoretical foundation for performance assessment and prediction relies on four components: preliminary observation of a system, identification of key or…

Abstract

A strong, useful theoretical foundation for performance assessment and prediction relies on four components: preliminary observation of a system, identification of key or dominating variables in the system, synthetic and vertical thinking, and successive refinement.

Details

The Science and Simulation of Human Performance
Type: Book
ISBN: 978-1-84950-296-2

Article
Publication date: 8 May 2018

Beth Sundstrom, Heather M. Brandt, Lisa Gray and Jennifer Young Pierce

Cervical cancer (CxCa) incidence and mortality remain unacceptably high in South Carolina, USA, presenting an ideal opportunity for intervention. To address this need, Cervical…

Abstract

Purpose

Cervical cancer (CxCa) incidence and mortality remain unacceptably high in South Carolina, USA, presenting an ideal opportunity for intervention. To address this need, Cervical Cancer-Free South Carolina developed an academic-community partnership with researchers and students at a public university to design, implement, and evaluate a theory-based CxCa communication campaign, It’s My Time. The paper aims to discuss this issue.

Design/methodology/approach

The goal of this campaign was to decrease CxCa by increasing human papillomavirus (HPV) vaccination and appropriate screening. This paper describes the development, implementation, and evaluation of a successful theory-based CxCa prevention communication campaign for college women based on formative audience research and targeted messages delivered to audience segments through new and traditional communication channels. The health belief model (HBM) served as a theoretical framework for the campaign throughout development, implementation, and evaluation.

Findings

This campaign demonstrated the effectiveness of the HBM to address CxCa prevention, including HPV vaccine acceptability. The campaign aimed to increase perceptions of susceptibility, which were low, by emphasizing that HPV is a sexually transmitted infection. A community-based grassroots approach to addressing disparities in CxCa prevention increased benefits and decreased barriers. Social media emerged as a particularly appropriate platform to disseminate cues to action. In total, 60 percent of participants who responded to an anonymous web-based survey evaluation indicated that they received the HPV vaccine as a result of campaign messages.

Originality/value

This paper offers practical suggestions to campaign planners about building academic-community partnerships to develop theory-based communication campaigns that include conducting formative research, segmenting target audiences, engaging with young people, and incorporating social media.

Details

Journal of Communication Management, vol. 22 no. 2
Type: Research Article
ISSN: 1363-254X

Keywords

Article
Publication date: 19 August 2021

Carolyn Jackson, Tamsin McBride, Kim Manley, Belinda Dewar, Beverley Young, Assumpta Ryan and Debbie Roberts

This paper aims to share the findings of a realist evaluation study that set out to identify how to strengthen nursing, midwifery and allied health professions (NMAHP) leadership…

2611

Abstract

Purpose

This paper aims to share the findings of a realist evaluation study that set out to identify how to strengthen nursing, midwifery and allied health professions (NMAHP) leadership across all health-care contexts in the UK conducted between 2018 and 2019. The collaborative research team were from the Universities of Bangor, Ulster, the University of the West of Scotland and Canterbury Christ Church University.

Design/methodology/approach

Realist evaluation and appreciative inquiry were used across three phases of the study. Phase 1 analysed the literature to generate tentative programme theories about what works, tested out in Phase 2 through a national social media Twitter chat and sense-making workshops to help refine the theories in Phase 3. Cross-cutting themes were synthesised into a leadership framework identifying the strategies that work for practitioners in a range of settings and professions based on the context, mechanism and output configuration of realist evaluation. Stakeholders contributed to the ongoing interrogation, analysis and synthesis of project outcomes.

Findings

Five guiding lights of leadership, a metaphor for principles, were generated that enable and strengthen leadership across a range of contexts. – “The Light Between Us as interactions in our relationships”, “Seeing People’s Inner Light”, “Kindling the Spark of light and keeping it glowing”, “Lighting up the known and the yet to be known” and “Constellations of connected stars”.

Research limitations/implications

This study has illuminated the a-theoretical nature of the relationships between contexts, mechanisms and outcomes in the existing leadership literature. There is more scope to develop the tentative programme theories developed in this study with NMAHP leaders in a variety of different contexts. The outcomes of leadership research mostly focussed on staff outcomes and intermediate outcomes that are then linked to ultimate outcomes in both staff and patients (supplemental). More consideration needs to be given to the impact of leadership on patients, carers and their families.

Practical implications

The study has developed additional important resources to enable NMAHP leaders to demonstrate their leadership impact in a range of contexts through the leadership impact self-assessment framework which can be used for 360 feedback in the workplace using the appreciative assessment and reflection tool.

Social implications

Whilst policymakers note the increasing importance of leadership in facilitating the culture change needed to support health and care systems to adopt sustainable change at pace, there is still a prevailing focus on traditional approaches to individual leadership development as opposed to collective leadership across teams, services and systems. If this paper fails to understand how to transform leadership policy and education, then it will be impossible to support the workforce to adapt and flex to the increasingly complex contexts they are working in. This will serve to undermine system integration for health and social care if the capacity and capability for transformation are not attended to. Whilst there are ambitious global plans (WHO, 2015) to enable integrated services to be driven by citizen needs, there is still a considerable void in understanding how to authentically engage with people to ensure the transformation is driven by their needs as opposed to what the authors think they need. There is, therefore, a need for systems leaders with the full skillset required to enable integrated services across place-based systems, particularly clinicians who are able to break down barriers and silo working across boundaries through the credibility, leadership and facilitation expertise they provide.

Originality/value

The realist evaluation with additional synthesis from key stakeholders has provided new knowledge about the principles of effective NMAHP leadership in health and social care, presented in such a way that facilitates the use of the five guiding lights to inform further practice, education, research and policy development.

Details

Leadership in Health Services, vol. 34 no. 4
Type: Research Article
ISSN: 1751-1879

Keywords

Article
Publication date: 13 March 2017

Melanie Rose Nova King, Ray J. Dawson, Steve J. Rothberg and Firat Batmaz

This study aims to investigate the effectiveness of a theory-driven realist evaluative research approach to better understand complex technology implementations in organizations.

317

Abstract

Purpose

This study aims to investigate the effectiveness of a theory-driven realist evaluative research approach to better understand complex technology implementations in organizations.

Design/methodology/approach

An institution wide e-learning implementation of lecture capture (LC), within a UK University, was chosen, and a realist evaluation framework was used, tailored for educational technology. The research was conducted over four, increasingly focused, evaluation cycles combining engagement analytics, user interviews and theory to refine what works (or does not work), for whom, in which contexts and why.

Findings

Despite explicit demand and corresponding investment, overall student engagement is lower than expected. Increased student use appears linked to particular staff attitudes and behaviours and not to specific disciplines or course content. The main benefits of LC are providing reassurance to the majority, aiding revision and understanding for the many and enabling catch-up for the few. Recommendations for future research are based on some unexpected outcomes uncovered, including evolving detrimental student behaviours, policy development based on technological determinism and future learner-centred system development for next-generation LC technologies.

Practical implications

The realist approach taken, and evaluation framework used, can be adopted (and adapted) for future evaluative research. Domain specific reference models, categorizing people and technology, supported analysis across multiple contexts.

Originality/value

This study responds to a call for more theory-based research in the field of educational technology. The authors demonstrate that a theory-driven approach provides real and practical recommendations for institutions and allows for greater insight into the political, economic and social complexity of technology implementation.

Details

Journal of Systems and Information Technology, vol. 19 no. 1/2
Type: Research Article
ISSN: 1328-7265

Keywords

1 – 10 of over 5000