Search results

Common literature review methods such as systematic review and narrative review are poorly suited to the investigation of complex management phenomena. Systematic reviews are highly driven by protocol and procedure, and are oft-criticized as reductive and poorly equipped to examine the interaction between phenomena and context, nonlinear processes, and empirical outcomes that are less predictable. Narrative reviews, on the other hand, are pluralistic and iterative and thus better suited to descriptions of the complex and unpredictable; however, they tend to lack methodological transparency, trustworthiness, and pragmatism in application. The “realist synthesis” approach to literature review can be seen as the middle-ground between these two common methods, offering both methodological rigor alongside flexibility and nuance. Realist synthesis takes an explanatory frame, with a focus on unearthing the theorized causal mechanisms at play beneath a phenomenon of interest.

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid change, complexity of context and inconsistent terminology. The purpose of this paper is to identify new evaluation approaches that will address these and reflect on the evaluation imperative for complex technology initiatives.

An initial investigation of traditional evaluative approaches used within the technology domain was broadened to investigate the evaluation practices within social and public policy domains. Realist evaluation, a branch of theory-based evaluation, was identified and reviewed in detail. The realist approach was then refined, proposing two additional necessary steps to support mapping the technical complexity of initiatives.

A refined illustrative example of a realist evaluation framework is presented, including two novel architectural edtech domain reference models to support mapping.

Recommendations include building individual evaluator capacity; adopting the realist framework; the use of architectural edtech domain reference models; phased evaluation to first build theories in technology “context” and then iteratively during complex implementation chains; and community contribution to a shared map of technical and organisational complexity.

This paper makes a novel contribution by arguing the imperative for a theory-based realist approach to help redefine evaluative thinking within the IT and complex system domain. It becomes an innovative proposal with the addition of two domain reference models that tailor the approach for edtech. Its widespread adoption will help build a shared evidence base that synthesizes and surfaces “what works, for whom, in which contexts and why”, benefiting educators, IT managers, funders, policymakers and future learners.

This study aims to examine the extent to which “What Works” reviews in youth justice enable understanding of the features of effectiveness (what works, for whom, in what circumstances and why?) specified in the Effects–Mechanisms–Moderators–Implementation–Economic cost (EMMIE) framework.

The EMMIE framework examined findings within a sample of “What Works” style reviews of preventative youth justice intervention effectiveness.

“What Works” style reviews of evaluations of preventative youth justice interventions often omit the requisite details required to examine all of the necessary elements of effectiveness contained within the EMMIE framework. While effectiveness measures were typically provided, the dominant evaluation evidence-base struggles to consider moderators of effect, mechanisms of change, implementation differences and cost-effectiveness. Therefore, “What Works” samples cannot facilitate sufficient understanding of “what works for whom, in what circumstances and why?”. The authors argue that Realist Synthesis can fill this gap and shed light on the contexts that shape the mechanisms through which youth justice interventions work.

The authors extended the approach adopted by an earlier review of effectiveness reviews (Tompson et al., 2020), considering more recent reviews of the effectiveness of preventative interventions using the EMMIE framework. Unlike previous reviews, the authors prioritised the utility of the EMMIE framework for assessing the factors affecting the effectiveness of preventative interventions in youth justice.

The strategy map represents a major contribution to the theory and practice of performance management. However, it has failed to realize its full potential due to a lack of theoretical and conceptual development. Therefore, the purpose of this paper is to revisit the theories of strategy maps to better understand how and in what circumstances they benefit performance management.

The study employs realist synthesis, a method of systematic literature review. A theory on how strategy maps work is extracted from performance management literature, which are subsequently evaluated through a critical examination of empirical studies.

A theory of how strategy maps are meant work is presented in relation to the generic performance management stages of problem structuring, development and use, where they can serve as a tool for discovery and by stimulating social interactions. Based on the findings, 12 propositions are offered related to the effective use of strategy maps within a performance management framework.

The introduction of the strategy map to performance management represented a breakthrough in how organizational performance could be understood and communicated. This study goes a step further by considering how they work and in what circumstances. In so doing, the study aims to open the way for new and more effective applications of strategy maps within the changing performance management context.

This study provides practitioners with actionable propositions which can help in effectively using strategy maps.

Distinguishing the aims and mechanisms of the strategy map along performance management systems has the potential to greatly increase their effectiveness in practice as a powerful, but underutilized tool. This paper also demonstrates how realist synthesis, currently an uncommon method in management studies, facilitated the creation of a new perspective of strategy maps to fit specifically within performance management.

The collection and dissemination of standardized performance information about students, teachers, schools and school systems offer potentially important tools for school accountability and resource allocation as well as school improvement in developing countries. However, performance monitoring systems in developing countries are in many cases copied from those in high-income countries without a clear understanding of their functioning in contexts of limited resources and capacity for change. The purpose of this paper is to examine the conditions under which and the mechanisms through which system-wide performance monitoring affects school-level organization and processes in low- and middle-income countries (LMICs).

The review employs realist synthesis because of the complexity and dynamism of conditions in LMICs, the wide variability in available literature and the aim of explaining how particular organizational outcomes arise, given particular conditions. The authors draw on findings from a systematic review of 22 studies and reports, published since 2001, related to the implementation of performance monitoring.

The findings highlight key barriers to the use of data to inform school accountability and improvement. Capacity to collect, interpret and use data is an important condition to both effective external accountability as well as improvement of schools.

The review uses realist approaches to building middle-level theories to help scholars, educational advisers, policy makers and educational leaders understand the causal processes that result in certain outcomes from monitoring activities and to identify the conditions that are necessary for those processes to have the desired outcomes.

This paper aims to share the findings of a realist evaluation study that set out to identify how to strengthen nursing, midwifery and allied health professions (NMAHP) leadership across all health-care contexts in the UK conducted between 2018 and 2019. The collaborative research team were from the Universities of Bangor, Ulster, the University of the West of Scotland and Canterbury Christ Church University.

Realist evaluation and appreciative inquiry were used across three phases of the study. Phase 1 analysed the literature to generate tentative programme theories about what works, tested out in Phase 2 through a national social media Twitter chat and sense-making workshops to help refine the theories in Phase 3. Cross-cutting themes were synthesised into a leadership framework identifying the strategies that work for practitioners in a range of settings and professions based on the context, mechanism and output configuration of realist evaluation. Stakeholders contributed to the ongoing interrogation, analysis and synthesis of project outcomes.

Five guiding lights of leadership, a metaphor for principles, were generated that enable and strengthen leadership across a range of contexts. – “The Light Between Us as interactions in our relationships”, “Seeing People’s Inner Light”, “Kindling the Spark of light and keeping it glowing”, “Lighting up the known and the yet to be known” and “Constellations of connected stars”.

This study has illuminated the a-theoretical nature of the relationships between contexts, mechanisms and outcomes in the existing leadership literature. There is more scope to develop the tentative programme theories developed in this study with NMAHP leaders in a variety of different contexts. The outcomes of leadership research mostly focussed on staff outcomes and intermediate outcomes that are then linked to ultimate outcomes in both staff and patients (supplemental). More consideration needs to be given to the impact of leadership on patients, carers and their families.

The study has developed additional important resources to enable NMAHP leaders to demonstrate their leadership impact in a range of contexts through the leadership impact self-assessment framework which can be used for 360 feedback in the workplace using the appreciative assessment and reflection tool.

Whilst policymakers note the increasing importance of leadership in facilitating the culture change needed to support health and care systems to adopt sustainable change at pace, there is still a prevailing focus on traditional approaches to individual leadership development as opposed to collective leadership across teams, services and systems. If this paper fails to understand how to transform leadership policy and education, then it will be impossible to support the workforce to adapt and flex to the increasingly complex contexts they are working in. This will serve to undermine system integration for health and social care if the capacity and capability for transformation are not attended to. Whilst there are ambitious global plans (WHO, 2015) to enable integrated services to be driven by citizen needs, there is still a considerable void in understanding how to authentically engage with people to ensure the transformation is driven by their needs as opposed to what the authors think they need. There is, therefore, a need for systems leaders with the full skillset required to enable integrated services across place-based systems, particularly clinicians who are able to break down barriers and silo working across boundaries through the credibility, leadership and facilitation expertise they provide.

The realist evaluation with additional synthesis from key stakeholders has provided new knowledge about the principles of effective NMAHP leadership in health and social care, presented in such a way that facilitates the use of the five guiding lights to inform further practice, education, research and policy development.

This chapter, together with those that follow, builds upon the ideas presented in the previous volume in this series (Maginn, Thompson, & Tonts, 2008). There we outlined our vision for a ‘pragmatic renaissance’ in contemporary qualitative research in urban studies. We argued that to survive as an effective and frequently used tool for policy development, a more systematic approach is needed in the way that qualitative-informed applied urban research is conceptualised and undertaken. In opening this volume we build on these initial ideas using housing as a meta-case study to progress the case for a systematic approach to qualitative research methods. We do this to both stimulate broad debate about the ways, in which qualitative research in urban/housing scholarship might be of greater use to policymakers and practitioners, as well as to suggest a way forward in realising the ‘pragmatic renaissance’.

In the UK signposting services can be developed as enhanced support for people with health and social care needs or service diversion to help primary and urgent care services manage their workload. This review considers these two conflicting purposes.

The review used a realist approach, initial searches to identify theory; we then selected 22 publications and extracted programme theories, from which we developed questions from three viewpoints: the service user, the front-line service provider and the commissioner. A rich sample of studies were found from purposive searching. To optimise the applicability of synthesis findings predominantly UK studies were included.

Users value signposting service that understand their needs, suggest a range of options and summarise potential actions. People with complex health and social care needs generally require extended time/input from signposting services. Front-line providers require initial and ongoing training, support/supervision, good knowledge of available services/resources and the ability to match users to them and a flexible response. Commissioned signposting services in England are diverse making evaluation difficult.

Meaningful evaluation of signposting services requires greater clarity around roles and service expectations. Signposting services alone fulfil the needs of a small number of users due to the unreconciled tension between efficient (transactional) service provision and effective (relational) service provision. This is underpinned by competing narratives of whether signposting represents diversion of inappropriate demand from primary care and other urgent care services or improved quality of care through a joined-up response encompassing health, social care and community/voluntary services.

Meta-synthesis of the research evidence adds value to the process of literature reviewing, providing useful knowledge for researchers, practitioners and policymakers. The aim of the chapter is to explain what meta-synthesis involves and how it illuminates our understanding of concepts. Previous papers on meta-synthesis (by the author) have reviewed research strategies in information behaviour research and methods for meta-synthesis, discussed application of meta-synthesis to research on information behaviour of women, and proposed methods suitable for integrating information literacy research. Meta-synthesis methods have been applied to many areas of social science research. The literature review examines how to reduce the risks involved in suitable for integrating qualitative research or qualitative and quantitative research; outlines the main approaches used in meta-synthesis before explaining the processes used in a meta-synthesis of research on information behaviour of women, reflects on the meta-synthesis methods used, and which might have been used, and shows what meta-synthesis achieves. Meta-synthesis should be used more in information behaviour research, but it is a rigorous process, requiring time and effort to get useful results. On the other hand, meta-synthesis provides more new knowledge, and a deeper understanding of our ideas, than a conventional literature review.

Given the complex nature of integrated care systems (ICSs), the geographical spread and the large number of organisations involved in partnership delivery, the importance of leadership cannot be overstated. This paper aims to present novel findings from a rapid realist review of ICS leadership in England. The overall review question was: how does leadership in ICSs work, for whom and in what circumstances?

Development of initial programme theories and associated context–mechanism–outcome configurations (CMOCs) were supported by the theory-gleaning activities of a review of ICS strategies and guidance documents, a scoping review of the literature and interviews with key informants. A refined programme theory was then developed by testing these CMOCs against empirical data published in academic literature. Following screening and testing, six CMOCs were extracted from 18 documents. The study design, conduct and reporting were informed by the Realist And Metanarrative Evidence Syntheses: Evolving Standards (RAMESES) training materials (Wong et al., 2013).

The review informed four programme theories explaining that leadership in ICSs works when ICS leaders hold themselves and others to account for improving population health, a sense of purpose is fostered through a clear vision, partners across the system are engaged in problem ownership and relationships are built at all levels of the system.

Despite being a rigorous and comprehensive investigation, stakeholder input was limited to one ICS, potentially restricting insights from varied geographical contexts. In addition, the recent establishment of ICSs meant limited literature availability, with few empirical studies conducted. Although this emphasises the importance and originality of the research, this scarcity posed challenges in extracting and applying certain programme theory elements, particularly context.

This review will be of relevance to academics and health-care leaders within ICSs in England, offering critical insights into ICS leadership, integrating diverse evidence to develop new evidence-based recommendations, filling a gap in the current literature and informing leadership practice and health-care systems.