Search results

Fidelity data in clinical trials are not only necessary for appraising the internal and external validity, but also could provide useful insights how to improve the application of an intervention in everyday settings. The purpose of this paper is to understand the current literature of fidelity measurements in complex interventions for people with intellectual disabilities (ID) and behaviours that challenge.

The electronic databases MEDLINE, Embase, PsycINFO, Web of Science and CINAHL Plus were searched for studies published between 1990 to 2017 that have mentioned fidelity in randomised controlled trials of complex interventions for people with ID and behaviours that challenge based on positive behaviour support or applied behaviour analysis principles. The authors also searched the grey literature and reference lists.

Five randomised controlled trials were included in the review. The authors found variable and inconsistent fidelity measurements reported in the studies. The most frequently provided fidelity elements found in four out of five studies were adherence of implementation, dose and some aspects of quality of delivery.

Research recommendations for a standardised approach of measuring fidelity in such studies are suggested.

The first review of such type that confirms the paucity of research measuring fidelity in complex interventions in this population.

Problem alcohol use (PAU) is common and associated with considerable adverse outcomes among patients receiving opioid agonist treatment (OAT). The purpose of this paper is to describe a qualitative feasibility assessment of a primary care-based complex intervention to promote screening and brief intervention for PAU, which also aims to examine acceptability and potential effectiveness.

Semi-structured interviews were conducted with 14 patients and eight general practitioners (GPs) who had been purposively sampled from practices that had participated in the feasibility study. The interviews were transcribed verbatim and analysed thematically.

Six key themes were identified. While all GPs found the intervention informative and feasible, most considered it challenging to incorporate into practice. Barriers included time constraints, and overlooking and underestimating PAU among this cohort of patients. However, the intervention was considered potentially deliverable and acceptable in practice. Patients reported that (in the absence of the intervention) their use of alcohol was rarely discussed with their GP, and were reticent to initiate conversations on their alcohol use for fear of having their methadone dose reduced.

Although a complex intervention to enhance alcohol screening and brief intervention among primary care patients attending for OAT is likely to be feasible and acceptable, time constraints and patients’ reticence to discuss alcohol as well as GPs underestimating patients’ alcohol problems is a barrier to consistent, regular and accurate screening by GPs. Future research by way of a definitive efficacy trial informed by the findings of this study and the Psychosocial INTerventions for Alcohol quantitative data is a priority.

To the best of the knowledge, this is the first qualitative study to examine the capability of primary care to address PAU among patients receiving OAT.

In this paper, I offer a series of reflections on some issues involved in evaluating complex public policy programmes in a context of rapid change and against the background of an activist government setting exacting standards for the delivery of public service outcomes. I argue that the overlay of pressures on programme evaluation as a result of the engagement with such complex public policy interventions presents a series of challenges for evaluation.

Intervention evaluations have not always accounted for long-term implementation of interventions. The purpose of this paper is to explore implementation of a primary care intervention during the lifespan of the trial and beyond.

Eight general practices participated in the trial (four control and four intervention). In-depth interviews (with nine GPs and four practices nurses who delivered the intervention) and observation methods were employed. Thematic analysis was utilized and Normalization Process Theory (NPT) constructs were compared with emergent themes.

Macro-level policy imperatives shaped practice priorities which resulted in the “whole system” new intervention not being perceived to be sustainable. Continued routinization of the intervention into usual care beyond the lifespan of the funded study was dependent on individualized monitoring and taking forward tacit knowledge.

The authors discuss the implications of these findings for sociological theories of implementation and understanding outcomes of research led complex interventions.

The study describes the complex interplay between macro processes and individual situated practices and contributes to understanding if, how, and why interventions are sustained beyond initial “research push”. The value of the study lies in describing the conditions and potential consequences of long-term implementation, which might be translated to other contexts.

Randomised controlled trials (RCTs) have been offered a privileged position in terms of the evidence base for preventive interventions for children, but practical and theoretical issues challenge this research methodology. This paper aims to address this issue.

This paper analyses practical and methodological issues of using RCTs within children's preventive services and presents the results of a qualitative study using data collected from parents who were asked to take part in an RCT of a preventive intervention.

Well recognised issues include the impossibility of blinding participants, the problem of identifying a pre‐eminent outcome measure for complex interventions, and problems with limiting access to equivalent interventions in real world settings. A further theoretical problem is the exclusion from RCTs of families who are most ready to change, resulting in a reduced level of intervention effectiveness. Qualitative evidence from one recent RCT suggests that this problem could be operating in some prevention trials. Increasing sample sizes can overcome some of these problems, but the cost of the necessarily huge trials becomes disproportionate to the intervention?

Given the limitations on RCTs in preventive settings, the paper argues their privileged position in terms of research evidence maybe undeserved.

The purpose of this paper is twofold: first, to introduce MEDIA PROTECT, a multi-modal intervention for parents and teachers with six components preventing problematic, and in the long run addictive, use of screen media by children; second, to present results of a formative evaluation of the teacher training, an important component of the intervention.

In presenting the intervention, the authors used a common framework to describe complex interventions systematically. For the evaluation, participants were recruited from German schools and kindergarten for a cluster-controlled trial. As part of a formative evaluation, the participants completed written questionnaires to determine the importance of different components of the intervention in addition to their overall satisfaction. Additional qualitative interviews with teachers were conducted.

The authors recruited n=50 kindergartens and n=9 schools; n=30 received the intervention. N=222 teachers participated in the training and n=192 completed the questionnaire (86 per cent). Seven qualitative interviews were conducted. Participants exhibited high levels of overall satisfaction with the training, considered it moderately to highly relevant to their work, and exhibited varying satisfaction levels with different components and multipliers. The qualitative data support these findings.

Few interventions to date have pursued a universal approach to the prevention of problematic screen media use. This evaluation of MEDIA PROTECT is the most recent study in Germany, which involves a complex programme through which teachers are taught ways to promote the age-adequate use of screen media in the family, parents are provided with guidance and healthy leisure time activities for children are fostered.

The teaching of cooking is an important aspect of school-based efforts to promote healthy diets among children, and is frequently done by external agencies. Within a limited evidence base relating to cooking interventions in schools, there are important questions about how interventions are integrated within school settings. The purpose of this paper is to examine how a mobile classroom (Cooking Bus) sought to strengthen connections between schools and cooking, and drawing on the concept of the sociotechnical network, theorise the interactions between the Bus and school contexts.

Methods comprised a postal questionnaire to 76 schools which had received a Bus visit, and case studies of the Bus’ work in five schools, including a range of school sizes and urban/rural locations. Case studies comprised observation of Cooking Bus sessions, and interviews with school staff.

The Cooking Bus forged connections with schools through aligning intervention and schools’ goals, focussing on pupils’ cooking skills, training teachers and contributing to schools’ existing cooking-related activities. The Bus expanded its sociotechnical network through post-visit integration of cooking activities within schools, particularly teachers’ use of intervention cooking kits.

The paper highlights the need for research on the long-term impacts of school cooking interventions, and better understanding of the interaction between interventions and school contexts.

This paper adds to the limited evidence base on school-based cooking interventions by theorising how cooking interventions relate to school settings, and how they may achieve integration.

This paper evaluates the implementation of a school-based physical activity intervention and discusses how the intervention outcomes can be influenced by the implementation.

In four of the nine lower secondary schools in which the intervention was conducted, the authors examined implementation fidelity, adaptation, quality, responsiveness and dose received. The authors conducted focus group interviews with teachers (n = 8) and students (n = 46) and made observations. Dose delivered was examined quantitatively, with weekly registrations.

Results showed that two out of four schools made few and positive adaptations, implemented the intervention with high fidelity and quality and responded positively. Four main factors were found to influence implementation: frame factors, intervention characteristics, participant characteristics and provider characteristics.

A cross-sectional design was used and may not represent implementation throughout the whole school year.

In terms of large-scale implementation, the intervention may be generalizable. However, intervention criteria such as adequate facilities and a flexible timetable may be unattainable for some schools. The intervention can be adapted without compromising its purpose, but adaptations should be a result of cooperation between students and teachers.

Process evaluations on this topic are rare. This study adds to a limited knowledge base concerning what factors may influence implementation of school-based physical activity interventions for adolescents.

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid change, complexity of context and inconsistent terminology. The purpose of this paper is to identify new evaluation approaches that will address these and reflect on the evaluation imperative for complex technology initiatives.

An initial investigation of traditional evaluative approaches used within the technology domain was broadened to investigate the evaluation practices within social and public policy domains. Realist evaluation, a branch of theory-based evaluation, was identified and reviewed in detail. The realist approach was then refined, proposing two additional necessary steps to support mapping the technical complexity of initiatives.

A refined illustrative example of a realist evaluation framework is presented, including two novel architectural edtech domain reference models to support mapping.

Recommendations include building individual evaluator capacity; adopting the realist framework; the use of architectural edtech domain reference models; phased evaluation to first build theories in technology “context” and then iteratively during complex implementation chains; and community contribution to a shared map of technical and organisational complexity.

This paper makes a novel contribution by arguing the imperative for a theory-based realist approach to help redefine evaluative thinking within the IT and complex system domain. It becomes an innovative proposal with the addition of two domain reference models that tailor the approach for edtech. Its widespread adoption will help build a shared evidence base that synthesizes and surfaces “what works, for whom, in which contexts and why”, benefiting educators, IT managers, funders, policymakers and future learners.

Systems of care and evidence-based practice possess distinct histories. Though each developed out of attempts to improve services to youth with emotional and behavioral disorders, they did so from perspectives so different as to appear diametrically opposed. Service systems exist at multiple levels, including the practice, program, and system levels (Rosenblatt, 1988, 2005; Rosenblatt & Woodbridge, 2003). Research on health and mental health service systems similarly varies, often by level of the service system, with the research methods, independent and dependent variables, populations of interest, and ultimately the consumers of the research product interacting differentially in the creation and understanding of what constitutes a knowledge base for service delivery. Systems of care and, with limited exceptions, evidence-based practices exist at different levels of the service delivery structure, require and derive from different research approaches, and speak to overlapping but historically different audiences.