Search results

This paper aims to present the process evaluation for a two‐year (2008‐2010) participatory action research project focusing on home‐school partnership in health learning, undertaken within the Schools for Health in Europe (SHE) in Eastern Finland.

Two intervention schools and two control schools (grade 5 pupils, parents, and selected school personnel) participated in a study. Process evaluation data were collected from intervention schools after 10 months of participation, by interviewing two classroom teachers and three families. In addition, program documents and relevant statistics were collected from schools during the intervention.

Teachers' opinions on the development process varied from more concrete expectations (School A teacher) to overall satisfaction to implementation (School B teacher). Parents believed that their children would benefit from the project later in life. The context and differences of the school environments were likely to affect the development process at the school level.

This paper demonstrates a process evaluation in two schools and, therefore, limits the generalizability of the findings.

The process evaluation was an essential part of this intervention study and may provide a useful structure and an example for process evaluation for future school‐based health intervention studies.

This study highlights the importance of planning the process evaluation structure before the start of the intervention, brings out the relevance of systematically assessing the process while it is ongoing, and illustrates process evaluation in an action research project.

This study aims to examine the extent to which “What Works” reviews in youth justice enable understanding of the features of effectiveness (what works, for whom, in what circumstances and why?) specified in the Effects–Mechanisms–Moderators–Implementation–Economic cost (EMMIE) framework.

The EMMIE framework examined findings within a sample of “What Works” style reviews of preventative youth justice intervention effectiveness.

“What Works” style reviews of evaluations of preventative youth justice interventions often omit the requisite details required to examine all of the necessary elements of effectiveness contained within the EMMIE framework. While effectiveness measures were typically provided, the dominant evaluation evidence-base struggles to consider moderators of effect, mechanisms of change, implementation differences and cost-effectiveness. Therefore, “What Works” samples cannot facilitate sufficient understanding of “what works for whom, in what circumstances and why?”. The authors argue that Realist Synthesis can fill this gap and shed light on the contexts that shape the mechanisms through which youth justice interventions work.

The authors extended the approach adopted by an earlier review of effectiveness reviews (Tompson et al., 2020), considering more recent reviews of the effectiveness of preventative interventions using the EMMIE framework. Unlike previous reviews, the authors prioritised the utility of the EMMIE framework for assessing the factors affecting the effectiveness of preventative interventions in youth justice.

Models of integrated care are prime examples of complex interventions, incorporating multiple interacting components that work through varying mechanisms to impact numerous outcomes. The purpose of this paper is to explore summative, process and developmental approaches to evaluating complex interventions to determine how to best test this mess.

This viewpoint draws on the evaluation and complex intervention literatures to describe the advantages and disadvantages of different methods. The evaluation of the electronic patient reported outcomes (ePRO) mobile application and portal system is presented as an example of how to evaluate complex interventions with critical lessons learned from this ongoing study.

Although favored in the literature, summative and process evaluations rest on two problematic assumptions: it is possible to clearly identify stable mechanisms of action; and intervention fidelity can be maximized in order to control for contextual influences. Complex interventions continually adapt to local contexts, making stability and fidelity unlikely. Developmental evaluation, which is more conceptually aligned with service-design thinking, moves beyond these assumptions, emphasizing supportive adaptation to ensure meaningful adoption.

Blended approaches that incorporate service-design thinking and rely more heavily on developmental strategies are essential for complex interventions. To maximize the benefit of this approach, three guiding principles are suggested: stress pragmatism over stringency; adopt an implementation lens; and use multi-disciplinary teams to run studies.

This viewpoint offers novel thinking on the debate around appropriate evaluation methodologies to be applied to complex interventions like models of integrated care.

The purpose of this chapter is to discuss theory-based evaluation of public–private partnership (PPP) projects/programmes and to develop an intervention logical framework. It aims to draw attention to the need to go beyond the measurement of project/programme results to address not only the question of whether or not the project/programme worked but also the how and why questions. The study follows an interpretative methodology. It analytically discloses the mechanics of theory-based evaluation in relation to a ‘PPP theory’ and describes a theory-based analytical framework that portrays an explicit path towards ultimate impacts so as to assess, in a more systematic and integrated way, the success or failure of a PPP. Theory-based evaluation is a promising evaluation approach that would fit into the complexities of PPP projects/programmes and would expand the available toolbox of evaluators. Proper use of theory-based evaluation in PPP interventions contributes to better policy formulation and project implementation, thus leading to improved socio-economic benefits derived from PPP projects and programmes. The main contribution of the study is that it develops a ‘PPP theory’ and a related logical intervention framework drawing on theory-based approaches. Although the framework is developed for a representative sector, transport, it can easily be applied to any other PPP intervention.

It has become accepted practice to include an evaluation alongside learning programmes that take place at work, as a means of judging their effectiveness. There is a tendency to focus such evaluations on the relevance of the intervention and the amount of learning achieved by the individual. The aim of this review is to examine existing evaluation frameworks that have been used to evaluate education interventions and, in particular, assess how these have been used and the outcomes of such activity.

A scoping review using Arskey and O’Malley’s five stage framework was undertaken to examine existing evaluation frameworks claiming to evaluate education interventions.

Forty five articles were included in the review. A majority of papers concentrate on learner satisfaction and/or learning achieved. Rarely is a structured framework mentioned, or detail of the approach to analysis cited. Typically, evaluations lacked baseline data, control groups, longitudinal observations and contextual awareness.

This review has implications for those involved in designing and evaluating work-related education programmes, as it identifies areas where evaluations need to be strengthened and recommends how existing frameworks can be combined to improve how evaluations are conducted.

This scoping review is novel in its assessment and critique of evaluation frameworks employed to evaluate work-related education programmes.

This paper aims to investigate the e-waste aspect of sustainability in education, with a specific interest in engineering education. Specifically, it focuses on recycling through reclaiming electronic components from e-waste and reusing them in repairs or in the design and construction of new devices.

A systematic literature review is performed according to the PRISMA methodology. In total, 27 articles are analysed as to publication parameters, characteristics and evaluation of educational interventions on e-waste and evaluation results across major domains of learning (cognitive, affective and 21st century skills).

The reviewed subject is under-research; publications are rare and mostly in conference proceedings. The majority of interventions take place at university level, in face-to-face mode, using a practical approach in hands-on labs. Educational methods draw from modern, learner-centred pedagogies such as collaborative learning and constructionism. Topics focus on innovative design and construction, while interventions tend to become embedded in engineering curricula/courses. Evaluation of learners’ gains across domains of learning is rare and follows informal procedures that shake the reliability of results. Domains other than the cognitive are scarcely and subjectively evaluated.

Contrary to other aspects of sustainability, the aspect of e-waste has not been reviewed. The applied, hands-on approach and the analytic, synthetic, collaboration and creativity skills it requires are all much valued in education. The current review, therefore, comes to inform, inspire and guide educators and researchers in planning and implementing activities on this subject.

Interventions that improve timely access to outpatient health services are essential in managing demand. This process evaluation aimed to describe the implementation, mechanism of impact and context of an intervention to reduce waiting for first appointments in an outpatient epilepsy clinic.

The UK Medical Research Council framework was used as the theoretical basis for a process evaluation alongside an intervention trial. The intervention, Specific Timely Appointments for Triage (STAT), is a data-driven approach that combines a one-off backlog reduction strategy with methods to balance supply and demand. A mixed methods process evaluation synthesised routinely collected quantitative and qualitative data, which were mapped to the domains of implementation, mechanisms of impact and contextual elements.

The principles of the STAT model were implemented as intended without adaptation. The STAT model reached all patients referred, including long waiters and was likely generalisable to other medical outpatient clinics. Mechanisms of impact were increased clinic capacity and elimination of unwanted variation. Contextual elements included the complexity of healthcare systems and the two-tier triage practice that contributes to prolonged waiting for patients classified as non-urgent.

This process evaluation shows how a data-driven strategy was applied in a medical outpatient setting to manage demand. Improving patient flow by reducing waiting in non-urgent, outpatient care is a complex problem. Understanding how and why interventions work is important for improved timeliness of care, and sustainability of public health services.

In recent years, a wide range of psychosocial health interventions have been implemented among military service members and their families. However, there are questions over the evaluative rigor of these interventions. We conducted a systematic review of this literature, rating each relevant study (k = 111) on five evaluative rigor scales (type of control group, approach to participant assignment, outcome quality, number of measurement time points, and follow-up distality). The most frequently coded values on three of the five scales (control group type, participant assignment, and follow-up distality) were those indicating the lowest level of operationally defined rigor. Logistic regression results indicate that the evaluative rigor of intervention studies has largely remained consistent over time, with exceptions indicating that rigor has decreased. Analyses among seven military sub-populations indicate that interventions conducted among soldiers completing basic training, soldiers returning from combat deployment, and combat veterans have had, on average, the greatest evaluative rigor. However, variability in mean scores across evaluative rigor scales within sub-populations highlights the unique methodological hurdles common to different military settings. Recommendations for better standardizing the intervention evaluation process are discussed.

Football exercise as an intervention for people with severe mental health problems has seen an increasing interest in the past years. To date, there is, however, no comprehensive review of the empirical evidence regarding the effectiveness of these interventions. In this review, the authors have comprised the research findings from the peer-review literature as well as the theoretical approaches to football exercise as an adjunct treatment. This overview will be informative to everybody who is planning to develop a football intervention for this population as well as to the people who are preparing evaluation studies that measure the effectiveness of such interventions. The paper aims to discuss these issues.

The authors identified research papers in the peer-review literature that feature empirical findings on “football interventions” that aim at improving mental and/or physical well-being in participants with mental health problems. The authors are using the term “football intervention” here in the sense that the participants actively took part in football exercise, so the authors excluded studies in which the participants only watched football or used football as a metaphor to discuss mental health problems. In a table, the authors indicate the definition of the target group, targeted outcomes, measured outcomes, form and frequency of the intervention as well as the research method(s).

The authors identified 16 studies on 15 projects. The majority of studies were qualitative and had positive findings in which the participants reported increased well-being and connectedness, elevation of symptoms and improved physical well-being. The outcomes of the quantitative studies, however, were mixed with some results suggesting that not all intended goals were achieved. There seems to be a need for more quantitative studies to triangulate the qualitative findings. Interestingly, most interventions take place in the UK. Many studies fail to give detailed methodological information and often the aims of the interventions are vague or not stated at all.

Due to the heterogeneity of the studies and relative scarcity of evaluation projects on football interventions for people with mental health problems, the authors could not conduct an in-depth systematic review. Furthermore, the information on methods was often unsatisfying and despite efforts to get more detailed input from the authors of cited papers, those gaps could not always be filled. Instead of coming up with a crystal-clear summary of whether and how football interventions work for everybody, topics were identified that need to be addressed in the planning of interventions, in evaluation studies, in implementation efforts and in the theoretical discourse.

This paper constitutes a helpful overview for everybody who is interested in the theoretical background of football interventions for people with mental health problems, for people who are planning to develop respective interventions, for researchers who engage in evaluation projects that look into the effectiveness of football interventions (or similar exercise interventions) as well as for the people who are interested in how football interventions can be implemented. This paper is likely to make a contribution to the advancement of alternative exercise interventions that aim at improving mental, physical and social health in people with mental health problems.

This paper will help putting the topic of football interventions (and similar, alternative exercise interventions) further up on the public health agenda by providing an overview of the empirical evidence at hand and by specifying advantages of the approach as well as pointing out actions that need to be taken to make football a recognised, evidence based and viable option for adjunct mental health treatment that is attractive to potential participants as well as funders as well as to the potential participants.

There is no comprehensive summary to date that provides a (reasonably) systematic overview of empirical findings for football interventions for people with MH problems. Furthermore, the literature on the theoretical background of these interventions has been somewhat patchy and heterogonous. This paper aims at filling both these gaps and identifies the issues that need to be covered in the planning of respective interventions and evaluations. This paper will be useful to everybody who is developing football interventions (or similar alternative adjunct exercise interventions), who is conducting evaluation research in this area and who is interested in the implementation of football interventions.

This paper reports on the implementation and evaluation of the JOBS programme in Ireland. This is a training intervention to promote re‐employment and improve mental health among unemployed people that was implemented on a pilot basis in the border region of the Republic and Northern Ireland. Programme participants were unemployed people recruited from local training and employment offices and health agencies. The evaluation indicated that the programme was implemented successfully and led to improved psychological and re‐employment outcomes for the intervention group, lasting up to 12 months post‐intervention. This paper reflects on the implementation issues that arose in adapting an international evidence‐based programme to the local setting and considers the implications of the evaluation findings for the roll out of the programme on a larger scale.