Search results

The purpose of this study is to describe the development of the 14-item reflective fostering fidelity rating (RFFR), an observational rating system to evaluate model fidelity of group facilitators in the Reflective Fostering Programme (RFP), a mentalisation-based psychoeducation programme to support foster carers. The authors assess usability, dimensionality, inter-rater reliability and discriminative ability of the RFFR.

Eighty video clip extracts documenting 20 RFP sessions were independently rated by four raters using the RFFR. The dimensionality of the RFFR was assessed using principal components analysis. Inter-rater agreement was assessed using the intra-class correlation coefficient.

The proportion of missing ratings was low at 2.8%. A single principal component summarised over 90% of the variation in ratings for each rater. The inter-rater reliability of individual item ratings was poor-to-moderate, but a summary score had acceptable inter-rater reliability. The authors present evidence that the RFFR can distinguish RFP sessions that differ in treatment fidelity.

To the best of the authors’ knowledge, this is the first investigation and report of the RFFR’s validity in assessing the programme fidelity of the RFP. The paper concludes that the RFFR is an appropriate rating measure for treatment fidelity of the RFP and useful for the purposes of both quality control and supervision.

Evaluation of any given student's responsiveness to intervention depends not only on how effective the intervention is, but also whether the intervention was delivered as intended as well as in the appropriate format and according to the most useful schedule. These latter elements are referred to as treatment integrity and treatment intensity, respectively. The purpose of this chapter is to define and describe how treatment integrity and intensity can be incorporated in the evaluation of outcomes associated with individualized intervention delivery.

Two common methods in community settings of assessing program fidelity, a critical implementation component for program effectiveness, are video and audio recordings of sessions. This paper aims to examine how these two methods compared when used for a home-based behavioral parenting-training model (SafeCare®).

Twenty-five SafeCare video-recorded sessions between home visitors and parents were scored by trained raters either using the video or audio-only portions of recordings. Sessions were coded using fidelity checklists, with items (n = 33) classified as one of two fidelity aspects, content [delivery of program components (n = 15)], or process [communication and rapport building (n = 11)]. Seven items were considered to overlap between constructs. Items were coded as having been done or not done appropriately. Coders rated items as “technological limitation” when scoring methods hindered coding. Analyses compared percent agreement and disagreement between audio and video coders.

Overall agreement between coders was 72.12%. Levels of agreement were higher for content items (M = 80.89%, SD = 19.68) than process items (58.54%, SD = 34.41). Disagreements due to technology limitations among audio coders were noted among 15 items; particularly, higher levels of disagreement were seen among process items (42.42%) than content items (9.64%).

Compared to video, fidelity monitoring via audio recordings was associated with some loss of process-related fidelity. However, audio recordings could be sufficient with supplements such as participant surveys, to better capture process items. Research should also examine how content and process fidelity relate to changes in family behavior to further inform optimal fidelity monitoring methods for program use.

The purpose of this paper is to test a mental wellness intervention, Mood Lifters (ML), that addresses significant barriers to mental health care. ML includes adults over 18 struggling with mental wellness or any life difficulties, except those with active suicidality, mania and psychosis, and addresses barriers to care using peer leaders in a manualized group format with a gamified point system.

Participants were recruited using online postings. Those eligible (76% female, 80% white) were randomly assigned to professional-led groups (N = 30), peer-led groups (N = 33) or a waitlist (N = 22; i.e. attended assigned condition if available). Participants completed pre- and postgroup measures (including the Patient Health Questionnaire-9, Generalized Anxiety Disorder-7 and Perceived Stress Scale), attended 15 weekly meetings and tracked “points” or at-home skills practice. Multiple imputation was used to account for attrition. Linear regressions were analyzed to determine the program’s impact on anxiety and depressive symptoms and perceived stress. Further analyses included comparisons between peer- and professional-led groups.

Participants in ML experienced significant reductions in anxiety symptoms. Completing more homework across the program led to significant reductions in anxiety and perceived stress. Finally, there were no significant differences in attendance, homework completed or outcomes between peer- and professional-led groups.

Overall, participation in the ML program led to reduced anxiety symptoms, and for those who completed more homework, reduced perceived stress. More accessible programs can make a significant impact on symptoms and are critical to address the overburdened care system. Additionally, there were no differences between leader types indicating that peers may be an effective way to address accessibility concerns.

ML is unique for three reasons: it takes a biopsychosocial/Research Domain Criteria approach to mental wellness (i.e. incorporates many areas relevant to mental health, does not focus on a specific diagnosis), overcomes major barriers to mental health care and uses a peer-delivery model. These attributes, taken together with the results of this study, present a care alternative for those with less access.

Many individuals in drug treatment courts (DTCs) have co-occurring mental health and substance use disorders (COD), which can negatively impact treatment engagement, behavioral health and criminal justice outcomes. This paper aims to report results of DTC participants with a COD, who received a 12-month wraparound treatment intervention called MISSION-Criminal Justice (MISSION-CJ) alongside DTC to improve treatment engagement and behavioral health outcomes and reduce reincarcerations.

In this pre-post, single-group pilot, 48 clients enrolled and 81% completed 12-month follow-up assessments (N = 39) and weekly MISSION-CJ fidelity for type and intensity of services delivered. Generalized linear mixed models (GLMMs) were computed with a fixed term for fidelity (e.g. high or low MISSION-CJ), time and a fidelity x time interaction term.

Among participants, at 12 months, 81% of the participants remained engaged in treatment at study completion, and 89% had high MISSON-CJ fidelity. Clients demonstrated significant reductions from baseline to 12 months in average nights in jail (B = −0.1849511, p < 0.0344), mental health symptoms via the Behavior and Symptom Identification Scale (BASIS) total and subscale scores (B = −0.121613, p < 0.0186) and trauma symptoms on the PTSD Checklist-5 (PCL-5) (B = −0.928791, p < 0.0138). High MISSION-CJ fidelity further improved criminal justice, and behavioral health outcomes.

This was the first reported 12-month MISSION-CJ trial. While feasible to implement, given the design limitations, future research should include a large randomized controlled trial.

The positive outcomes demonstrated in programme efficacy trials and the apparent ineffectiveness of programmes in community settings have prompted investigators and practitioners to examine implementation fidelity. Critically important, but often overlooked, are the implementers who deliver evidence‐based programmes. This article distinguishes fidelity at the programme level from implementer fidelity. Two components of implementer fidelity are defined. It is proposed that implementer adherence and competence are related but unique constructs that can be reliably measured for training, monitoring and outcomes research. Observational measures from a school‐based preventive intervention are used to illustrate the contributions of implementer adherence and competence. Distinguishing implementer adherence to the manual from competence in programme delivery is the next step in child mental health programme implementation research.

Evidence‐based practices (EBPs) are increasingly being implemented in real‐world settings. While intervention effectiveness is dependent on fidelity, interventions are often adapted to service settings according to the needs of stakeholders at multiple levels. This study aims to examine the naturalistic implementation of The Incredible Years (IY) parenting programme in a residential substance abuse treatment programme for pregnant and parenting women.

The study took place in a residential substance abuse treatment programme serving pregnant and parenting women and their children. Participants included 120 female clients. The primary IY facilitator was a master's level counselling psychologist. In person observations of IY sessions were completed by a trained bachelor's level anthropologist. Ethnographic field notes were collected and then coded in keeping with a priori themes and to identify emergent themes. The Parent Group Leader Checklist was used to evaluate quality and integrity of the IY basic parent programme.

Quantitative analyses indicate that fidelity varied by type of checklist activity. Specifically, adherence to the IY programme was highest in beginning topic activities, setup, and home activity review, and lowest in role play, vignettes, and wrap‐up activities. Qualitative analyses revealed a number of adaptations in implementation of IY. Adaptations fit into two broad categories: modification of programme delivery and modification of programme content. Within each of these categories modifications included organisation‐driven adaptations, provider‐driven adaptations, and consumer‐driven adaptations.

Changes to evidence‐based practice generally take two forms – adaptations consistent with model intent and theoretical approach and changes that represent drift from core elements of the EBP. The challenge for implementation science is to develop frameworks in which models can be adapted enough to make them viable for the service context (or the service context adapted to fit the model), yet avoid drift and maintain fidelity. Attending to the complexities of adaptation prior to and during implementation in a planned way is likely to help organisations better utilise EBPs to meet their unique needs while maintaining fidelity.

The paper shows that identification of types of intervention adaptations and drift allows for consideration of systematic approaches, frameworks, and processes to increase adherence during EBP implementation in community mental health and substance abuse treatment settings.

The purpose of this study was to investigate the quality of current studies that assess and treat anxiety symptoms in children with autism spectrum disorder (ASD). More specifically the study aimed to answer the following questions: What are the qualities of the current studies using cognitive behavioral therapies (CBTs) to treat anxiety symptoms in children with ASD? Did studies make necessary modifications and adaptations to CBTs according to the evidence-based strategies and implement these versions of CBTs with precise fidelity? Were the selected measurements appropriate for assessing the anxiety symptoms in children with ASD?

A systematic review protocol was developed from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (Moher et al., 2009). A rubric was adapted based on the CEC (Council for Exceptional Children; Cook, 2014) group comparison studies standards and the Evaluative Method for Determining EBP in Autism (Reichow et al., 2007). The 3-point Likert Scale (Chard et al., 2009) was adapted to score each study based on the rubric.

CBT is a first-line treatment with significant mixed results. Current studies use adapted versions of existing CBTs for children with ASD without reporting empirical evidence to these adaptations and changes. Reporting of the implementation fidelity is still an issue in the treatment of anxiety. Anxiety measurements that were designed for typically developing children failure to detect unusual anxiety symptoms in children with ASD.

The first limitation of this study was including a variety of studies across CBT programs and types of anxiety symptoms. Types of anxiety and CBT treatments may require separate analyses with specific indicators. Due to the limited studies, reviews could not be analyzed across types of CBT programs. The second limitation was the types of studies. Most of the studies were pilot studies. Pilot studies might use various instruments and CBTs components for making selections to produce the best effects and results. The final limitation was the lack of examination of the data analysis process.

These findings are important because due to the variety of changes or adaptation to CBTs, inappropriate implementations and failure to detect unusual anxiety symptoms of children with ASD may cause significant differences in treatment responses and outcomes. The study demonstrated that the majority of the studies used adapted versions of existing CBTs without reporting empirical evidence for these adaptations and changes. The findings have shown that reporting of the implementation fidelity is still an issue. Moreover, the majority of studies had used anxiety measurements that were designed for typically developing children, not for children with ASD.

Functional assessment-based interventions are a tertiary support that have been incorporated in many three-tiered models of prevention to support students who do not respond to more global prevention efforts. Although endorsed by host of reputable organizations (e.g., National Association of School Psychologists) and mandated in Individuals with Disabilities Education Act (IDEA, 1997, 2004), concerns have been raised that this mandate may not be warranted if functional assessment-based interventions do not meet minimum criteria to establish this as an evidence-based practice. One issue contributing to this concern is variability in the functional assessment process. John Umbreit and colleagues (2007) have attempted to address this concern by introducing a systematic approach that includes (a) a Function Matrix to analyze functional assessment data and identify the hypothesized function(s) of the target behavior and (b) a Function-Based Intervention Decision Model to guide intervention planning. In this chapter, we applied the core quality indicators for single-case research developed by Horner, Carr, Halle, McGee, Odom, and Wolery (2005) to studies conducted using this practice to determine the extent to which this systematic approach to functional assessment-based interventions met the standards for evidence-based practices for use in educational settings across the K-12 continuum for students with or at-risk for high incidence disabilities. If this practice is deemed to meet criteria, then this systematic approach may be particularly useful in meeting the mandate established in IDEA. Results suggest that it may be appropriate to establish this systematic method as a promising practice.

Tiered systems may hold particular benefit for students with intensive intervention needs, as this continuum of supports is ideally comprised of evidence-based practices. In this chapter, we explore three intensive interventions: functional assessment-based interventions (FABI), First Step to Success, and Read 180 as they feature a range of practices and programs to meet students' academic, behavioral, and social needs. We define and describe each intervention, including information on essential features necessary for drawing valid inferences: treatment integrity, social validity, as well as student performance. We also provide examples of supporting evidence, featuring treatment-outcome studies documenting intervention effectiveness. We close with clarifications and considerations for meeting the multiple needs of students requiring intensive intervention efforts.