The Reflective Fostering Programme fidelity rating scale: development and inter-rater reliability
Journal of Children's Services
ISSN: 1746-6660
Article publication date: 13 February 2023
Issue publication date: 7 March 2023
Abstract
Purpose
The purpose of this study is to describe the development of the 14-item reflective fostering fidelity rating (RFFR), an observational rating system to evaluate model fidelity of group facilitators in the Reflective Fostering Programme (RFP), a mentalisation-based psychoeducation programme to support foster carers. The authors assess usability, dimensionality, inter-rater reliability and discriminative ability of the RFFR.
Design/methodology/approach
Eighty video clip extracts documenting 20 RFP sessions were independently rated by four raters using the RFFR. The dimensionality of the RFFR was assessed using principal components analysis. Inter-rater agreement was assessed using the intra-class correlation coefficient.
Findings
The proportion of missing ratings was low at 2.8%. A single principal component summarised over 90% of the variation in ratings for each rater. The inter-rater reliability of individual item ratings was poor-to-moderate, but a summary score had acceptable inter-rater reliability. The authors present evidence that the RFFR can distinguish RFP sessions that differ in treatment fidelity.
Originality/value
To the best of the authors’ knowledge, this is the first investigation and report of the RFFR’s validity in assessing the programme fidelity of the RFP. The paper concludes that the RFFR is an appropriate rating measure for treatment fidelity of the RFP and useful for the purposes of both quality control and supervision.
Keywords
Acknowledgements
The reflective fostering study from which the data was drawn for this study was funded by Klynveld Peat Marwick Goerdeler (KPMG) and the Segelman trust but these organisations did not act as sponsors for the research work or contribute to any decisions made as part of the research process.
Citation
Midgley, N., Cirasola, A., Sprecher, E.A., Redfern, S., Wright, H., Rider, B. and Martin, P. (2023), "The Reflective Fostering Programme fidelity rating scale: development and inter-rater reliability", Journal of Children's Services, Vol. 18 No. 1, pp. 18-46. https://doi.org/10.1108/JCS-01-2022-0002
Publisher
:Emerald Publishing Limited
Copyright © 2023, Emerald Publishing Limited