Search results

The need for randomised controlled trials (RCTs) to support evidence‐based services to improve outcomes for children is increasingly recognised by researchers and policy‐makers. However, this brings a pressing requirement to build research capacity for conducting RCTs and to address the concerns of practitioners who may be suspicious about the method. This article reviews a variety of texts on the subject, ranging from analyses of the historical and political context of RCTs, to concise introductions of the key methodological and practical issues, to more in‐depth discussions of complex designs and statistics. The article seeks to help readers navigate these resources by focusing on seven questions that seem particularly salient for those considering whether and how to commission, undertake, participate in or use results from RCTs.

The chapter considers the change of position of the Home Office on the value of randomised controlled trials (RCTs) in England and Wales which took place around 2003 after the end of the Crime Reduction Programme (CRP). Before the CRP Home Office researchers had shown little interest in RCTs; after it, they came close to arguing that no other kinds of evaluation research were worth doing. This represented a reversal of a position that had dominated Home Office thinking on the issue for almost 30 years – that RCTs were in general impractical and unlikely to produce clear-cut results. This view was based in part on the experience of RCTs in the 1970s, which led influential researchers to conclude that the method could not be transferred from medicine to criminal justice. But, disappointed with the lack of definite results from the CRP, the Home Office turned back to RCTs as a potential source of certainty about what works. The chapter considers two recent scholarly exchanges on the question, in relation to an evaluation of a community crime reduction programme, for which an experimental design was attempted but not achieved, and to Lawrence Sherman's recent advocacy of RCTs and his use of research on restorative justice as an example of the successful use of the method. The chapter argues that the restorative justice research, while of very high quality, does not provide as clear an example of the use of an RCT as Sherman claims, and concludes with some reflections on the inherent difficulties of criminal justice evaluation, and on the lack of a predictable, rational relationship between research quality and policy influence.

The article's purpose is to critique a paper (Stewart‐Brown et al., 2011) in a previous issue of the Journal of Children's Services, which challenges the utility of randomised controlled trials (RCTs) to evaluate the effectiveness of preventive interventions for children.

The article is a critical reflection on the primary issue discussed by Stewart‐Brown et al. namely that RCTs do not work well in the evaluation of complex social interventions.

The author finds fault with several of the claims made in the earlier essay and concludes that RCTs remain the most credible research methodology for estimating programme impacts. It is certainly true that RCTs do not tell us everything about programmes and implementation. However, if researchers are attempting to assess whether social interventions have the intended measured impact on their participants, then RCTs do indeed represent the “gold standard” research design.

The article is a re‐assertion of the value of RCTs in research on preventive interventions in children's services.

In this chapter, we examined issues related to research design and research management as applied to scientific research conducted in applied school settings. In terms of research design, randomized controlled trials (RCTs) have an important role to play in advancing a scientific agenda for school-based research. However, given their enormous cost and complexity, it is important to carefully time their implementation in the development cycle. We suggested that the use of RCTs is most appropriate in the later stages of the development cycle when the focus is on demonstrating the efficacy and/or effectiveness of an intervention and establishing its generalizability under the real world conditions of schooling. We also recommended establishing a hierarchy of evidence for an intervention that involves implementing a cost-efficient mix of single case, quasi-experimental, and true experimental designs where appropriate and feasible. In examining issues related to the management of research and the implementation of a knowledge development agenda for schools, it has become apparent that treatment integrity is a keystone variable. We discussed the importance of treatment integrity, with attention to the impact on internal and external validity. Finally, we offered practical considerations to support high-quality, respectful school-based inquiry.

The purpose of this article is to provide several viewpoints about monitoring aspects related to recent deployments of a new technology (IPv6).

Several views and domains were used, with a common point: the Portuguese research and education network (RCTS).

A significant amount of work is yet to be done, in order to mature the deployment of this new internet technology.

The equipment whence the data were collected still has some limitations regarding the new technology. Future datasets may benefit from wider deployments.

The work also demonstrates that IPv6 deployment is in its early stages, which is negative, given the projected dates for IPv4 exhaustion.

The findings and the work described will be useful for people trying to deploy IPv6 networks in the short or medium timeframe.

The paper's purpose is to participate in a debate about the role of randomised controlled trials in evaluation of preventive interventions for children.

The paper is a response to critiques on Stewart‐Brown et al. published in the Journal of Children's Services, Vol. 6 No. 4, pp. 228–35.

Randomised controlled trials are likely to be at their best in the evaluation of interventions that do not require the active engagement and personal development of participants. The latter may depend on a series of interventions and events that potentiate each other over time. Randomised controlled trials are likely to be least valuable in evaluating universal level interventions that aim to change population norms. Because of the challenges involved in conducting RCTs in this setting they cannot be relied upon to give accurate estimates of programme effect and therefore do not deserve the privileged position that has been accorded them in the hierarchy of evidence.

This paper develops the argument that the privileged position of RCTs in the evidence hierarchy of preventive services for children is undeserved.

Randomised controlled trials (RCTs) are considered the bedrock of evidence‐based practice. However, they raise important issues about participant recruitment and the ethics of group allocation. In this article we report relevant literature on how participants with learning disabilities and other stakeholders experience RCTs. Five quantitative studies have been published, one reporting on the views of people with learning disabilities and the remainder on carers' views. Despite the methodological limitations of the studies, carers appear to have a positive experience of RCT participation, and people with learning disabilities are able to understand certain aspects of research methods. We conclude that stakeholders are not hostile to RCTs, but the therapeutic misconception may affect their experiences. Further research is required to investigate interventions that might help promote RCTs in learning disabilities.

This paper's aim is to explore the uses and limitations of randomised controlled trials (RCTs) for evaluating complex interventions, with a particular focus on sample recruitment and retention issues.

This is an invited critique of a previous paper.

RCTs have many limitations. It is particularly important to consider issues relating to the sample they recruit and retain. Nonetheless, they remain a uniquely powerful way to exclude other potential explanations for outcomes and therefore provide robust evidence for the effectiveness of specific interventions.

It is hoped that vigorous debate may contribute to a deepened understanding of the nature, limitations and potential contribution of RCTs to understanding the impact of different ways of helping people.

Though we have recently witnessed the “exponential production of digital data to measure, analyze, and predict educational performance” (Salajan & Jules, this volume), there has not been sufficient attention given to the quantitative methods that are used to process and transform this data in order to arrive at findings related to “what works”. This chapter addresses this gap by discussing a range of constraints that affect the main methods used for this purpose, with these methods being known as “impact evaluation.” Specifically, this chapter addresses its purpose, first, by making explicit the methodological assumptions, technical weaknesses, and practical shortcomings of the two main forms of impact evaluation—regression analysis and randomized controlled trials. Although the idea of Big Data and the ability to process it is receiving more attention, the underlying point here is that these new initiatives and advances in data collection are still dependent on methods that have serious limitations. To that end, not only do proponents of Big Data avoid or downplay discussion of the methodological pitfalls of impact evaluation, they also fail to acknowledge the political and organizational dynamics that affect the collection of data. To the extent that such methods will increasingly be used to guide public policy around the globe, it is essential that stakeholders inside and outside education systems are informed about their weaknesses—methodologically and in terms of their inability to take the politics out of policymaking. While the promises of Big Data are seductive, they have not replaced the human element of decision making.

Randomised controlled trials (RCTs) have been offered a privileged position in terms of the evidence base for preventive interventions for children, but practical and theoretical issues challenge this research methodology. This paper aims to address this issue.

This paper analyses practical and methodological issues of using RCTs within children's preventive services and presents the results of a qualitative study using data collected from parents who were asked to take part in an RCT of a preventive intervention.

Well recognised issues include the impossibility of blinding participants, the problem of identifying a pre‐eminent outcome measure for complex interventions, and problems with limiting access to equivalent interventions in real world settings. A further theoretical problem is the exclusion from RCTs of families who are most ready to change, resulting in a reduced level of intervention effectiveness. Qualitative evidence from one recent RCT suggests that this problem could be operating in some prevention trials. Increasing sample sizes can overcome some of these problems, but the cost of the necessarily huge trials becomes disproportionate to the intervention?

Given the limitations on RCTs in preventive settings, the paper argues their privileged position in terms of research evidence maybe undeserved.