Search results

Calls for the development and dissemination of evidence-based programs to support children and families have been increasing for decades, but progress has been slow. This paper aims to argue that a singular focus on evaluation has limited the ways in which science and research is incorporated into program development, and advocate instead for the use of a new concept, “scientific accompaniment,” to expand and guide program development and testing.

A heuristic is provided to guide research–practice teams in assessing the program’s developmental stage and level of evidence.

In an idealized pathway, scientific accompaniment begins early in program development, with ongoing input from both practitioners and researchers, resulting in programs that are both effective and scalable. The heuristic also provides guidance for how to “catch up” on evidence when program development and science utilization are out of sync.

While implementation models provide ideas on improving the use of evidence-based practices, social service programs suffer from a significant lack of research and evaluation. Evaluation resources are typically not used by social service program developers and collaboration with researchers happens late in program development, if at all. There are few resources or models that encourage and guide the use of science and evaluation across program development.

This article informs school leaders and staffs about existing research findings on the use of data-driven decision-making in creating class rosters. Given that teachers are the most important school-based educational resource, decisions regarding the assignment of students to particular classes and teachers are highly impactful for student learning. Classroom compositions of peers can also influence student learning.

A literature review was conducted on the use of data-driven decision-making in the rostering process. The review addressed the merits of using various quantitative metrics in the rostering process.

Findings revealed that, despite often being purposeful about rostering, school leaders and staffs have generally not engaged in data-driven decision-making in creating class rosters. Using data-driven rostering may have benefits, such as limiting the questionable practice of assigning the least effective teachers in the school to the youngest or lowest performing students. School leaders and staffs may also work to minimize negative peer effects due to concentrating low-achieving, low-income, or disruptive students in any one class. Any data-driven system used in rostering, however, would need to be adequately complex to account for multiple influences on student learning. Based on the research reviewed, quantitative data alone may not be sufficient for effective rostering decisions.

Given the rich data available to school leaders and staffs, data-driven decision-making could inform rostering and contribute to more efficacious and equitable classroom assignments.

This article is the first to summarize relevant research across multiple bodies of literature on the opportunities for and challenges of using data-driven decision-making in creating class rosters.

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid change, complexity of context and inconsistent terminology. The purpose of this paper is to identify new evaluation approaches that will address these and reflect on the evaluation imperative for complex technology initiatives.

An initial investigation of traditional evaluative approaches used within the technology domain was broadened to investigate the evaluation practices within social and public policy domains. Realist evaluation, a branch of theory-based evaluation, was identified and reviewed in detail. The realist approach was then refined, proposing two additional necessary steps to support mapping the technical complexity of initiatives.

A refined illustrative example of a realist evaluation framework is presented, including two novel architectural edtech domain reference models to support mapping.

Recommendations include building individual evaluator capacity; adopting the realist framework; the use of architectural edtech domain reference models; phased evaluation to first build theories in technology “context” and then iteratively during complex implementation chains; and community contribution to a shared map of technical and organisational complexity.

This paper makes a novel contribution by arguing the imperative for a theory-based realist approach to help redefine evaluative thinking within the IT and complex system domain. It becomes an innovative proposal with the addition of two domain reference models that tailor the approach for edtech. Its widespread adoption will help build a shared evidence base that synthesizes and surfaces “what works, for whom, in which contexts and why”, benefiting educators, IT managers, funders, policymakers and future learners.

The purpose of this paper is to analyze the evidence-based research strategy (EBRS) used to evaluate eight projects that applied the neighborhood approach for disaster risk reduction (NA-DRR) in informal urban settlements in Colombia, Guatemala, Haiti, Honduras, Jamaica and Peru, between 2012 and 2017.

The study covers the first five of the seven EBRS stages: first, identify relevant interventions; second, prepare evaluation questions; third, select evidence sources and implement a search strategy; fourth, appraise evidences and identify gaps; fifth, create an evaluation design to include an extensive literature review, followed by a mixed research method with surveys, focus groups and interviews; disaster risk modeling; georeferencing analysis; and engineering inspections. The last two stages: sixth, apply the evidence, and seventh, evaluate the evidence application, will be addressed in a near future.

Even though the reference to “evidence” is frequent in the DRR field, it is largely based on descriptive processes, anecdotal references, best practices, lessons learned and case studies, and particularly deficient on the subject of informal and precariousness settlements. The evaluation allowed a deep and broad analysis of NA-DRR in urban informal settlements, comparing it with other DRR strategies implemented by different stakeholders in fragile urban settings, assessing the effectiveness and sustainability of the various DRR interventions.

The abundant data, information and knowledge generated will serve as foundation for forthcoming thematic peer-reviewed publications informing evidence-based DRR research, policy and practice, with emphasis on informal and precariousness settlements in particular.

To elaborate the nature of fact-checking in the domain of political information by examining how fact-checkers assess the validity of claims concerning the Russo-Ukrainian conflict and how they support their assessments by drawing on evidence acquired from diverse sources of information.

Descriptive quantitative and qualitative content analysis of 128 reports written by the fact-checkers of Snopes – an established fact-checking organisation – during the period of 24 February 2022 – 28 June, 2023. For the analysis, nine evaluation grounds were identified, most of them inductively from the empirical material. It was examined how the fact-checkers employed such grounds while assessing the validity of claims and how the assessments were bolstered by evidence acquired from information sources such as newspapers.

Of the 128 reports, the share of assessments indicative of the invalidity of the claims was 54.7%, while the share of positive ratings was 26.7%. The share of mixed assessments was 15.6%. In the fact-checking, two evaluation grounds, that is, the correctness of information and verifiability of an event presented in a claim formed the basis for the assessment. Depending on the topic of the claim, grounds such as temporal and spatial compatibility, as well as comparison by similarity and difference occupied a central role. Most popular sources of information offering evidence for the assessments include statements of government representatives, videos and photographs shared in social media, newspapers and television programmes.

As the study concentrated on fact-checking dealing with political information about a specific issue, the findings cannot be extended to concern the fact-checking practices in other contexts.

The study is among the first to characterise how fact-checkers employ evaluation grounds of diverse kind while assessing the validity of political information.

The purpose of this paper is to report how an evaluation tool originally developed for Age-Friendly Cities was pilot-tested in the context of the Dementia Friendly Community (DFC) initiative of the city of Sheffield/UK. It presents finding and outputs on which other communities with dementia friendly agendas can draw.

The original evaluation tool was adapted to a focus on dementia friendliness. Data collection involved scoping conversations, documentary analysis, interviews and group discussions. Following evidence appraisal, Sheffield’s approach to dementia friendliness was assessed. A local steering group was central to the study.

The evidence indicates areas of strength in Sheffield’s approach to dementia friendliness: involvement of older people; service provision; collaboration; monitoring and evaluation. Scope for improvement was identified around resource allocation, and use of existing guidance on dementia friendliness. Recommendations for policy and practice include enhancing pooling of resources, more detailed recording of resources allocated to dementia-related activity, and collection of evidence on how people affected by dementia have shaped the city’s DFC initiative. Key research outputs are an adaptable logic model and an emerging evaluation framework for DFCs.

The study was a short pilot with limited resources. Its findings and outputs must be considered preliminary.

The findings and outputs provide a basis for further research. The study has suggested key components of an evaluation framework for DFCs. It is informing ongoing work to develop such a framework.