Search results

There are five factors acting as a barrier to the effective evaluation of educational technology (edtech), which are as follows: premature timing, inappropriate techniques, rapid change, complexity of context and inconsistent terminology. The purpose of this paper is to identify new evaluation approaches that will address these and reflect on the evaluation imperative for complex technology initiatives.

An initial investigation of traditional evaluative approaches used within the technology domain was broadened to investigate the evaluation practices within social and public policy domains. Realist evaluation, a branch of theory-based evaluation, was identified and reviewed in detail. The realist approach was then refined, proposing two additional necessary steps to support mapping the technical complexity of initiatives.

A refined illustrative example of a realist evaluation framework is presented, including two novel architectural edtech domain reference models to support mapping.

Recommendations include building individual evaluator capacity; adopting the realist framework; the use of architectural edtech domain reference models; phased evaluation to first build theories in technology “context” and then iteratively during complex implementation chains; and community contribution to a shared map of technical and organisational complexity.

This paper makes a novel contribution by arguing the imperative for a theory-based realist approach to help redefine evaluative thinking within the IT and complex system domain. It becomes an innovative proposal with the addition of two domain reference models that tailor the approach for edtech. Its widespread adoption will help build a shared evidence base that synthesizes and surfaces “what works, for whom, in which contexts and why”, benefiting educators, IT managers, funders, policymakers and future learners.

To ensure that more people will benefit from integrated care initiatives, scaling-up of successful initiatives is the way forward. However, new challenges present themselves as knowledge on how to achieve successful large-scale implementation is scarce. The EU-funded project SCIROCCO uses a step-based scaling-up strategy to explore what to scale-up, and how to scale-up integrated care initiatives by matching the complementary strengths and weaknesses of five European regions involved in integrated care. The purpose of this paper is to describe a multi-method evaluation protocol designed to understand what factors influence the implementation of the SCIROCCO strategy to support the scaling-up of integrated care.

The first part of the protocol focuses on the assessment of the implementation fidelity of the SCIROCCO step-based strategy. The objective is to gain insight in whether the step-based strategy is implemented as it was designed to explore what works and does not work when implementing the scaling-up strategy. The second part concerns a realist evaluation to examine what it is about the SCIROCCO’s strategy that works for whom, why, how and in which circumstances when scaling-up integrated care.

The intended study will provide valuable information on the implementation of the scaling-up strategy which will help to explain for what specific reasons the implementation succeeds and will facilitate further improvement of project outcomes.

The expected insights could be useful to guide the development, implementation and evaluation of future scaling-up strategies to advance the change towards more sustainable health and care systems.

A range of strategies to improve pharmaceutical care has been implemented by population health management (PHM) initiatives. However, which strategies generate the desired outcomes is largely unknown. The purpose of this paper is to identify guiding principles underlying collaborative strategies to improve pharmaceutical care and the contextual factors and mechanisms through which these principles operate.

The evaluation was informed by a realist methodology examining the links between PHM strategies, their outcomes and the contexts and mechanisms by which these strategies operate. Guiding principles were identified by grouping context-specific strategies with specific outcomes.

In total, ten guiding principles were identified: create agreement and commitment based on a long-term vision; foster cooperation and representation at the board level; use layered governance structures; create awareness at all levels; enable interpersonal links at all levels; create learning environments; organize shared responsibility; adjust financial strategies to market contexts; organize mutual gains; and align regional agreements with national policies and regulations. Contextual factors such as shared savings influenced the effectiveness of the guiding principles. Mechanisms by which these guiding principles operate were, for instance, fostering trust and creating a shared sense of the problem.

The guiding principles highlight how collaboration can be stimulated to improve pharmaceutical care while taking into account local constraints and possibilities. The interdependency of these principles necessitates effectuating them together in order to realize the best possible improvements and outcomes.

This is the first study using a realist approach to understand the guiding principles underlying collaboration to improve pharmaceutical care.

While sustainability-oriented education is increasingly placing importance on engaging students in inter- and transdisciplinary learning processes with societal actors and authentic challenges in the centre, little research attends to how and what students learn in such educational initiatives. This paper aims to address this by opening the “black box” of learning in a Challenge Lab curriculum with transformational sustainability ambitions.

Realist evaluation was used as an analytical frame that takes social context into account to unpack learning mechanisms and associated learning outcomes. A socio-cultural perspective on learning was adopted, and ethnographic methods, including interviews and observations, were used.

Three context-mechanism-outcome (CMO) configurations were identified, capturing what students placed value and emphasis on when developing capabilities for leading sustainability transformations: engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives; navigating purposeful and transformative change via backcasting; and “whole-person” learning from the inside-out as an identity-shaping process, guided by personal values.

The findings of this paper can inform the design, development, evaluation and comparison of similar educational initiatives across institutions, while leaving room for contextual negotiation and adjustment.

This paper delineates and discusses important learning mechanisms and outcomes when students act as co-creators of knowledge in a sustainability-oriented educational initiative, working with authentic challenges together with societal actors.

The purpose of this paper is to evaluate the impact of leadership development programmes, underpinned by Transformational Learning Theory (TLT).

A corpus-informed analysis was conducted using survey data from 690 participants. Data were collected from participants’ responses to the question “please tell us about the impact of your overall experience”, which culminated in a combined corpus of 75,053 words.

Findings identified patterns of language clustered around the following frequently used word types, namely, confidence; influence; self-awareness; insight; and impact.

This in-depth qualitative evaluation of participants’ feedback has provided insight into how TLT can be applied to develop future health-care leaders. The extent to which learning has had a transformational impact at the individual level, in relation to their perceived ability to influence, holds promise for the wider impact of this group in relation to policy, practice and the promotion of clinical excellence in the future. However, the latter can only be ascertained by undertaking further realist evaluation and longitudinal study to understand the mechanisms by which transformational learning occurs and is successfully translated to influence in practice.

Previous research has expounded traditional leadership theories to guide the practice of health-care leadership development. The paper goes some way to demonstrate the impact of using the principles of TLT within health-care leadership development programmes. The approach taken by The Florence Nightingale Foundation has the potential to generate confident leaders who may be instrumental in creating positive changes across various clinical environments.

The purpose of this paper is to gain insight into how population health management (PHM) strategies can successfully integrate and reorganize public health, health care, social care and community services to improve population health and quality of care while reducing costs growth, this study compared four large-scale transformation programs: Greater Manchester Devolution, Vancouver Healthy City Strategy, Gen-H Cincinnati and Gesundes Kinzigtal.

Following the realist methodology, this explorative comparative case-study investigated PHM initiatives' key features and participants' experiences of developing such initiatives. A semi-structured interview guideline based on a theoretical framework for PHM guided the interviews with stakeholders (20) from different sectors.

Five initial program theories important to the development of PHM were formulated: (1) create trust in a shared vision and understanding of the PHM rationale to establish stakeholders' commitment to the partnership; (2) create shared ownership for achieving the initiative's goals; (3) create shared financial interest that reduces perceived financial risks to provide financial sustainability; (4) create a learning environment to secure initiative's credibility and (5) create citizens' and professionals' awareness of the required attitudes and behaviours.

The study highlights initial program theories for the implementation of PHM including different strategies and structures underpinning the initiatives. These insights provide a deeper understanding of how large-scale transformation could be developed.

The aims of the paper were to highlight the dearth of applied practitioner research concerning the expression of neurodiversity at work and develop an epistemological framework for a future research agenda.

A systematic empty review protocol was employed, with three a priori research questions, inquiring as to the extent of neurodiversity research within mainstream work psychology, psychology in general and lastly within cross-disciplinary academic research. The results of the final search were quality checked and categorized to illustrate where studies relevant to practice are currently located.

The academic literature was found to be lacking in contextualized, practical advice for employers or employees. The location and foci of extracted studies highlighted a growing science-practitioner gap.

The research focused on common neurominority conditions such as autism and dyslexia; it is acknowledged that the neurodiversity definition itself is broader and more anthropological in nature. A need for a comprehensive research agenda is articulated, and research questions and frameworks are proposed.

Guidance is given on applying disability accommodation to both individual and organizational targets.

The disability employment gap is unchanged since legislation was introduced. The neurodiversity concept is no longer new, and it is time for multi-disciplinary collaborations across science and practice to address the questions raised in this paper.

This paper offers an original analysis of the neurodiversity paradox, combining systematic inquiry with a narrative synthesis of the extant literature. The conceptual clarification offers clear directions for researchers and practitioners.