Search results

This paper aims to explore the use of developmental evaluation methods with community of practice programmes experiencing change or transition to better understand how to target support resources.

The practical use of a number of developmental evaluation methods was explored in three organizations over a nine-month period using an action research design. The research was a collaborative process involving all the company participants and the academic (the author) with the intention of developing the practices of the participants as well as contributing to scholarship.

The developmental evaluation activities achieved the objectives of the knowledge managers concerned: they developed a better understanding of the contribution and performance of their communities of practice, allowing support resources to be better targeted. Three methods (fundamental evaluative thinking, actual-ideal comparative method and focus on strengths and assets) were found to be useful. Cross-case analysis led to the proposition that developmental evaluation methods act as a structural mechanism that develops the discourse of the organization in ways that enhance the climate for learning, potentially helping develop a learning organization.

Developmental evaluation methods add to the options available to evaluate community of practice programmes. These supplement the commonly used activity indicators and impact story methods.

Developmental evaluation methods are often used in social change initiatives, informing public policy and funding decisions. The contribution here is to extend their use to organizational community of practice programmes.

This chapter insists that evaluation of the process and results of cocreation is a precondition for continuous improvement and helps maintain support from external sponsors and funders. The main benefits of systematic evaluation of cocreation are learning and legitimacy rather than control and allocation. The chapter scrutinizes the two most common evaluation tools, formative and summative evaluation, and finds that they both fail to appreciate the emergent character of cocreation processes. The solution to this problem is to supplement formative and summative evaluation with developmental evaluation, which prompts the participating actors to engage in a critical interrogation of what they are doing, the reasons for doing it, and the results they achieve. Finally, the chapter explains how the commitment of developmental evaluation to using real-time data in the evaluation of change theories can be pursued through a collective impact strategy.

The purpose of this paper is to discuss recent developments in school developmental supervisory evaluation in the Pudong New Area of Shanghai in the Chinese Mainland.

The main research approach is qualitative, using documentary analysis and interviews of an inspector, principals and teachers from two primary schools.

There were perceived positive and negative impacts of school supervision and evaluation.

The paper highlights the implications for fostering a shared school‐government community of school supervision and evaluation, promoting a dynamic approach for addressing contextual differences as well as achieving better coherence among educational reform, supervision and evaluation policies.

The purpose of this paper is to examine the applicability of Prochaska and colleagues' “stages of change model,” which has generated substantial support in the therapeutic literature as a useful framework for understanding the dynamics of motivation to change problem behaviors, in a leadership development context.

A group of over 70 supervisors/managers was studied over a period of nine months as they participated in a company‐sponsored leadership development effort.

Results provide initial evidence that the stages of change model has the potential for being reliably and validly assessed in a leadership development context. Participants' stage scores related in meaningful ways to relevant criteria such as job attitudes, perceptions of personal leadership areas needing improvement, and evaluations of actual development module content and presentation over a nine‐month period.

Participants were drawn from only one organization and this was the first major leadership development effort undertaken by this organization.

Study results provide support for the appropriateness of applying the stages of change model and its measurement in a leadership development context. Results demonstrate that the stages of change model appears to offer useful and pragmatic insight into motivation to learn and on improving the effectiveness of leadership development activities.

The present study is unique in that makes use of a stages of change model to empirically examine differential patterns of relationships between participants' stages of change and their organizational attitudes, leadership developmental needs, and longitudinal reactions to the development effort.

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish higher education, with particular reference to the longitudinal evaluative research of support of teaching and learning (T&L) (the Quality Enhancement Framework or QEF). The chapter will discuss the features of the research process which are shaped by evaluation theory. We adopt a theoretical position on policy research which foregrounds the situated experience of policy as a core research focus. Policy is depicted as being underscored by an implicit theory of change which is used to structure and orientate the research focus. The design of the research is characterised by the involvement of potential users of the research output, with implications on the way in which findings are articulated, presented and ultimately used, along with aspects of the evaluative research design. The case study of the QEF will be contextualised, and the intersection between the design features and theoretical approaches, and the use and usability of research outputs, will be established.

The purpose of this paper is to present a detailed case study of the evaluation strategies of a complex, multi-faceted response to a public health emergency: drug-related overdose deaths. It sets out the challenges of evaluating such a complex response and how they were overcome. It provides a pragmatic example of the rationale and issues faced to address the what, the why and particularly the how of the evaluation.

The case study overviews British Columbia’s Provincial Response to the Overdose Public Health Emergency, and the aims and scope of its evaluation. It then outlines the conceptual approach taken to the evaluation, setting out key methodological challenges in evaluating large-scale, multi-level, multisectoral change.

The evaluation is developmental and summative, utilization focused and system informed. Defining the scope of the evaluation required a strong level of engagement with government leads, grantees and other evaluation stakeholders. Mixed method evaluation will be used to capture the complex pattern of relationships that have informed the overdose response. Working alongside people with drug use experience to both plan and inform the evaluation is critical to its success.

This case study builds on a growing literature on evaluating large-scale and complex service transformation, providing a practical example of this.

Just as architects design houses, evaluators design evaluations. Although the way in which the actual construction of the building proceeds is of crucial importance, it is the design stage, which includes all the specifications for the types of materials to be used, as well as the aesthetic and structural qualities of the final building, which is critical to the well‐being of the project. There are important parallels which exist between the work of the architect and the work of the evaluator. Like the architect, the evaluator produces a design, but does not necessarily carry out what has been designed in practice—the implementation of the evaluation can be left to others. Equally, the evaluator, like the architect, often acts as a specialist consultant on behalf of a client.

Because language deficits are often present among individuals with pervasive developmental disorders, evaluation of language abilities is an essential part of a comprehensive assessment. According to Wetherby, Prizant, and Schuler (2000), assessment of language skills for persons with autistic spectrum disorders must focus on three broad areas of concern: linguistics, social and emotional communication, and language-related cognitive functions. Linguistic evaluation includes assessment of both expressive and receptive language. Examination of social and emotional communication must assess reciprocity of social interaction, understanding and expression of emotion, and regulatory strategies to manage emotional arousal. Finally, evaluation of language-related cognitive domains must include an assessment of the use of symbolic representations and imitation (Wetherby et al.).

This chapter explores the relevance of critical race theory (CRT) and queer theory to the relational aspects of program evaluation. Often conceptual binaries that undergird traditional evaluation theory and practice (e.g., internal versus external evaluation, subjective versus objective analysis, observation versus intervention, and insider versus outsider positionalities) adversely influence rigid social roles between evaluator and participant limit a study's effectiveness in supporting programs for equity in contemporary school districts. To illustrate this approach, an array of problems within a program evaluation of a district-wide ethnic studies reform initiative is presented. Approaches to these challenges rooted in tenets of CRT and queer theory illustrate how the district was able to clarify goals and develop an effective implementation plan that focused on effective ethnic studies curriculum and pedagogy.