Search results

This paper seeks to address current limitations in approaches to training evaluation by presenting a conceptual model of work‐based learning and an associated evaluation framework.

The model and framework presented in this paper are based on a critical review of current approaches to learning evaluation and insights from learning transfer research and programme theory.

This paper sets out a conceptual model of workplace learning based on five elements: a pre‐learning stage, the trigger (need) for learning, the learning event, application of learning and the impact of learning. A linked criterion evaluation framework is also described. It is proposed that this provides a scientifically robust but practitioner friendly framework for workplace learning evaluation.

While most organisations wish to evaluate the effectiveness of their investment in employee training and development, few do. One of the barriers to effective learning evaluation is the failure to ground approaches in a contemporary and comprehensive model of workplace learning. The model and framework set out in this paper aim to assist evaluation by addressing this gap in a practitioner friendly way.

This paper sets out a novel, flexible and comprehensive conceptual model of workplace learning along with an innovative approach to training evaluation that addresses limitations in existing approaches. It is hoped that this will contribute to the debate on appropriate evaluation methods and assist practitioners to undertake evaluation in a more credible manner.

Reports on an effort to implement good practices in learning evaluation. Reviews learning evaluation practices and gathers data using a dedicated software system. Demonstrates learning takes place within complex social systems populated by a multiplicity of factors that influence perceptions of learning and performance outcomes. Argues that technology enables cost‐effective evaluations to be implemented that encompass a broad spectrum of influencing variables and acknowledge the empowered status of the learner. Discusses the implications for evaluation methodologies and the role of trainers within organisations.

This paper seeks to argue that workplace learning evaluation theory and practice is still an emergent field and that this creates a number of challenges for practitioners and researchers alike.

This is a descriptive paper based on a critical review of existing approaches and the research literature.

While programme evaluation has a long history, workplace learning evaluation is yet to establish itself as a distinct field. This has a number of consequences including the lack of a single or settled view on how workplace learning should be evaluated or what specific aspects of learning should be investigated.

The need to demonstrate a return on investment in organisational learning is as pressing as ever. To become more effective training evaluation methods need to be grounded in a theory. This article aims to provide an informed perspective on the current state of workplace evaluation along with insights into how evaluation can be placed on firmer theoretical foundations in order to produce robust findings in a practitioner friendly way.

This paper provides original insights into the development of workplace evaluation approaches and the challenges the field faces.

It has become accepted practice to include an evaluation alongside learning programmes that take place at work, as a means of judging their effectiveness. There is a tendency to focus such evaluations on the relevance of the intervention and the amount of learning achieved by the individual. The aim of this review is to examine existing evaluation frameworks that have been used to evaluate education interventions and, in particular, assess how these have been used and the outcomes of such activity.

A scoping review using Arskey and O’Malley’s five stage framework was undertaken to examine existing evaluation frameworks claiming to evaluate education interventions.

Forty five articles were included in the review. A majority of papers concentrate on learner satisfaction and/or learning achieved. Rarely is a structured framework mentioned, or detail of the approach to analysis cited. Typically, evaluations lacked baseline data, control groups, longitudinal observations and contextual awareness.

This review has implications for those involved in designing and evaluating work-related education programmes, as it identifies areas where evaluations need to be strengthened and recommends how existing frameworks can be combined to improve how evaluations are conducted.

This scoping review is novel in its assessment and critique of evaluation frameworks employed to evaluate work-related education programmes.

The purpose of this paper is to explore how enterprises are to arrange its learning processes in order to optimise the integration and creation of sustainable organisational learning. The paper describes a lite learning evaluation technology that makes processual real-time evaluation of implementation of new knowledge and competences in practice context.

The research is based on a case study that is designed and planned as a mixed method inquiry. The empirical case study is based on data from a large Danish enterprise from the telecommunication industry conducting a leadership and sales training programme. Case study analysis uses data drawn from the implemented pulse survey followed up with qualitative interviews with the course participants.

The authors show results on two levels. On the individual level, processual real-time lite learning evaluation tools create transparency and adaptability. On the organisational level, tool shapes the organisational capacity to improve routines and practices for how to work with organisational learning and learning data in general. Instead of treating learning and development as something that happens “automatically”, organisations now have a tool for informed decisions aimed at creating sustainable organisational learning processes and results.

The paper prompts insights that call for enterprises to enhance focus and dialogue on how to work in new and smart ways with learning at a multi-stakeholder level in organisations. The design and deployment of a real-time lite evaluation tool in organisations are key to bolster learning and competence development, so that organisations and societies can become more responsive in responding to the challenges posed by today’s knowledge economy.

Many universities are currently investing significant sums of money into refurbishing existing learning spaces and/or building further infrastructure (including Next Generation Learning Spaces (NGLS)) to support learning and teaching in the face-to-face context. While this is usually welcome by staff and students, there is often a concern that designs are not informed by input from appropriate stakeholders.

This chapter brings together information from a range of sources to provide practical ideas and advice on designing robust, whole-of-lifecycle evaluations for learning space projects. By incorporating pre- and post-occupancy stages, involving a wide array of stakeholders and looking beyond surveys and focus groups as evaluation techniques, universities can ensure that future designs take into consideration the experiences and context of staff and students at the institution as well as lessons learned from previous projects.

This research paper presents an innovative evaluation methodology which was developed as part of a doctoral research study in a voluntary sector youth organisation in England.

The transformative methodology synthesises aspects of appreciative inquiry, participatory evaluation and transformative learning and engages the whole organisation in evaluating impact. Using an interpretive paradigm, data were collected from youth workers via semi-structured interviews prior and post implementation of the transformative evaluation methodology.

Drawing on thematic analysis of the youth workers' experiences, it is argued that the illuminative and transformative nature of the methodology enabled the learning and development functions of evaluation to be realised. Further, it is argued that this form of evaluation not only supports the collection of evidence to demonstrate impact externally, but that the process itself has the potential to enhance practice, improve outcomes “in the moment” and promotes organisational learning.

The research findings are limited by the small-scale nature of the project. Further research is needed to investigate the supporting and enabling factors that underpin participatory practices in organisation evaluation; and in particular to investigate the experience of the managers and trustees as these were not the focus of this research.

This article makes a significant contribution to knowledge in regard to the design and use of participatory evaluation. It evidences the benefits in relation to generating practice improvements and for practitioners themselves in terms of countering the negatives effects of performativity. Transformative evaluation offers an innovative structure and process through which organisational learning can be realised.

The paper aims to disseminate solutions to common problems in student evaluation processes. It proposes that student evaluation can be applied to quality assurance and improving learning and teaching. The paper presents solutions in the areas of: presenting outcomes as performance indicators, constructing appropriate surveys, improving response rates, reporting student feedback to students and student engagement as a feature of university quality assurance.

The research approach of this paper is comparative case study, allowing in-depth exploration of multiple perspectives and practices at seven Australian universities. Process and outcome data were rigorously collected, analysed, compared and contrasted.

The paper provides empirical evidence for student evaluation as an instrument of learning and teaching data analysis for quality improvement. It suggests that collecting data about student engagement and the student experience will yield more useful data about student learning. Furthermore, findings indicate that students benefit from more authentic inclusion in the evaluation process and outcomes.

Because of the chosen research approach, the research results may lack generalisability. Therefore, researchers are encouraged to test the proposed propositions further and apply to their own university contexts.

The paper includes recommendations at the institution- and sector-wide levels to effectively use student evaluation as a university performance indicator and as a tool of change.

This paper fulfils an identified need to examine student evaluation processes across institutions and focuses on the role of student evaluation in quality assurance.

If initial attempts to incorporate reusable multimedia into university teaching were characterised by the efforts of enthusiasts, the growth of the “Learning Object” economy in recent years requires a more strategic and systems‐based approach. While all reusable learning materials have a value, both financial and educational, multimedia in particular is expensive to produce. Systematic production (based on ISO 14915 standards (2002a, 2002b, 2003) and evaluation of fitness for purpose, are paramount. Human‐centred design (HCD) methodologies, based on the ISO 13407 standard (1999), are now well‐established for ensuring that investments in technology result in benefits, and in particular lay heavy emphasis on the frequent use of evaluation. This paper reviews literature in educational multimedia production, interaction design, HCD, and pedagogy, and summarises ongoing attempts to standardise a definition of reusable learning objects (RLOs), with multimedia learning objects (MLOs) as a specific example. We conclude that evaluation must be central to the production and deployment of MLOs. We report on experiences applying an existing framework for effective production of multimedia learning resources, and propose revisions to this framework to add effective evaluation mechanisms.

The purpose of this study is to investigate the nature of organizational learning within Taiwanese Institute of Technology academic libraries, and in particular the nature of the relationship between the institutional evaluation of those libraries by the Ministry of Education and the responses of the libraries.

Semi‐structured interviews were conducted with 24 library personnel, including directors. These interviews explored issues arising from a survey of the related literature and resulted in the emergence of five important themes.

The results of this study show that few libraries seem to be achieving what might be considered an adequate level of organizational learning, largely it seems because of internal barriers and structural problems with how the evaluation is implemented.

The findings of this study apply only to the particular types of libraries studied and to the concept of evaluation and organizational learning as it applies to these types of libraries. The research implications are that library management needs to operate in a transformational manner; then continue with assistance and consistency to give support needed to all parts of the library organization so that comprehensive and meaningful choices can be made.

Libraries must engage more in a learning dialogue with their particular contexts, thus enhancing their capacity for true organizational learning. These implications concerning learning processes can also be applied widely to academic libraries outside Taiwan.

The contribution of the study lies in two directions: extension and development of existing conceptual frameworks in organizational learning; and examination of the use of evaluation procedures in an unusual setting.