Search results

Training evaluation is an elusive concept, especially when it comes to practice. The practice of evaluation in training has received a lot of criticism. This criticism is largely explained by the unsystematic, informal, and ad hoc evaluation that has been conducted by training institutions. In Malaysia, training activities are monitored by the government. Organisations are required to obtain training services from approved training providers registered with the government. Examines the clients’ demand toward evaluation, the commitment given by training providers, and the overall practice of evaluation by the training providers in Malaysia. Finds that the government, client and economic situations have influenced the evaluation practice in a positive direction.

This paper is based on a study which examined the current training evaluation activity and challenges that face Kuwaiti organisations. The study sample was five UK organisations (recognised as best practice organisations in their T&D activities) and 77 Kuwaiti organisations (40 government and 37 private). Interviews and questionnaires were used. The study reveals that the majority of respondents, both in government and in private sectors, only evaluate their training programme occasionally. The most popular evaluation tools and technique used by government and private sectors were questionnaires. The most common model used by Kuwaiti organisations is the Kirkpatrick model, while the most common level of evaluation for both government and private sector is reaction type.

Despite heightened interest in return on investment (ROI) and increased accountability for training professionals to prove their bottom‐line organizational value, many practitioners are deterred from comprehensive measurement and ROI evaluation due to concerns about the cost, time, and human resources necessary to fully implement the process. The purpose of this two‐part series is to present ten best practice, cost‐saving approaches for developing a credible, economical ROI strategy.

A systemic approach to measuring training's impact begins with an evaluation framework. For the purposes of this article, Jack Phillips’ five‐level framework for capturing the financial impact of training programs was referenced. Based on over 20 years of research and global applications, Phillips’ ROI model also includes techniques for isolating the impact of other variables, besides training, on performance improvement.

Many organizations around the globe are using cost‐saving approaches so they can begin conducting ROI evaluation within their current budget, while others use cost‐saving approaches in order to increase the number of ROI studies they conduct. The ten cost‐saving approaches for measuring programs at the ROI level have been proven to significantly decrease resource requirements while still providing sound, credible data. Despite these factors, establishing an evaluation culture is no easy task. In many ways, implementing a system‐wide ROI effort is similar to implementing a large‐scale change initiative.

Practical application of these cost‐saving approaches allows the resource‐constrained training function to present their work in terms of financial benefits that leaders understand and have come to expect. It is a vital step in establishing business partnerships that will enhance commitment for training programs, products, and services going forward.

By evaluating training programs with the ROI in mind, training functions can be perceived in a more credible light. Programs aligned with organization strategy are offered, while others that add little value are redesigned and sometimes eliminated. Trainers, designers and developers can use the findings of an ROI evaluation to increase training alignment with business needs and to improve the efficiency of the training design, development, and delivery life cycle.

Changes in the format of library materials, increased amounts of information, and the speed at which information is being produced have created an unrelenting need for training for library staff members. Additionally, library employees are retiring in greater numbers and their accompanying expertise is being lost. The purpose of this study was to document evaluation practices currently used in library training and continuing education programs for library employees, including metrics used in calculating return-on-investment (ROI). This research project asked 272 library training professionals to identify how they evaluate training, what kind of training evaluation practices are in place, how they select programs to evaluate for ROI, and what criteria are important in determining an effective method for calculating ROI.

This paper aims to advance the debate regarding the use of training evaluation tools, chiefly the Kirkpatrick model, in reaction to minimal use of the tools reported in the literature and the economic changes that have characterised the industrialised world in the past 20 years.

The main argument – the need to design new evaluation tools – emerges from an extensive literature review of criticism of the Kirkpatrick model. The approach is deductive; the argument emerges from extant literature.

The main findings of the literature review show that the major criticisms of the Kirkpatrick model, though rigorous, are not relevant in today's post‐industrial economy. Issues of complexity, accuracy and refinement, which are relevant in stable industrial organisations, must be revised in the new economic world.

This paper is based on a literature review and presents a call for new research. As such, it is not grounded in original empirical evidence, beyond that presented in the cited articles.

The paper calls for training evaluation tools that align better with modern organisational reality. If the research community responds to this call, the results will benefit practitioners directly. This paper also presents practical advice about the use of existing evaluation techniques.

A new angle on criticisms of existing training evaluation systems does not reiterate classic criticisms based on logic and mathematics but rather takes a pragmatic and economic approach. Thus, this paper offers evidence of theoretically grounded paradoxes of the consequences of existing criticisms of training evaluation.

This study’s purpose is to contribute to literature on training evaluation following Kirkpatrick’s four-levels model and estimating each training program’s return on investment (ROI) using evidence from a small firm.

This case study uses data collected at a logistics company based upon training output indicators like training program evaluation data; individual performance evaluation reports; information on attained objectives; service and productivity levels; quality audit reports; and accounting data.

Results show that all the training programs addressed report evaluation procedures at the four different levels (reactions, learning, behavior and results). ROI for each training program was estimated based upon costs and benefits associated to each program. The two training programs presenting above-average returns address work quality and conditions. The program addressing corporate social responsibility issues produced below-average results.

Limitations to this study may result from collecting data in a single moment in time and using data from a single organization, excluding generalization and extrapolation of results.

This case study should inspire managers in small and medium enterprises (SME) to implement training evaluation practices and ROI estimation. Having the ROI estimation available allows better management of the training budget, as ROI’s presentation is an argument to assign value and progress.

The originality of this study regards the way it reports training evaluation practices at the four levels established by Kirkpatrick’s framework (2005) and complements it with ROI estimation regarding five training courses run at a Portuguese SME logistics firm.

Management training and development is currently in vogue. There appears to be a growing belief in the benefits of investment in training and development. When a market is buoyant is the time to consider and anticipate the consequences of a future downturn in demand. Such a downturn in demand may demonstrate increasing pressure to “justify” investment in training and development. There is a long established academic body of knowledge on the subject of evaluating training and development. From research evidence and the authors′ experience, the sponsors and the providers of training and development pay scant attention to systematic evaluation of these activities and investments. It is the authors′ contention that when the market′s critical assessment of the value of training and development increases there will be an increasing interest in evaluation. An overview of the history of evaluation traditions is provided and the state of play is commented upon. It is noted that there is a shortfall between theory and practice. It is argued that evaluation is a worthwhile and important activity and ways through the evaluation literature maze and the underpinnings of the activity are demonstrated, especially to management. Similarly the literature on evaluation techniques is reviewed. Tables are provided which demonstrate areas of major activity and identify relatively uncharted waters. This monograph provides a resource whereby practitioners can choose techniques which are appropriate to the activity on which they are engaged. It highlights the process which should be undertaken to make that choice in order that needs of the major stakeholders in the exercise are fully met.

The aim of this paper is to present a study performed in conjunction with a branch of the Italian Public Italian Administration, the ISSP (Istituto Superiore di Studi Penitenziari – the Higher Institute of Penitentiary Studies). The study aimed to develop a Transfer of Training (ToT) evaluation methodology that would be both scientifically robust and practitioner-friendly, in an attempt to build a bridge between researchers and practitioners on the topic of ToT process evaluation.

The ToT evaluation system was built using an action research approach, taking into account workplace specifics and stakeholder needs. An “action research” (with the researchers involved throughout the ToT system building phases) and a “Focus Group” (to identify factors influencing ToT and to define items and grids for behavioural assessment) were used.

This study showed that the active engagement of stakeholders (trainees and their supervisors and colleagues) is useful in designing context-adapted tools for ToT evaluation and can help organizations improve the fit between their training activities and organizational goals.

The generalizability of the proposed ToT evaluation methodology is not guaranteed, and the procedure needs to be replicated and monitored in different organizational settings and cultures.

The organization decided to implement the ToT evaluation methodology developed during the course of the study for all future training programmes.

In this study, a ToT evaluation methodology was built that will be both scientifically robust and practitioner-friendly.

In recent years, the literature on program evaluation has examined multi‐stakeholder evaluation, but training evaluation models and practices have not generally taken this problem into account. The aim of this paper is to fill this gap.

This study identifies intersections between methodologies and approaches of participatory evaluation, and techniques and evaluation tools typically used for training. The study focuses on understanding the evaluation needs of the stakeholder groups typically involved in training programs. A training program financed by the European Social Fund in Italy is studied, using both qualitative and quantitative methodologies (in‐depth interviews and survey research).

The findings are as follows: first, identification of evaluation dimensions not taken into account in the return on investment training evaluation model of training evaluation, but which are important for satisfying stakeholders' evaluation needs; second, identification of convergences/divergences between stakeholder groups' evaluation needs; and third, identification of latent variables and convergences/divergences in the attribution of importance to them among stakeholders groups.

The main limitations of the research are the following: first, the analysis was based on a single training program; second, the study focused only on the pre‐conditions for designing a stakeholder‐based evaluation plan; and third, the analysis considered the attribution of importance by the stakeholders without considering the development of consistent and reliable indicators.

These results suggest that different stakeholder groups have different evaluation needs and, in operational terms are aware of the convergence and divergence between those needs.

The results of the research are useful in identifying: first, the evaluation elements that all stakeholder groups consider important; second, evaluation elements considered important by one or more stakeholder groups, but not by all of them; and third, latent variables which orient stakeholders groups in training evaluation.

The results of an evaluation provide the means for decision‐making about whether a course or program accomplished what it was designed and developed to accomplish. A three‐part evaluation schema provides a framework for describing a methodology that facilitates such determinations. Details on how to develop and use trainee reaction forms and opinionnaires are provided along with examples. The use of questionnaires and interview schedules for follow‐up evaluation is discussed and guidelines as well as examples are included. Information on evaluation strategies, including the use of pre‐ and post‐tests, and treatment and control groups, is followed by advice on the analysis and interpretation of data. Provides details on formulating recommendations, developing a plan of action to correct deficiencies, and reporting evaluation results.