Search results

Training evaluation is an elusive concept, especially when it comes to practice. The practice of evaluation in training has received a lot of criticism. This criticism is largely explained by the unsystematic, informal, and ad hoc evaluation that has been conducted by training institutions. In Malaysia, training activities are monitored by the government. Organisations are required to obtain training services from approved training providers registered with the government. Examines the clients’ demand toward evaluation, the commitment given by training providers, and the overall practice of evaluation by the training providers in Malaysia. Finds that the government, client and economic situations have influenced the evaluation practice in a positive direction.

This paper is based on a study which examined the current training evaluation activity and challenges that face Kuwaiti organisations. The study sample was five UK organisations (recognised as best practice organisations in their T&D activities) and 77 Kuwaiti organisations (40 government and 37 private). Interviews and questionnaires were used. The study reveals that the majority of respondents, both in government and in private sectors, only evaluate their training programme occasionally. The most popular evaluation tools and technique used by government and private sectors were questionnaires. The most common model used by Kuwaiti organisations is the Kirkpatrick model, while the most common level of evaluation for both government and private sector is reaction type.

This paper presents research‐based insight on the challenges of evaluating training activities in today's complex organizational settings.

The research is taken from three case studies conducted in the New Zealand manufacturing sector, as well as sources of relevant literature. The commentary takes a critical‐realist perspective and challenges learning and development professionals to address the poor reputation of training evaluation.

Human resource practitioners recognise the importance of gaining feedback from learning events, but research reports question the thoroughness of evaluation processes, claiming they rarely happen to the satisfaction of management. Consequently, training budgets become an easy target during periods of rationalization. The problem centres on overcoming the complexity of defining a meaningful cause/effect relationship between the training and resultant benefit. This research discovered the presence of an “evaluation vacuum” and nine thematic areas requiring close attention. The paper offers reasons why the evaluation of training is becoming increasingly difficult.

The findings are contextual and may not fit all settings, but they offer a comparative account of training evaluation in both straightforward and complex learning environments.

The paper has real and practical implications for human resource professionals. Evaluation of training is not a trivial issue and organizations need to get much better at explaining the beneficial outcomes derived from investments in training.

This paper will be of value to human resource professionals and managers, assisting them to think differently about evaluating training. Innovative concepts such as the “evaluation vacuum” and the term “learning bleed” clarify priorities and contribute to a new perspective on evaluation.

This paper aims to advance the debate regarding the use of training evaluation tools, chiefly the Kirkpatrick model, in reaction to minimal use of the tools reported in the literature and the economic changes that have characterised the industrialised world in the past 20 years.

The main argument – the need to design new evaluation tools – emerges from an extensive literature review of criticism of the Kirkpatrick model. The approach is deductive; the argument emerges from extant literature.

The main findings of the literature review show that the major criticisms of the Kirkpatrick model, though rigorous, are not relevant in today's post‐industrial economy. Issues of complexity, accuracy and refinement, which are relevant in stable industrial organisations, must be revised in the new economic world.

This paper is based on a literature review and presents a call for new research. As such, it is not grounded in original empirical evidence, beyond that presented in the cited articles.

The paper calls for training evaluation tools that align better with modern organisational reality. If the research community responds to this call, the results will benefit practitioners directly. This paper also presents practical advice about the use of existing evaluation techniques.

A new angle on criticisms of existing training evaluation systems does not reiterate classic criticisms based on logic and mathematics but rather takes a pragmatic and economic approach. Thus, this paper offers evidence of theoretically grounded paradoxes of the consequences of existing criticisms of training evaluation.

This study’s purpose is to contribute to literature on training evaluation following Kirkpatrick’s four-levels model and estimating each training program’s return on investment (ROI) using evidence from a small firm.

This case study uses data collected at a logistics company based upon training output indicators like training program evaluation data; individual performance evaluation reports; information on attained objectives; service and productivity levels; quality audit reports; and accounting data.

Results show that all the training programs addressed report evaluation procedures at the four different levels (reactions, learning, behavior and results). ROI for each training program was estimated based upon costs and benefits associated to each program. The two training programs presenting above-average returns address work quality and conditions. The program addressing corporate social responsibility issues produced below-average results.

Limitations to this study may result from collecting data in a single moment in time and using data from a single organization, excluding generalization and extrapolation of results.

This case study should inspire managers in small and medium enterprises (SME) to implement training evaluation practices and ROI estimation. Having the ROI estimation available allows better management of the training budget, as ROI’s presentation is an argument to assign value and progress.

The originality of this study regards the way it reports training evaluation practices at the four levels established by Kirkpatrick’s framework (2005) and complements it with ROI estimation regarding five training courses run at a Portuguese SME logistics firm.

Management training and development is currently in vogue. There appears to be a growing belief in the benefits of investment in training and development. When a market is buoyant is the time to consider and anticipate the consequences of a future downturn in demand. Such a downturn in demand may demonstrate increasing pressure to “justify” investment in training and development. There is a long established academic body of knowledge on the subject of evaluating training and development. From research evidence and the authors′ experience, the sponsors and the providers of training and development pay scant attention to systematic evaluation of these activities and investments. It is the authors′ contention that when the market′s critical assessment of the value of training and development increases there will be an increasing interest in evaluation. An overview of the history of evaluation traditions is provided and the state of play is commented upon. It is noted that there is a shortfall between theory and practice. It is argued that evaluation is a worthwhile and important activity and ways through the evaluation literature maze and the underpinnings of the activity are demonstrated, especially to management. Similarly the literature on evaluation techniques is reviewed. Tables are provided which demonstrate areas of major activity and identify relatively uncharted waters. This monograph provides a resource whereby practitioners can choose techniques which are appropriate to the activity on which they are engaged. It highlights the process which should be undertaken to make that choice in order that needs of the major stakeholders in the exercise are fully met.

The purpose of this paper was to test the mediating role of training duration in relationship between firm characteristics and training evaluation practices. In this paper, the authors also investigated if this mediating effect differs with respect to the size of the firm.

The authors collected data from 260 professionals of 90 call centers.

The authors found that training duration mediates the relationship between firm size and training evaluation. The authors also found that indirect effect of firm size on training evaluation through training duration differs across different levels of firm size but not across different levels of ownership.

This is a cross-sectional study that emphasized on training evaluation practices only.

The study has implication for both evaluation researchers and practitioners in terms of designing training evaluation policies and practices.

This is the first study in its nature that explains the intervening role of training duration in relationship of firm characteristics and training evaluation practices.

The aim of this paper is to present a study performed in conjunction with a branch of the Italian Public Italian Administration, the ISSP (Istituto Superiore di Studi Penitenziari – the Higher Institute of Penitentiary Studies). The study aimed to develop a Transfer of Training (ToT) evaluation methodology that would be both scientifically robust and practitioner-friendly, in an attempt to build a bridge between researchers and practitioners on the topic of ToT process evaluation.

The ToT evaluation system was built using an action research approach, taking into account workplace specifics and stakeholder needs. An “action research” (with the researchers involved throughout the ToT system building phases) and a “Focus Group” (to identify factors influencing ToT and to define items and grids for behavioural assessment) were used.

This study showed that the active engagement of stakeholders (trainees and their supervisors and colleagues) is useful in designing context-adapted tools for ToT evaluation and can help organizations improve the fit between their training activities and organizational goals.

The generalizability of the proposed ToT evaluation methodology is not guaranteed, and the procedure needs to be replicated and monitored in different organizational settings and cultures.

The organization decided to implement the ToT evaluation methodology developed during the course of the study for all future training programmes.

In this study, a ToT evaluation methodology was built that will be both scientifically robust and practitioner-friendly.

In recent years, the literature on program evaluation has examined multi‐stakeholder evaluation, but training evaluation models and practices have not generally taken this problem into account. The aim of this paper is to fill this gap.

This study identifies intersections between methodologies and approaches of participatory evaluation, and techniques and evaluation tools typically used for training. The study focuses on understanding the evaluation needs of the stakeholder groups typically involved in training programs. A training program financed by the European Social Fund in Italy is studied, using both qualitative and quantitative methodologies (in‐depth interviews and survey research).

The findings are as follows: first, identification of evaluation dimensions not taken into account in the return on investment training evaluation model of training evaluation, but which are important for satisfying stakeholders' evaluation needs; second, identification of convergences/divergences between stakeholder groups' evaluation needs; and third, identification of latent variables and convergences/divergences in the attribution of importance to them among stakeholders groups.

The main limitations of the research are the following: first, the analysis was based on a single training program; second, the study focused only on the pre‐conditions for designing a stakeholder‐based evaluation plan; and third, the analysis considered the attribution of importance by the stakeholders without considering the development of consistent and reliable indicators.

These results suggest that different stakeholder groups have different evaluation needs and, in operational terms are aware of the convergence and divergence between those needs.

The results of the research are useful in identifying: first, the evaluation elements that all stakeholder groups consider important; second, evaluation elements considered important by one or more stakeholder groups, but not by all of them; and third, latent variables which orient stakeholders groups in training evaluation.

The results of an evaluation provide the means for decision‐making about whether a course or program accomplished what it was designed and developed to accomplish. A three‐part evaluation schema provides a framework for describing a methodology that facilitates such determinations. Details on how to develop and use trainee reaction forms and opinionnaires are provided along with examples. The use of questionnaires and interview schedules for follow‐up evaluation is discussed and guidelines as well as examples are included. Information on evaluation strategies, including the use of pre‐ and post‐tests, and treatment and control groups, is followed by advice on the analysis and interpretation of data. Provides details on formulating recommendations, developing a plan of action to correct deficiencies, and reporting evaluation results.