Search results

It has become accepted practice to include an evaluation alongside learning programmes that take place at work, as a means of judging their effectiveness. There is a tendency to focus such evaluations on the relevance of the intervention and the amount of learning achieved by the individual. The aim of this review is to examine existing evaluation frameworks that have been used to evaluate education interventions and, in particular, assess how these have been used and the outcomes of such activity.

A scoping review using Arskey and O’Malley’s five stage framework was undertaken to examine existing evaluation frameworks claiming to evaluate education interventions.

Forty five articles were included in the review. A majority of papers concentrate on learner satisfaction and/or learning achieved. Rarely is a structured framework mentioned, or detail of the approach to analysis cited. Typically, evaluations lacked baseline data, control groups, longitudinal observations and contextual awareness.

This review has implications for those involved in designing and evaluating work-related education programmes, as it identifies areas where evaluations need to be strengthened and recommends how existing frameworks can be combined to improve how evaluations are conducted.

This scoping review is novel in its assessment and critique of evaluation frameworks employed to evaluate work-related education programmes.

A number of studies on Kirkpatrick’s four-level training evaluation model have been published, since its inception in 1959, either investigating it or applying it to evaluate the training process. The purpose of this bibliometric analysis is to reconsider the model, its utility and its effectiveness in meeting the need to evaluate training activities and to explain why the model is still worth using even though other later models are available.

This study adopts a “5Ws+1H” model (why, when, who, where, what and how); however, “when” and “how” are merged in the methodology. A total of 416 articles related to Kirkpatrick’s model published between 1959 and July 2020 were retrieved using Scopus.

The Kirkpatrick model continues to be useful, appropriate and applicable in a variety of contexts. It is adaptable to many training environments and achieves high performance in evaluating training. The overview of publications on the Kirkpatrick model shows that research using the model is an active and growing area. The model is used primarily in the evaluation of medical training, followed by computer science, business and social sciences.

This paper presents a comprehensive bibliometric analysis to reconsider the model, its utility, its effectiveness in meeting the need to evaluate training activities, its importance in the field measured by the growth in studies on the model and its applications in various settings and contexts.

This study aims to explore the benefits of participation in a topic-relevant business conference as a learning and professional development apparatus for senior managers of small- and medium-sized enterprises (SMEs). It analyzes the experiences of 12 Australian SME senior managers who participated in a Belt and Road Initiative (BRI) business conference in Hong Kong.

Data were collected through pre- and post-conference attendance interviews. Kirkpatrick and Kirkpatrick’s (2005) four levels framework was used to analyze the effectiveness of business conferences on the participants at four different levels of the framework: reaction, learning, behavior and results.

This study finds that the business conference has shown effectiveness for the participants at the reaction (Level 1), learning (Level 2) and behavior (Level 3), indicating that participation in a topic-relevant business conference is useful for the learning and professional development of SME managers. However, only a moderate level of results (Level 4) was identified from attending the BRI conference, which implies that the content and quality of the business conference may influence the achievement of expected results.

This paper contributes a new understanding of the benefits of topic-relevant business conference participation as a learning and professional development apparatus for SME senior managers.

The purpose of this paper is to explore the underlying asymmetries in training evaluation practices of call centre (CC) firms based on their “in-house” and “subcontractor” ownership heterogeneity.

The data for this qualitative inquiry were collected from key informants of 13 different CCs in Pakistan through semi-structured interviews.

The findings revealed various asymmetries in training evaluation practices among in-house and subcontractor CCs based on five different dimensions of two renowned training evaluation frameworks.

Training evaluation professionals can benefit from training evaluation methods identified in this study for measuring training evaluation practice and advancing future research.

This study has theoretically contributed to the existing research on firm heterogeneity and human resource management by focussing on training evaluation practices in CCs.

The purpose of this paper is to evaluate behavioural changes and operational outcomes resulting from a Master’s programme on improvement knowledge and leadership in the Swedish welfare sector. The welfare sector is the collective term for tax-funded services the state, county councils and municipalities are responsible to provide.

A survey combined open-ended and closed questions using a five-point Likert scale. The questions were based on the learning objectives of the Master’s programme. The survey was sent to 139 graduates and achieved a response rate of 41 per cent (57 respondents). Responses were entered into a survey programme to enable the descriptive presentation of data; open-ended responses were analysed using conventional content analysis.

Respondents reported their increased knowledge and changed behaviours had impacted operational outcomes, e.g. processes efficiency, compliance with guidelines and quality. They said the programme was of value to themselves and society but requested more leadership knowledge. All respondents recommended the programme to others.

By operationalizing the Kirkpatrick framework, the paper describes outcomes on levels three and four, and the use of numerous best practice techniques for adult learning. This is valuable knowledge for organisers of improvement knowledge educations.

The purpose of this paper is to identify and apply best practices in university entrepreneurship education to the creation of a new MBA in entrepreneurship and innovation management. It is a direct response to calls for a total re-envisioning of entrepreneurship education and criticism that existing programs lack rigour, content, pedagogy, measurement and an established definition.

This article uses reviews of the literature to identify normative best practices and how to apply them to the new program. An entrepreneurship program design framework (EPDF) was created and applied to a new MBA program being developed in central Germany.

Most studies describe aspects of current programs (e.g. lists of courses) but almost none say what should be in a program. Others provide abstract guidance (e.g. programs should define entrepreneurship) but do not give specific recommendations (e.g. what the definition should be). The proposed EPDF provided a rigorous structure for reviewing the literature, designing the new program and establishing specific best practice recommendations for defining program goals, content, pedagogy and measurement of student transformation.

The entrepreneurship literature is largely silent on normative best practice guidance, so the proposed application of best practices should be evaluated in that context.

Previous articles present relatively abstract frameworks and concepts, whereas this article is a direct application of the practical implications of these concepts. The proposed normative best practice guidelines may be somewhat controversial, but should stimulate useful discussion.

The purpose of this paper is to critically review existing models for evaluating organizational human resource (HR) interventions and to develop a practical but robust model for use by practitioners and researchers.

A literature review was undertaken of existing models developed to evaluate organizational HR interventions. Based on these a new model was developed.

The paper suggests that many of the existing models are either outdated or lack practical focus. The SOAP‐M model offers five levels for evaluation, four suitable for HR professionals and a fifth level for researchers.

The paper is conceptual and the model needs to be tested by organizations to assess whether this has utility for HR practitioners.

The model offers a framework for conducting evaluations of organizational interventions and encourages HR practitioners to improve their practice by conducting robust evaluations of the interventions they use.

The paper offers a new and previously unpublished model, which could help HR practitioners improve their use of evaluation.

This study conducts a systematic literature review (SLR) on serious games in the BPM domain, in order to answer: RQ1 – what are BPM games' learning objectives and design characteristics? RQ2 – How rigorously each BPM game was evaluated in terms of their effectiveness and entertainment?.

We followed the PRISMA statement method and systematic literature review standards. Selected studies were analyzed using qualitative evidence synthesis.

We found 37 references that describe 15 serious games within the BPM domain. Games are diverse, ranging from analog games that need a few hours to complete to complex digital games that require many sessions. Game evaluation should be generally improved as 27 publications can be considered as quasi-experimental or nonexperimental and potentially biased.

Future research should focus on developing new games about process implementation and using other notations besides BPMN. More studies testing the reliability and validity of existing games in different settings and using more rigorous experimental designs are also necessary. Practitioners can identify thoroughly evaluated games relevant to their learning objectives.

This is the first SLR about BPM games. It consolidates the literature, analyzes, describes and categorizes 15 games with respect to their internal characteristics and evaluation reliability.

Interprofessional education (IPE) has been highly promoted as a means of enhancing interprofessional practice and thereby having a positive impact on healthcare systems and patient outcomes. Various documents mention that sufficient evidence has been accumulated to demonstrate the effectiveness of IPE, yet it is not completely clear what type of evidence is being alluded to. The objective of this review was to gather evidence about IPE programs that resulted in effective long-term outcomes in healthcare. Secondary outcomes included identification of the types of models that met the success criteria, barriers and facilitators of such successful programs if any.

A systematic search was conducted in PubMed, Web of Science, CINAHL and Scopus. The review considered studies that targeted undergraduate and postgraduate students among more than one health profession and included those in the English language published between 2010 and end of 2020.

Five studies have been identified and described in this review. These papers evaluated different IPE programs and models.

1. This systematic review investigated the evidence of the existence of IPE programs and the findings show there is no robust specific evidence of long-term impact on healthcare and on patients' outcomes. 2. The conclusion from this review is that it is still unclear what format constitutes a successful and efficient program. 3. Appropriate longitudinal studies need to be designed to identify the impact of IPE on long-term health outcomes.

Overall, the studies show that although there is an emphasis on practice-based learning, there is no robust specific evidence of long-term impact on healthcare and on patients' outcomes. Appropriate longitudinal studies need to be designed to identify the impact of IPE on long-term health outcomes.

Examines the costs and benefits of human resources development (HRD) and arguments for measuring the return on investment in HRD. Outlines a framework for evaluation of HRD effectiveness.