Search results

Relative performance evaluation (RPE) is a widely practiced employee appraisal process in the services industry. In a global delivery model, teams are spread across different geographical locations. The team members work on various tasks under the guidance of different managers and at times under more than one manager for performing the same task. Such complexities make RPE of the team members quite challenging. The paper proposes a methodical step-by-step approach to simplify the evaluation process without compromising on the rigour.

RPE has followed three different approaches. First is the traditional way, wherein evaluators had a common meeting to discuss and arrive at relative evaluation and ranking of members of the peer group employees. In the second, the number of evaluators and employees in a peer group were split in to 2 subgroups. The evaluators provided independent ratings and rankings. Simple mathematical tool then derived the combined ranking. In the third approach, each evaluator evaluated each employee in the peer group and provided the relative ranking for each employee. Again, mathematical tools provided the final ranking considering inputs from all evaluators. All the three evaluation approaches were analysed through an inter-rater agreement method.

All the three approaches for evaluation provided similar results giving confidence that less time-consuming methods could be adopted by evaluators without compromising on the rigour of the evaluation. The outcome of the exercise proved effective as the complaints reaching the ombudsmen reduced as compared to the earlier years. Considerable evaluation time was also saved. The study described in this paper is carried out in a non-unionized, Indian private sector services firm. Its effectiveness in other set ups is yet to be tested.

The research is carried out in the Indian Engineering services firm operating in the Knowledge based sector. Though study results are encouraging, the adaptability of methodology across different sectors and geographies is yet to be tested. More broad based studies are needed to evaluate suitability across firms and regions.

Relative evaluation exercise is challenging for evaluators. Although openness in evaluation is desired, it also makes evaluators uncomfortable in appearing to be taking sides or being opposing a candidate's ranking. The proposed approach brings in anonymity to each evaluator without scarifying individual evaluation.

The proposed methodology can be deployed across different services industries as the proposed methodology is business domain agnostic. It can be easily ported and tailored to align with an individual organization's evaluation philosophy. The suitability and effectiveness of the method can be studied under various types of firms like manufacturing, private, public, NGO, labour oriented, etc. As the proposed method reduces efforts, the stake holders can focus on understanding the relation between employee performance measurement, employee engagement, and long-term outcomes related to employee performance evaluation.

The proposed employee evaluation method leverages inter-rater reliability and agreement tool as a consensus approach to the relative performance ranking exercise. Such an approach to relative performance ranking is original as no prior studies with such an approach are found in the existing Literature.

During the evaluation of European Capital of Culture (ECoC) Aarhus 2017, the evaluation organisation rethinkIMPACTS 2017 formulated a set of “dilemmas” capturing the main challenges arising during the design of the ECoC evaluation. This functioned as a framework for the evaluation process. This paper aims to present and discuss the relevance of the “Evaluation Dilemmas Model” as subsequently applied to the Galway 2020 ECoC programme evaluation.

The paper takes an empirical approach including auto-ethnography and interview data to document and map the dilemmas involved in undertaking an evaluation in two different European cities. Evolved via a process of practice-based research, the article addresses the development of and the arguments for the dilemmas model and considers its potential for wider applicability in the evaluation of large-scale cultural projects.

The authors conclude that the “Evaluation Dilemmas Model” is a valuable heuristic for considering the endogenous and exogenous issues in cultural evaluation.

The model developed is useful for a wide range of cultural evaluation processes including – but not limited to – European Capitals of Culture.

What has not been addressed in the academic literature is the process of evaluating ECoCs; especially how evaluators often take part in an overall process that is not just about the evaluation but also planning and delivering a project that includes stakeholder management and the development of evaluation criteria, design and methods.

Evaluating emergency preparedness exercises is crucial for assessing enhanced knowledge, facilitating learning and implementing knowledge in organizations. The cognitive process of motivation for action is a precursor for action, coping behavior and individual learning. This study aims to focus on how guided evaluation of emergency preparedness exercises can enhance cognitive motivation and influence the mental readiness of exercise participants.

This is a conceptual paper with a model approach design. The main conceptual contribution is suggesting a model for guided evaluation in emergency preparedness exercises. We present a theoretical background for understanding the increase in motivation based on social cognitive learning theory. In particular, this study discusses how different evaluation steps contribute to enhanced motivation and learning for exercise participants.

Increased motivation and enhanced personal performance standards could be achieved through using processes that lead to self-efficacy in guided exercise evaluation. Specifically, sources of enhanced motivation, such as repeated coping experiences, self-regulation processes, mastery motivation and performance motivation, would proliferate the readiness of individual crisis managers and teams.

This article suggests an evaluation model for use in emergency preparedness exercises. This approach combines bottom-up and top-down processes for debriefing, reflection and feedback, both individually and in teams. This approach aims to enhance exercise participants’ motivation and utilize exercise evaluation for organizational learning.

The conceptual discussion leads to developing implications for evaluation practice, suggesting how to structure evaluation and why. This study is novel for its explanation of how to use evaluation in the learning process.

The authors compare perspectives on external evaluation of health service provision between Norway and the USA. External inspection and accreditation are examples of internationally wide-spread external evaluation methods used to assess the quality of care given to patients. Different countries have different national policy strategies and arrangements set up to do these evaluations. Although there is growing attention to the impact and effects on quality and safety from external evaluation, there is still a gap in knowledge to how structures and processes influence these outcomes. Accordingly, the purpose of this article is to describe the structures and processes in external evaluation designed to promote quality improvement in Norway and the USA with attention to comparison of enablers and barriers in external evaluation systems.

Data collection consisted of documentary evidence retrieved from governmental policies, and reviews of the Joint Commission (the US), international guidelines, recommendations and reports from the International Society for Quality in Health Care, and the World Health Organization, and policies and regulations related to Norwegian governmental bodies such as the Ministry of Health and Care Services, the Norwegian Directorate of Health, and the Norwegian Board of Health Supervision . Data were analyzed inspired by a deductive, direct content analytical framework.

The authors found that both accreditation and inspection are strategies put in place to ensure that healthcare providers have adequate quality systems as well as contributing to the wider risk and safety enhancing management and implementation processes in the organizations subjected to evaluation. The US and the Norwegian external regulatory landscapes are complex and include several policymaking and governing institutions. The Norwegian regulatory framework for inspection has replaced an individual blame logic with a model which “blames” the system for inadequate quality and patient harm. This contrasts with the US accreditation system, which focuses on accreditation visits. Although findings indicate an ongoing turning point in accreditation, findings also demonstrate that involving patients and next of kin directly in adverse event inspections is a bigger part of a change in external inspection culture and methods than in processes of accreditation.

The message of this paper is important for policymakers, and bodies of inspection and accreditation because knowledge retrieved from the comparative document study may contribute to better understanding of the implications from the different system designs and in turn contribute to improving external evaluations.

Although there is a growing attention to the impact and effects on quality and safety from external evaluation, the implications of different regulatory strategies and arrangements for evaluation on quality and safety remain unclear.

This study aims to establish a robust evaluation framework for suppliers within the automotive supply chain, specifically in the stamping sector. The primary objectives are to elucidate the performance criteria of suppliers, identify indicators and scales for measuring these criteria and find the importance of the criteria.

The evaluation framework comprises a criteria hierarchy and indicators developed based on the evaluation criteria of major automotive manufacturers. Specific indicators and measurement scales are recommended for assessing suppliers. Importance weights for the criteria are assigned based on the input of nine experts using the Analytic Hierarchy Process (AHP). Finally, four sheet metal stamping tooling (SMST) suppliers are evaluated by four specialists using the proposed evaluation framework.

The study introduces a novel classification of criteria, encompassing financial and commercial perspectives, delivery capability, supplier facility and cultural approaches and business process necessities. The findings underscore the significance of financial and commercial stability in the selection of SMST suppliers, emphasizing their role in mitigating risks associated with disruptions, bankruptcies and unforeseen events. Additionally, several SMST evaluation factors identified in this study contribute to the development of resilience capabilities, highlighting the crucial importance of their inclusion and assessment in the proposed evaluation framework.

This research presents a comprehensive model for evaluating SMST suppliers, which tackles the multidisciplinary challenges within the automotive supply chain. Given the inadequacy or nonexistence of current SMTS selection models, this study bridges the gap by exploring potential and necessary criteria, alongside 116 specific indicators and measurement scales.

Course evaluations are formative and are used to evaluate learnings of the students for a course. Anomalies in the evaluation process can lead to a faulty educational outcome. Learning analytics and educational data mining provide a set of techniques that can be conveniently applied to extensive data collected as part of the evaluation process to ensure remedial actions. This study aims to conduct an experimental research to detect anomalies in the evaluation methods.

Experimental research is conducted with scientific approach and design. The researchers categorized anomaly into three categories, namely, an anomaly in criteria assessment, subject anomaly and anomaly in subject marks allocation. The different anomaly detection algorithms are used to educate data through the software R, and the results are summarized in the tables.

The data points occurring in all algorithms are finally detected as an anomaly. The anomaly identifies the data points that deviate from the data set’s normal behavior. The subject which is consistently identified as anomalous by the different techniques is marked as an anomaly in evaluation. After identification, one can drill down to more details into the title of anomalies in the evaluation criteria.

This paper proposes an analytical model for the course evaluation process and demonstrates the use of actionable analytics to detect anomalies in the evaluation process.

This paper aims to optimize the air pressure regulation scheme of the aeroengine pressure test bench.

Based on the requirements of pressure regulation process and the operating mechanism of aeroengine pressure test bench, a grey performance evaluation index system is constructed. The combination of principal component analysis and grey theory is employed to assign weights to grey indexes. The grey target evaluation model is introduced to evaluate the performance of historical regulation processes, and the evaluation results are analyzed to derive optimization mechanism for pressure regulating schemes.

A case study based on monitoring data from nearly 300 regulation processes verifies the feasibility of the proposed method. On the one hand, the improved principal component analysis method can achieve rational weighting for grey indexes. On the other hand, the method comparison intuitively shows that the proposed method performs better.

The pressure test bench is a fundamental technical equipment in the aviation industry, serving the development and testing of aircraft engines. Due to the complex system composition, the pressure and flow adjustment of the test bench heavily rely on manual experience, leading to issues such as slow adjustment speed and insufficient accuracy. This paper proposes a performance evaluation method for the regulation process of pressure test bench, which can draw knowledge from historical regulation processes, provide guidance for the pressure regulation of test benches, and ultimately achieve the goal of reducing equipment operating costs.

The study aims to introduce a cloud-based virtual reality (VR) approach and investigate its applicability and performance in aiding the remote design evaluation process by assessing the clients' convenience perception toward cloud-based VR-aided design evaluation (Cloud-based VR Approach) compared to 3D model-aided design evaluation (3D Model Approach) and rendering images-aided design evaluation (Image Approach).

A multicriteria comparative study was conducted with 26 university students using the “analytic hierarchy process” technique to compare the three approaches applied to home finishing material selection tasks based on the five “service convenience” dimensions, consisting of access convenience, decision convenience, transaction convenience, benefit convenience and post-benefit convenience.

The results indicated that the “Cloud-based VR Approach” was perceived to be more convenient than the “3D Model Approach” and the “Image Approach” based on the aspects of “decision convenience”, “transaction convenience”, “benefit convenience” and “post-benefit convenience”. The only aspect that the Cloud-based VR Approach was comparatively less convenient than the 3D Model Approach and Image Approach for the user was “access convenience”. Overall, the findings showed that the developed Cloud-based VR Approach had more potential than the conventional approaches in aiding the design evaluation process under ongoing social distancing measures requiring designers and clients to work remotely.

The disastrous impacts of the COVID-19 pandemic on logistical systems have resulted in massive disruptions to the construction industry worldwide. Various construction activities have been halted and most meetings moved online. Design evaluation conducted between clients and designers is one of the important activities affected by such an impact. Thus, this study presents the Cloud-based VR Approach as an innovative means to maintain essential ongoing activities and meeting of the current design evaluation approach with respect to the social distancing measures.

Scholars have pointed out the need for improvement and refinement in public management research, also depending on the role of public administration for sustainable development. Because government organizations employ a substantial portion of the workforce, management practices in the public sector are critical areas for designing, implementing and delivering policies that can achieve the goals set forth in the 2030 Agenda for Sustainable Development. For these purposes, and in implementation of the Next Generation EU (NGEU) programme, Italy recently launched an ambitious National Recovery and Resilience Plan (NRRP) which includes, among other things, upskilling goals for staff employed, following the modernization process that has involved the Italian public sector in the last decades, with the Decree n. 150/2009. Aiming both to understand the extent of the application of the reform and to answer some basic questions (why, what and how) concerning Individual Performance Appraisal Systems (IPAS), we conducted a content analysis on the 220 documents already produced by the Italian Ministries. The study has been conducted in two different steps of the reform process and provides solid evidence of the reforms' effects on designing and implementing individual performance systems. The analyzed documents reveal no longer-term vision in implementing the IPAS that involves some critical performance management utilities such as training, development, fair pay and deployment of employees, raising new questions about a sustainable approach to the individual performance management process even in public organizations.

While providing users with important innovation resources, an enterprise's open innovation community also encounters numerous challenges. The purpose of this study is to develop and implement a set of evaluation systems that can identify the ecological characteristics of open innovation communities. The ultimate goal is to provide guidance for the construction and establishment of these communities.

Firstly, from the perspective of information ecology, according to the connotation requirements of the four information elements of “people, information, technology and environment” in the information ecosystem, this paper determines the evaluation indicators of enterprise's open innovation community from the aspects of integrity and dynamics. Then, based on the improved analytic hierarchy process, the weight of each index is calculated, and a complete evaluation system of open innovation community is established. Finally, the fuzzy comprehensive evaluation method is used to conduct empirical research on Haier Open Partnership (HOPE) platform, analyze its ecological situation and existing problems and put forward the corresponding construction recommendations.

The application of HOPE platform shows that the evaluation index system built in this study has good applicability, and the whole research process and research method have certain reference value for future studies. At the same time, the study found that the HOPE platform has good ecological characteristics, but there are also some problems. This study can directly promote the construction of the platform.

This paper, for the first time, combines the information ecology theory with the open innovation community. The results demonstrate that the open innovation community is an information ecosystem, which can be evaluated ecologically. Furthermore, to ensure the calculation's scientific accuracy, the index weight is determined by constructing a judgment matrix based on expert questionnaires.