Search results

Relative performance evaluation (RPE) is a widely practiced employee appraisal process in the services industry. In a global delivery model, teams are spread across different geographical locations. The team members work on various tasks under the guidance of different managers and at times under more than one manager for performing the same task. Such complexities make RPE of the team members quite challenging. The paper proposes a methodical step-by-step approach to simplify the evaluation process without compromising on the rigour.

RPE has followed three different approaches. First is the traditional way, wherein evaluators had a common meeting to discuss and arrive at relative evaluation and ranking of members of the peer group employees. In the second, the number of evaluators and employees in a peer group were split in to 2 subgroups. The evaluators provided independent ratings and rankings. Simple mathematical tool then derived the combined ranking. In the third approach, each evaluator evaluated each employee in the peer group and provided the relative ranking for each employee. Again, mathematical tools provided the final ranking considering inputs from all evaluators. All the three evaluation approaches were analysed through an inter-rater agreement method.

All the three approaches for evaluation provided similar results giving confidence that less time-consuming methods could be adopted by evaluators without compromising on the rigour of the evaluation. The outcome of the exercise proved effective as the complaints reaching the ombudsmen reduced as compared to the earlier years. Considerable evaluation time was also saved. The study described in this paper is carried out in a non-unionized, Indian private sector services firm. Its effectiveness in other set ups is yet to be tested.

The research is carried out in the Indian Engineering services firm operating in the Knowledge based sector. Though study results are encouraging, the adaptability of methodology across different sectors and geographies is yet to be tested. More broad based studies are needed to evaluate suitability across firms and regions.

Relative evaluation exercise is challenging for evaluators. Although openness in evaluation is desired, it also makes evaluators uncomfortable in appearing to be taking sides or being opposing a candidate's ranking. The proposed approach brings in anonymity to each evaluator without scarifying individual evaluation.

The proposed methodology can be deployed across different services industries as the proposed methodology is business domain agnostic. It can be easily ported and tailored to align with an individual organization's evaluation philosophy. The suitability and effectiveness of the method can be studied under various types of firms like manufacturing, private, public, NGO, labour oriented, etc. As the proposed method reduces efforts, the stake holders can focus on understanding the relation between employee performance measurement, employee engagement, and long-term outcomes related to employee performance evaluation.

The proposed employee evaluation method leverages inter-rater reliability and agreement tool as a consensus approach to the relative performance ranking exercise. Such an approach to relative performance ranking is original as no prior studies with such an approach are found in the existing Literature.

During the evaluation of European Capital of Culture (ECoC) Aarhus 2017, the evaluation organisation rethinkIMPACTS 2017 formulated a set of “dilemmas” capturing the main challenges arising during the design of the ECoC evaluation. This functioned as a framework for the evaluation process. This paper aims to present and discuss the relevance of the “Evaluation Dilemmas Model” as subsequently applied to the Galway 2020 ECoC programme evaluation.

The paper takes an empirical approach including auto-ethnography and interview data to document and map the dilemmas involved in undertaking an evaluation in two different European cities. Evolved via a process of practice-based research, the article addresses the development of and the arguments for the dilemmas model and considers its potential for wider applicability in the evaluation of large-scale cultural projects.

The authors conclude that the “Evaluation Dilemmas Model” is a valuable heuristic for considering the endogenous and exogenous issues in cultural evaluation.

The model developed is useful for a wide range of cultural evaluation processes including – but not limited to – European Capitals of Culture.

What has not been addressed in the academic literature is the process of evaluating ECoCs; especially how evaluators often take part in an overall process that is not just about the evaluation but also planning and delivering a project that includes stakeholder management and the development of evaluation criteria, design and methods.

A successful process of design concept evaluation has positive influence on subsequent processes. This study aims to consider the evaluation information at multiple stages and the interaction among evaluators and improve the credibility of evaluation results.

This paper proposes a multi-stage approach for design concept evaluation based on complex network and bounded confidence. First, a network is constructed according to the evaluation data. Depending on the consensus degree of evaluation opinions, the number of evaluation rounds is determined. Then, bounded confidence rules are applied for the modification of preference information. Last, a planning function is constructed to calculate the weight of each stage and aggregate information at multiple evaluation stages.

The results indicate that the opinions of the evaluators tend to be consistent after multiple stages of interactive adjustment, and the ordering of design concept alternatives tends to be stable with the progress of the evaluation.

Updating preferences according to the bounded confidence rules, only the opinions within the trust threshold are considered. The attribute information of the node itself is inadequately considered.

This method addresses the need for considering the evaluation information at each stage and minimizes the impact of disagreements within the evaluation group on the evaluation results.

The authors compare perspectives on external evaluation of health service provision between Norway and the USA. External inspection and accreditation are examples of internationally wide-spread external evaluation methods used to assess the quality of care given to patients. Different countries have different national policy strategies and arrangements set up to do these evaluations. Although there is growing attention to the impact and effects on quality and safety from external evaluation, there is still a gap in knowledge to how structures and processes influence these outcomes. Accordingly, the purpose of this article is to describe the structures and processes in external evaluation designed to promote quality improvement in Norway and the USA with attention to comparison of enablers and barriers in external evaluation systems.

Data collection consisted of documentary evidence retrieved from governmental policies, and reviews of the Joint Commission (the US), international guidelines, recommendations and reports from the International Society for Quality in Health Care, and the World Health Organization, and policies and regulations related to Norwegian governmental bodies such as the Ministry of Health and Care Services, the Norwegian Directorate of Health, and the Norwegian Board of Health Supervision . Data were analyzed inspired by a deductive, direct content analytical framework.

The authors found that both accreditation and inspection are strategies put in place to ensure that healthcare providers have adequate quality systems as well as contributing to the wider risk and safety enhancing management and implementation processes in the organizations subjected to evaluation. The US and the Norwegian external regulatory landscapes are complex and include several policymaking and governing institutions. The Norwegian regulatory framework for inspection has replaced an individual blame logic with a model which “blames” the system for inadequate quality and patient harm. This contrasts with the US accreditation system, which focuses on accreditation visits. Although findings indicate an ongoing turning point in accreditation, findings also demonstrate that involving patients and next of kin directly in adverse event inspections is a bigger part of a change in external inspection culture and methods than in processes of accreditation.

The message of this paper is important for policymakers, and bodies of inspection and accreditation because knowledge retrieved from the comparative document study may contribute to better understanding of the implications from the different system designs and in turn contribute to improving external evaluations.

Although there is a growing attention to the impact and effects on quality and safety from external evaluation, the implications of different regulatory strategies and arrangements for evaluation on quality and safety remain unclear.

The purpose of this paper is to establish a maturity evaluation model for the application of construction steel structure welding robotics suitable for the actual situation and specific characteristics of engineering projects in China and then to assess the maturity level of the technology in the application of domestic engineering projects more scientifically.

The research follows a qualitative and quantitative analysis method. In the first stage, the structure of the maturity model is constructed and the evaluation index system is designed by using the ideas of the capability maturity model and WSR methodology for reference. In the second stage, the design of the evaluation process and the selection of evaluation methods (analytic hierarchy process method, multi-level gray comprehensive evaluation method). In the third stage, the data are collected and organized (preparation of questionnaires, distribution of questionnaires, questionnaire collection). In the fourth stage, the established maturity evaluation model is used to analyze the data.

The evaluation model established by using multi-level gray theory can effectively transform various complex indicators into an intuitive maturity level or score status. The conclusion shows that the application maturity of building steel structure welding robot technology in this project is at the development level as a whole. The maturity levels of “WuLi – ShiLi – RenLi” are respectively: development level, development level, between starting level and development level. Comparison of maturity evaluation values of five important factors (from high to low): environmental factors, technical factors, management factors, benefit factors, personnel and group factors.

In this paper, based on the existing research related to construction steel structure welding robot technology, a quantitative and holistic evaluation of the application of construction steel structure welding robot technology in domestic engineering projects is conducted for the first time from a project perspective by designing a maturity evaluation index system and establishing a maturity evaluation model. This research will help the project team to evaluate the application level (maturity) of the welding robot in the actual project, identify the shortcomings and defects of the application of this technology, then improve the weak links pertinently, and finally realize the gradual improvement of the overall application level of welding robot technology for building steel structure.

Course evaluations are formative and are used to evaluate learnings of the students for a course. Anomalies in the evaluation process can lead to a faulty educational outcome. Learning analytics and educational data mining provide a set of techniques that can be conveniently applied to extensive data collected as part of the evaluation process to ensure remedial actions. This study aims to conduct an experimental research to detect anomalies in the evaluation methods.

Experimental research is conducted with scientific approach and design. The researchers categorized anomaly into three categories, namely, an anomaly in criteria assessment, subject anomaly and anomaly in subject marks allocation. The different anomaly detection algorithms are used to educate data through the software R, and the results are summarized in the tables.

The data points occurring in all algorithms are finally detected as an anomaly. The anomaly identifies the data points that deviate from the data set’s normal behavior. The subject which is consistently identified as anomalous by the different techniques is marked as an anomaly in evaluation. After identification, one can drill down to more details into the title of anomalies in the evaluation criteria.

This paper proposes an analytical model for the course evaluation process and demonstrates the use of actionable analytics to detect anomalies in the evaluation process.

The study aims to introduce a cloud-based virtual reality (VR) approach and investigate its applicability and performance in aiding the remote design evaluation process by assessing the clients' convenience perception toward cloud-based VR-aided design evaluation (Cloud-based VR Approach) compared to 3D model-aided design evaluation (3D Model Approach) and rendering images-aided design evaluation (Image Approach).

A multicriteria comparative study was conducted with 26 university students using the “analytic hierarchy process” technique to compare the three approaches applied to home finishing material selection tasks based on the five “service convenience” dimensions, consisting of access convenience, decision convenience, transaction convenience, benefit convenience and post-benefit convenience.

The results indicated that the “Cloud-based VR Approach” was perceived to be more convenient than the “3D Model Approach” and the “Image Approach” based on the aspects of “decision convenience”, “transaction convenience”, “benefit convenience” and “post-benefit convenience”. The only aspect that the Cloud-based VR Approach was comparatively less convenient than the 3D Model Approach and Image Approach for the user was “access convenience”. Overall, the findings showed that the developed Cloud-based VR Approach had more potential than the conventional approaches in aiding the design evaluation process under ongoing social distancing measures requiring designers and clients to work remotely.

The disastrous impacts of the COVID-19 pandemic on logistical systems have resulted in massive disruptions to the construction industry worldwide. Various construction activities have been halted and most meetings moved online. Design evaluation conducted between clients and designers is one of the important activities affected by such an impact. Thus, this study presents the Cloud-based VR Approach as an innovative means to maintain essential ongoing activities and meeting of the current design evaluation approach with respect to the social distancing measures.

Advances in information technology now permit the recording of massive and diverse process data, thereby making data-driven evaluations possible. This study discusses whether teachers’ information literacy can be evaluated based on their online information behaviors on online learning and teaching platforms (OLTPs).

First, to evaluate teachers’ information literacy, the process data were combined from teachers on OLTP to describe nine third-level indicators from the richness, diversity, usefulness and timeliness analysis dimensions. Second, propensity score matching (PSM) and difference tests were used to analyze the differences between the performance groups with reduced selection bias. Third, to effectively predict the information literacy score of each teacher, four sets of input variables were used for prediction using supervised learning models.

The results show that the high-performance group performs better than the low-performance group in 6 indicators. In addition, information-based teaching and behavioral research data can best reflect the level of information literacy. In the future, greater in-depth explorations are needed with richer online information behavioral data and a more effective evaluation model to increase evaluation accuracy.

The evaluation based on online information behaviors has concrete application scenarios, positively correlated results and prediction interpretability. Therefore, information literacy evaluations based on behaviors have great potential and favorable prospects.

The purpose of this study is to propose a framework for the convergence of maturity model and education and evaluation in accounting.

The present research was conducted in two phases. In the first phase, to determine the indicators of convergence of the maturity model and education and evaluation in accounting, a Meta-Synthesis method was used. The conceptual model includes two dimensions of “Teaching and learning processes” and “Evaluation methods"; five levels of initial, repeatable, defined, managed and optimized; and a total number of 35 indicators. In the second phase, a questionnaire was developed, and academics as accounting faculty members in Iranian public universities were employed to fill out the questionnaire electronically and present a final framework. Having received the questionnaires, 66 questionnaires were analyzed statistically.

The results showed that the two dimensions of “Teaching and learning processes” and “Evaluation methods” considering initial, repeatable, defined, managed and optimized levels include 35 indicators, which form a framework for the convergence of maturity model and education and evaluation in accounting. The results show that both dimensions have positive and significant regression path coefficients in the convergence model. Moreover, the dimension of teaching and learning processes has the highest regression path coefficient indicating a greater impact on the convergence model. Besides, all five levels have positive and significant regression path coefficients with dimensions. Finally, in this study, all indicators were prioritized according to five levels.

Due to the success of maturity models and the urgent developments that require transformative improvements in accounting education, maturity models can respond to the challenges associated with education and learning in accounting. Thus, conceiving an image of the convergence of maturity model, education and evaluation in accounting seems imperative which has been scarcely investigated previously.

While providing users with important innovation resources, an enterprise's open innovation community also encounters numerous challenges. The purpose of this study is to develop and implement a set of evaluation systems that can identify the ecological characteristics of open innovation communities. The ultimate goal is to provide guidance for the construction and establishment of these communities.

Firstly, from the perspective of information ecology, according to the connotation requirements of the four information elements of “people, information, technology and environment” in the information ecosystem, this paper determines the evaluation indicators of enterprise's open innovation community from the aspects of integrity and dynamics. Then, based on the improved analytic hierarchy process, the weight of each index is calculated, and a complete evaluation system of open innovation community is established. Finally, the fuzzy comprehensive evaluation method is used to conduct empirical research on Haier Open Partnership (HOPE) platform, analyze its ecological situation and existing problems and put forward the corresponding construction recommendations.

The application of HOPE platform shows that the evaluation index system built in this study has good applicability, and the whole research process and research method have certain reference value for future studies. At the same time, the study found that the HOPE platform has good ecological characteristics, but there are also some problems. This study can directly promote the construction of the platform.

This paper, for the first time, combines the information ecology theory with the open innovation community. The results demonstrate that the open innovation community is an information ecosystem, which can be evaluated ecologically. Furthermore, to ensure the calculation's scientific accuracy, the index weight is determined by constructing a judgment matrix based on expert questionnaires.