Search results

The purpose of this study is to compare the evaluation of search result lists and documents, in particular evaluation criteria, elements, association between criteria and elements, pre/post and evaluation activities, and the time spent on evaluation.

The study analyzed the data collected from 31 general users through prequestionnaires, think aloud protocols and logs, and post questionnaires. Types of evaluation criteria, elements, associations between criteria and elements, evaluation activities and their associated pre/post activities, and time were analyzed based on open coding.

The study identifies the similarities and differences of list and document evaluation by analyzing 21 evaluation criteria applied, 13 evaluation elements examined, pre/post and evaluation activities performed and time spent. In addition, the authors also explored the time spent in evaluating lists and documents for different types of tasks.

This study helps researchers understand the nature of list and document evaluation. Additionally, this study connects elements that participants examined to criteria they applied, and further reveals problems associated with the lack of integration between list and document evaluation. The findings of this study suggest more elements, especially at list level, be available to support users applying their evaluation criteria. Integration of list and document evaluation and integration of pre, evaluation and post evaluation activities for the interface design is the absolute solution for effective evaluation.

This study fills a gap in current research in relation to the comparison of list and document evaluation.

Recent teacher evaluation reforms around the globe substantially increased the number of teacher observations, consequently raising observers' (typically school administrators') observational loads. The purpose of this study is to examine associations between observational loads and school administrator turnover, reported time use and strain.

The study uses education administrative data from the state of Tennessee to examine the link between observational loads and school administrator outcomes of interest. The results present credible regression estimates that isolate variation in observational loads within schools over time and within observers over time.

The evidence suggests individual school administrators allocate a set amount of time to observations that is insensitive to observational load and seemingly assign observations to colleagues strategically. School administrator reports do not suggest observational loads are associated with negative unintended consequences on administrator strain or observer turnover.

The study contributes to the literature on teacher evaluation by shedding light on how the constraints posed by an evaluation system may affect the work of school administrators. It also extends the job demands-resources theory that describes worker responses to new job demands.

The paper's aim is to assess the usefulness of the real time evaluation methodology in rapid‐onset humanitarian emergencies and to document recurring constraints and lessons learned.

A small online survey was carried out among evaluation interviewees and a desk study was commissioned to extrapolate good practice and learning points from nine evaluations carried out by one international organisation over the past three years.

Given the changing nature of the environment in which agencies work during rapid onset humanitarian emergencies, impact evaluations are often carried out too late for important programme changes to be made. The real time evaluation methodology has proved to be a reliable and useful tool for process evaluation during rapid‐onset emergencies. However, it is clear, if it does not take place within the first two months the chances of programmatic changes being made are minimal. There is still the issue of organisational learning and ensuring that lessons learned from one response are taken forward in future programmes. Although there are several risks associated with the methodology, these are outweighed by the advantages, both immediate and longer‐term.

The findings from these studies has been of value to the organisation but they are also of interest to the wider humanitarian sector struggling with the practicalities of process evaluations in a challenging environment.

Although there are several publications about real time evaluation methodology, this paper is one of the first attempts by a non‐UN organisation to critically review evaluation reports and to document the staff opinions.

Interest in the nature of influence attempts in the performance evaluation process has increased in recent years. Researchers have conducted a number of important and revealing cross‐sectional investigations, but there remains virtually no longitudinal work in this area. The present study attempted to address this need by conducting a multi‐period investigation of influence tactics use and affect that addressed three questions: (1) Are individuals consistent in their use of influence tactics across evaluation periods? (2) Are prior‐period performance ratings reflected in subsequent influence tactic use? (3) What role does affect, both supervisor and subordinate, play in this process? A latent variable structural model was tested using longitudinal data from managers and employees of food services units. Our results indicated that there is a cycle of continued influence tactic use across time periods, performance ratings help to determine subsequent tactic use, and both supervisor and subordinate affect play a role in the influence‐evaluation process. Implications of these results and directions for future research are discussed.

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning (ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and establishing a reliable relation with customers and undeniably to grow the business.

An experimental quantitative research method was conducted in which the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-real-time data set to uncover the best-fitting RE.

The rule-based automated evaluation of RE has changed the testing landscape, with the removal of longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with high-quality results and can possibly bring a new revolution in the testing practice to start a service line “Machine Learning Testing as a service” (MLTaaS) and the possibility of integrating with DevOps that can specifically help agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.

A small data set was considered for A/B phase study and was captured for ten movies from three theaters operating in a single location in India, and simulation phase study was captured for two movies from three theaters operating from the same location in India. The research was limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.

The best-fitting RE facilitates the business to make personalized recommendations, long-term customer loyalty forecasting, predicting the company's future performance, introducing customers to new products/services and shaping customer's future preferences and behaviors.

The proposed rule-based ML approach named “2-stage locking evaluation” is self-learned, automated by design and largely produces time-bound conclusive result and improved decision-making process. It is the first of a kind to examine the business domain and task of interest. In each stage of the evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution. Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with end-to-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by identifying the missing test metrics coverage.

The paper uses social cognitive theory to explore reactions to performance evaluation processes as situated cognitions by examining the relationship between key elements of employees’ schemas about an organizational environment, preparation for evaluation, and these reactions. The paper aims to discuss these issues.

Survey of 260 employees of eight organizations in a Midwestern US city.

Job resource adequacy, communication adequacy, coworker relationships, and preparation time are significantly and positively associated with employee reactions to performance evaluation processes. Preparation time moderates the association between organizational context and employee reactions.

A social cognitive perspective on performance evaluation broadens the scope of extant research. This study is limited by cross-sectional design but opens the door to future experimental and longitudinal research.

Performance evaluation processes are situated in an organizational context. Organizational interventions to improve perceptions of this key process could focus on better communication and encouraging preparation, especially if job resources are less adequate.

This research contributes to the performance evaluation literature by applying social cognitive theory to performance evaluation reactions as situated cognitions, calling attention to the broader organizational context in which these processes occur.

Current models of transaction credit in the e-commerce network face many problems, such as the one-sided measurement, low accuracy and insufficient anti-aggression solutions. This paper aims to address these problems by studying the transaction credit problem in the crowd transaction network.

This study divides the transaction credit into two parts, direct transaction credit and recommended transaction credit, and it proposes a model based on the crowd transaction network. The direct transaction credit comprehensively includes various factors influencing the transaction credit, including transaction evaluation, transaction time, transaction status, transaction amount and transaction times. The recommendation transaction credit introduces two types of recommendation nodes and constructs the recommendation credibility for each type. This paper also proposes a “buyer + circle of friends” method to store and update the transaction credit data.

The simulation results show that this model is superior with high accuracy and anti-aggression.

The direct transaction credit improves the accuracy of the transaction credit data. The recommendation transaction credit strengthens the anti-aggression of the transaction credit data. In addition, the “buyer + circle of friends” method fully uses the computing of the storage ability of the internet, and it also solves the failure problem of using a single node.

Management training and development is currently in vogue. There appears to be a growing belief in the benefits of investment in training and development. When a market is buoyant is the time to consider and anticipate the consequences of a future downturn in demand. Such a downturn in demand may demonstrate increasing pressure to “justify” investment in training and development. There is a long established academic body of knowledge on the subject of evaluating training and development. From research evidence and the authors′ experience, the sponsors and the providers of training and development pay scant attention to systematic evaluation of these activities and investments. It is the authors′ contention that when the market′s critical assessment of the value of training and development increases there will be an increasing interest in evaluation. An overview of the history of evaluation traditions is provided and the state of play is commented upon. It is noted that there is a shortfall between theory and practice. It is argued that evaluation is a worthwhile and important activity and ways through the evaluation literature maze and the underpinnings of the activity are demonstrated, especially to management. Similarly the literature on evaluation techniques is reviewed. Tables are provided which demonstrate areas of major activity and identify relatively uncharted waters. This monograph provides a resource whereby practitioners can choose techniques which are appropriate to the activity on which they are engaged. It highlights the process which should be undertaken to make that choice in order that needs of the major stakeholders in the exercise are fully met.

The purpose of this paper is to suggest indicators for evaluating Iran national nanotechnology plan at the policy and program levels. Governments are required to evaluate their policies to be accountable to their audience and to improve their policies. The evaluation of nanotechnology policies both in academic and governmental institutions has been mainly confined to measuring the research and development outputs, while explicit models are required for evaluating national nanotechnology policies and programs. In other words, although policies are translated into several activities, it does not mean that the evaluation of activities results in holistic policy evaluation.

The present research uses a qualitative method to explore the process of developing a model for assessing nanotechnology policymaking in Iran. Expert opinions have been collected and analyzed by using local and international surveys to validate different elements of the model.

The model includes the evaluation time-frame, criteria and indicators for policy and program levels. Assessment at program and policy levels, as distinct from activity evaluation, measures the ultimate objectives of a policy intervention, based on which the outcome and impact indicators should be developed. This paper argues that the Iran national nanotechnology policy should be evaluated and revised triennially based on the model in which 46 outcome and impact indicators are included under three general criteria of appropriateness, efficiency and effectiveness. A number of these indicators should be evaluated annually or biannually.

The evaluation of nanotechnology plans has been mainly limited to assessing the research and development outputs that are related to the activity level. The main value of this work is developing outcome and impact indicators that are related to policy and program levels.

The purpose of this paper is to understand the underlying logics applied by different project evaluation approaches and to propose an alternative research agenda.

This paper explores the project evaluation literature via conducting a qualitative research applying systematic literature review and thematic analysis.

The project evaluation literature has mainly concentrated on the objective aspects of project evaluation and overlooked the subjective aspects that reflect the temporal, dynamic, complex and subjective nature of today’s projects. The authors propose a meta-framework that helps project practitioners to select an appropriate project evaluation criterion for their projects by considering the strengths and limitations of their preferred project evaluation model as well as making project evaluators aware of the underlying logics associated to diverse project evaluation approaches.

This study suggests that new conceptual approaches to deal with some of the major challenges in the project evaluation field. Practice-based views, narrative analysis and actor-network theory are likely to be useful tools to better understand and cope with the projects’ uncertainty and complexity.

The findings of this research assist project management practitioners and particularly project evaluators to enhance their understanding of the subjectivity, complexity and dynamics of current projects. To increase the reflexivity and resilience of project evaluation practice, this study also proposes new directions to apply different criteria, sub-criteria and indicators to the evaluation practice.

The originality of this study relies on transcending the conventional objective and rational approaches prevailing in current project evaluation practices. It proposes a research agenda that pave the way to address the shortcomings of conventional project evaluation practice.