Search results

The roles recommendation systems play in health care have become crucial in achieving effective care and in meeting the needs of modern care giving. As a result, efforts have been geared toward using recommendation systems in the management of chronic diseases. Effectiveness of these systems is determined by evaluation following implementation and before deployment, using certain metrics and criteria. The purpose of this study is to ascertain whether consideration of criteria during the design of a recommendation system can increase acceptance and usefulness of the recommendation system.

Using survey-style requirements gathering method, the specific health and technology needs of people living with chronic diseases were gathered. The result was analyzed using quantitative method. Sets of harmonized criteria and metrics were used along with requirements gathered from stakeholders to establish relationship among the criteria and the requirements. A matching matrix was used to isolate requirements for prioritization. These requirements were used in the design of a mobile app.

Matching criteria against requirements highlights three possible matches, namely, exact, inferential and zero matches. In any of these matches, no requirement was discarded. This allows priority features of the system to be isolated and accorded high priority during the design. This study highlights the possibility of increasing the acceptance rate and usefulness of a recommendation system by using metrics and criteria as a guide during the design process of recommendation systems in health care. This approach was applied in the design of a mobile app called Recommendations Sharing Community for Aged and Chronically Ill People. The result has shown that with this method, it is possible to increase acceptance rate, robustness and usefulness of the product.

Inability to know the evaluation criteria beforehand, inability to do functional analysis of requirements, lack of well-defined requirements and often poor cooperation from people living with chronic diseases during requirements gathering for fear of stigmatization, confidentiality and privacy breaches are possible limitations to this study.

The result has shown that with this method, it is possible to isolate more important features of the system and use them during the design process, thereby speeding up the design and increasing acceptance rate, robustness and usefulness of the system. It also helps to see in advance the likely features of the system that will enhance its usefulness and acceptance, thereby increasing the confidence of the developers in their ability to deliver a system that will meet users’ needs. As a result, developers know beforehand where to concentrate their efforts during system development to ascertain the possibility of increasing usefulness and acceptance rate of a recommendation system. In addition, it will also save time and cost.

This paper demonstrates originality by highlighting and testing the possibility of using evaluation criteria and metrics during the design of a recommender system with a view to increasing acceptance and enhancing usefulness. It also shows the possibility of using the metrics and criteria in system’s development process for an exercise other than evaluation.

Competitive sourcing is the government’s term for transferring the operation of an internal process or function to either an external supplier or a reengineered government team. The competitively sourced function is managed through performance metrics. These metrics must be thorough, appropriate and well designed to ensure the government is receiving the level of service required to fulfill its various missions. This research effort develops a performance metric evaluation system that was synthesized from metric design literature, Total Quality Management concepts, and the Government Performance Results Act. Use of the system in a case study is discussed along with how to evaluate the results. Results indicate that some Air Force performance metrics have insufficient and improperly designed metrics.

The purpose of this paper is to develop a nuanced framework for evaluating a channel partner’s performance in distribution channel relationships. Given a channel partner’s task environment characteristics (high/low munificence, dynamism and complexity), the study examines which performance metrics (output, activity or capability) are most relevant for evaluating its performance levels effectively.

The study adopts self-administered cross-sectional survey-based research design. Matched data were collected from 252 channel partners – manager relationship dyads. The latent change score (LCS) model within SEM framework provides mean paired-differences of the relevance ratings for each metrics. This was used to assess the empirical validity of the hypothesized relationships.

The study demonstrates the importance of calibrating performance evaluation metrics to a channel partner’s task environment state, made possible by its holistic approach to performance evaluation. Based on an extensive analysis, it shows that no single metric is relevant within all environmental states; rather, it could be dysfunctional, a result that differs from vast majority of the literature.

Investigates individual linkages between task environment dimensions and performance metrics to provide a fuller understanding of these relationships. Also provides a theoretical framework to support further research on the topic.

The study provides managerial guidelines (and extensive graphical analysis) for nuanced and dynamic evaluation of channel partners’ performance that can enable firms to identify and promote their most valuable channel partners and prevent the deterioration of others.

First one to develop and empirically validate a nuanced framework for evaluating performance of exchange partners that operate under diverse task environment states.

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning (ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and establishing a reliable relation with customers and undeniably to grow the business.

An experimental quantitative research method was conducted in which the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-real-time data set to uncover the best-fitting RE.

The rule-based automated evaluation of RE has changed the testing landscape, with the removal of longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with high-quality results and can possibly bring a new revolution in the testing practice to start a service line “Machine Learning Testing as a service” (MLTaaS) and the possibility of integrating with DevOps that can specifically help agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.

A small data set was considered for A/B phase study and was captured for ten movies from three theaters operating in a single location in India, and simulation phase study was captured for two movies from three theaters operating from the same location in India. The research was limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.

The best-fitting RE facilitates the business to make personalized recommendations, long-term customer loyalty forecasting, predicting the company's future performance, introducing customers to new products/services and shaping customer's future preferences and behaviors.

The proposed rule-based ML approach named “2-stage locking evaluation” is self-learned, automated by design and largely produces time-bound conclusive result and improved decision-making process. It is the first of a kind to examine the business domain and task of interest. In each stage of the evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution. Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with end-to-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by identifying the missing test metrics coverage.

The purpose of this paper is to outline the contribution of marketing to program evaluation in the school sector. Schools are increasingly the target of government-funded environmental education initiatives and this paper aims to illustrate, through a sector-wide program case study, how marketing metrics can improve overall program evaluation. Existing school-based program evaluations are often not accompanied by rigorous evaluation of their impact beyond educational outcomes. Evaluation focuses instead on improving satisfaction of those already participating, rather than looking at the wider issues of program adoption and engagement levels across the sector. This paper also aims to look at how traditional marketing's evaluation metrics can address this gap.

A case study is used involving a sector-wide recycling program whose objective is to reduce waste across all schools across a State in Australia. The program, administered by a government agency, had only been evaluated within an educational outcome context. Using existing data on the program from across the school sector, marketing metrics are calculated to provide new insight into the program's wider impact.

This research illustrates the relevance of marketing metrics to educational sector activities. It illustrates how to embed metrics into the program and identifies insights they can offer as a supplement to existing educational outcome measures. Such measures are highly useful to funding bodies deciding on a program's development and continuation.

Marketing provides empirically-based program metrics that are easy and cost-effective to obtain, objective in their measure, and provide feedback loops to participants. Having impacts more clearly measured allows for effective program administration within the childhood education sector. This paper delivers practical guidelines for program administrators.

The paper brings marketing into an environmental education context, illustrating its contribution for better measurement of behaviour change. It gives marketing practitioners and academic researchers a framework within which they can use already available program data to better gauge the uptake and impact of their efforts.

Citations have been used as a common basis to measure the academic accomplishments of scientific books. However, traditional citation analysis ignored content mining and without consideration of citation equivalence, which may lead to the decline of evaluation reliability. Hence, this paper aims to integrate multi-level citation information to conduct multi-dimensional analysis.

In this paper, books’ academic impacts were measured by integrating multi-level citation resources, including books’ citation frequencies and citation-related contents. Specifically, firstly, books’ citation frequencies were counted as the frequency-level metric. Secondly, content-level metrics were detected from multi-dimensional citation contents based on finer-grained mining, including topic extraction on the metadata and citation classification on the citation contexts. Finally, differential metric weighting methods were compared with integrate the multi-level metrics and computing books’ academic impacts.

The experimental results indicate that the integration of multiple citation resources is necessary, as it can significantly improve the comprehensiveness of the evaluation results. Meanwhile, compared with the type differences of books, disciplinary differences need more attention when evaluating the academic impacts of books.

Academic impact assessment of books via integrating multi-level citation information can provide more detailed evaluation information and cover shortcomings of methods based on single citation data. Moreover, the method proposed in this paper is publication independent, which can be used to measure other publications besides books.

This paper aims to understand the nature of citations and metrics in the larger system of knowledge production involving universities, funding agencies, publishers, and indexing and data analytic services.

First, the normative and social constructivist views of citations are reviewed to be understood as co-existing conditions. Second, metrics are examined through the processes of commensuration by tracing the meanings of metrics embedded in various kinds of documents and contexts. Third, the steering effects of citations and metrics on knowledge production are discussed. Finally, the conclusion addresses questions pertaining to the validity and legitimacy of citations as data and their implications for knowledge production and the conception of information.

The normative view of citations is understood as an ideal speech situation; the social constructivist view of citation is recognised in the system of knowledge production where citing motivations are influenced by epistemic, social and political factors. When organisational performances are prioritised and generate system imperatives, motives of competition become dominant in shaping citing behaviour, which can deviate from the norms and values in the academic lifeworld. As a result, citations and metrics become a non-linguistic steering medium rather than evidence of research quality and impact.

This paper contributes to the understanding of the nature of citations and metrics and their implications for the conception of information and knowledge production.

Destination marketing systems (DMS) represent a vital inter-organisational information system (IOIS) for supporting the collaborative e-marketing strategies of tourism firms and the competitiveness of tourism destinations. However, many DMS have failed to deliver the expected outcomes, while the performance measurement of DMS has not been thoroughly investigated in the literature so far. The study synthesises research from the fields of DMS, IOIS and collaborative practices for investigating the perceptions of various tourism DMS stakeholders about the evaluation of DMS performance. The paper aims to discuss these issues.

The study conducted a nation-wide survey for measuring the perceptions of various tourism DMS stakeholders in Greece about the importance of the roles that DMS should serve as well as the items that should be used for measuring the performance of these DMS’ roles.

The findings showed that the public and private stakeholders held different perceptions about the roles of DMS as well as about the metrics that need to be used for evaluating DMS performance. The findings also showed that the perceptions that stakeholders hold about the roles of the DMS influence their perceptions about the performance evaluation of DMS.

The findings are based on evaluating a specific type of IOIS and sector/context. Thus, caution is required in generalising the results to other types of IOIS and social/environmental contexts.

The study highlighted that the performance and success of DMS, and of IOIS projects in general, require the nurturing of a collaborative culture and the co-ordination of the various stakeholders’ perceptions and interests.

The study addresses the gap in DMS performance evaluation and it contributes to the literature about IOIS evaluation by adopting a stakeholders approach.

The purpose of this study is to provide a framework to identify performance metrics for evaluating research and development collaborations.

The framework is developed through a review of similar centres and academic studies, followed by surveys and interviews of researchers and industry practitioners for the case of the Construction Innovation Centre (CIC). The proposed framework consists of identification of existing industry research and development needs, development of a research roadmap representing top research priorities, and identification of the most important services to provide to industry partners, which form the context for defining performance evaluation metrics.

A research roadmap is presented, outlining top research areas and methods and a list of the most in-demand services including research, practical and training and outreach services. Metrics for evaluating the performance of proposed projects, completed projects and a collaborative research centre are also identified.

This study presents a novel approach to defining performance metrics for the evaluation of research and development collaborations. The approach and findings of this study can be adopted by other collaborative research centres and initiatives around the world to develop effective metrics for performance measurement. The proposed framework provides a platform for defining performance metrics in the context of the research roadmap and top-priority services applicable to the research and development collaboration.