Search results

1 – 10 of over 69000
Article
Publication date: 1 June 1992

Patricia L. Rees

Puts forward a framework for carrying out user evaluations ofexpert systems, which is based on user evaluations carried out in alarge British company over a period of three years…

Abstract

Puts forward a framework for carrying out user evaluations of expert systems, which is based on user evaluations carried out in a large British company over a period of three years. The aim of this framework is to ensure that the aims of the expert system designers and the users of the system are congruent. This in turn will make it more likely that the expert system will be a success. The framework covers aspects such as user type, updating knowledge, training, the value of the system and the human/computer interface.

Details

Industrial Management & Data Systems, vol. 92 no. 6
Type: Research Article
ISSN: 0263-5577

Keywords

Article
Publication date: 7 June 2013

Lynne Hall, Susan Jane Jones, Ruth Aylett, Marc Hall, Sarah Tazzyman, Ana Paiva and Lynne Humphries

This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults.

Abstract

Purpose

This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults.

Design/methodology/approach

In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team.

Findings

The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in‐role game experience.

Practical implications

This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user.

Originality/value

Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in‐game experience that added value to the user experience whilst also yielding relevant results for the development team.

Details

Interactive Technology and Smart Education, vol. 10 no. 2
Type: Research Article
ISSN: 1741-5659

Keywords

Article
Publication date: 4 October 2011

Nadjla Hariri and Yaghoub Norouzi

The present study aims to review the literature concerning Digital Libraries (DLs) and user interfaces in order to identify, determine, and suggest evaluation criteria for a DLs…

4402

Abstract

Purpose

The present study aims to review the literature concerning Digital Libraries (DLs) and user interfaces in order to identify, determine, and suggest evaluation criteria for a DLs user interface. Accordingly, this study's objectives are threefold: explore which criteria exert a significant relationship with the DLs user interface; identify a set of criteria that appears to be useful for evaluating DLs user interface; and determine evaluation criteria that have more frequency and occurrence in the related texts reviewed.

Design/methodology/approach

To do it, first, identifying related texts was necessary. Consequently, keywords such as “DLs user interface evaluation”,” DLs user interfaces”, “DLs evaluation”, “DLs usability”, “user interface evaluation”, “DLs research”, “web sites user interface evaluation”, “user interface standards”, and the like have been searched in the web as well as in some leading databases including Emerald, Proquest, SagePub, ScienceDirect, LISA, ERIC, ACM, and Springer. After identifying and accessing more than 100 evaluative works and some related articles, theoretical and empirical, nearly 50 sources were chosen for final examination.

Findings

After reviewing related texts, three major categories are identified: user interface and DLs; DLs and usability; and other studies related to user interface; each one of three identified categories has its own subcategories. Additionally, 22 evaluation criteria for assessing DLs interface have been identified.

Research limitations/implications

The review does not claim to be comprehensive.

Practical implications

Hopefully, criteria such as feedback, ease of use, match between system and the real world, customization, user support, user workload, interaction, compatibility, visibility of system status, user experience, flexibility, and accessibility which have been less considered should be applied more in future, particularly user‐oriented, studies. Furthermore, it is expected that criteria mentioned here could help related bodies pay more attention to the evaluation of EISs, especially DLs interface.

Originality/value

It can be said that this study has contributed to the research into the evaluation of DL interface.

Details

The Electronic Library, vol. 29 no. 5
Type: Research Article
ISSN: 0264-0473

Keywords

Article
Publication date: 1 April 2004

Scott Nicholson

This conceptual piece presents a framework to aid libraries in gaining a more thorough and holistic understanding of their users and services. Through a presentation of the…

3326

Abstract

This conceptual piece presents a framework to aid libraries in gaining a more thorough and holistic understanding of their users and services. Through a presentation of the history of library evaluation, a multidimensional matrix of measures is developed that demonstrates the relationship between the topics and perspectives of measurement. These measurements are then combined through evaluation criteria, and then different participants in the library system view those criteria for decision making. By implementing this framework for holistic measurement and cumulative evaluation, library evaluators can gain a more holistic knowledge of the library system and library administrators can be better informed for their decision‐making processes.

Details

Journal of Documentation, vol. 60 no. 2
Type: Research Article
ISSN: 0022-0418

Keywords

Article
Publication date: 1 November 2006

Sudatta Chowdhury, Monica Landoni and Forbes Gibb

The main objective of this paper is to review work on the usability and impact of digital libraries.

7196

Abstract

Purpose

The main objective of this paper is to review work on the usability and impact of digital libraries.

Design/methodology/approach

Work on the usability and impact of digital libraries is reviewed. Specific studies on the usability and impact of digital libraries in specific domains are also discussed in order to identify general and specific usability and impact measures.

Findings

The usability studies reviewed in this paper show that a number of approaches have been used to assess usability. In addition to the technical aspects of digital library design (e.g. architecture, interfaces and search tools), there are a number of usability issues such as globalisation, localisation, language, culture issues, content and human information behaviour. Digital libraries should, however, be evaluated primarily with respect to their target users, applications and contexts.

Research limitations/implications

Although a digital library evaluation study may have several objectives, ranging from the evaluation of its design and architecture to the evaluation of its usability and its impact on the target users, this paper focuses on usability and impact.

Originality/value

This paper provides insights into the state‐of‐the art in relation to the usability and impact of digital libraries.

Details

Online Information Review, vol. 30 no. 6
Type: Research Article
ISSN: 1468-4527

Keywords

Article
Publication date: 14 August 2020

Rajat Kumar Behera, Pradip Kumar Bala and Rashmi Jain

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain…

Abstract

Purpose

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning (ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and establishing a reliable relation with customers and undeniably to grow the business.

Design/methodology/approach

An experimental quantitative research method was conducted in which the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-real-time data set to uncover the best-fitting RE.

Findings

The rule-based automated evaluation of RE has changed the testing landscape, with the removal of longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with high-quality results and can possibly bring a new revolution in the testing practice to start a service line “Machine Learning Testing as a service” (MLTaaS) and the possibility of integrating with DevOps that can specifically help agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.

Research limitations/implications

A small data set was considered for A/B phase study and was captured for ten movies from three theaters operating in a single location in India, and simulation phase study was captured for two movies from three theaters operating from the same location in India. The research was limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.

Practical implications

The best-fitting RE facilitates the business to make personalized recommendations, long-term customer loyalty forecasting, predicting the company's future performance, introducing customers to new products/services and shaping customer's future preferences and behaviors.

Originality/value

The proposed rule-based ML approach named “2-stage locking evaluation” is self-learned, automated by design and largely produces time-bound conclusive result and improved decision-making process. It is the first of a kind to examine the business domain and task of interest. In each stage of the evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution. Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with end-to-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by identifying the missing test metrics coverage.

Article
Publication date: 11 November 2021

Chunnian Liu, Qi Tian and Xiaogang Zhu

This study aimed to analyze existing problems in the dissemination and management of emergency information on social media platforms, improve social media users' experience…

Abstract

Purpose

This study aimed to analyze existing problems in the dissemination and management of emergency information on social media platforms, improve social media users' experience regarding such information, increase the efficiency of emergency information dissemination and curb the spread of misinformation.

Design/methodology/approach

In this study, the emergency information quality on social media platforms was examined. Based on the evaluation principles of the quality of mature information, social media information characteristics and the rules of emergency information dissemination, combined with relevant academic research results, an index to evaluate the quality of emergency information on social media was constructed. In addition, the authors have introduced cloud theory as an information quality evaluation method and used social media users' emotional characteristics to assess information quality evaluation results. A comprehensive system for evaluating emergency information quality, including indexes, methods and detection strategies was established. Based on a comprehensive system, a case study was conducted on the forest fires in Sichuan Province and the African swine fever events as reported on the Zhihu platform. In accordance with the results of the case study, the authors expanded the research and introduced the emotional characteristics of social media users as an independent evaluation dimension to evaluate the quality of emergency information on social media.

Findings

The comprehensive system's effectiveness was verified through the case study. Further, it was found that users' emotional characteristics (reflected in their information behavior) are inconsistent with their evaluation of websites' information quality regarding major emergencies. Integrating users' emotional characteristics into the information evaluation system can enhance its effectiveness following major emergencies.

Originality/value

First, an evaluation index system of emergency information quality on social media about major emergencies was offered. Unlike the commonly available index system for information quality evaluation, this proposed evaluation index system not only accounted for the characteristics of social media, such as massive disordered information, multiple information sources and rapid dissemination, but also for the characteristics of emergency events, such as variability and the absence of precursors. This proposed evaluation index system enhances the pertinence of the information quality evaluation and compensates for the shortcoming that the current research only focuses on evaluating social media information quality in a broad context, but pays insufficient attention to major emergencies. Second, cloud theory was introduced as a method to evaluate the emergency information quality found on social media. Existing research has primarily included the use of traditional statistical methods, which cannot transform numerical values into qualitative concepts effectively. Various indeterminate factors inevitably affect the quality of emergency information on social media platforms, and the traditional methods cannot eliminate this uncertainty in the evaluation process. The method to assess emergency information quality based on cloud theory can effectively compensate for the gaps in the research and improve the accuracy of information quality assessment. Third, the inspection and the dynamic adjustment of assessment results are absent in the research on information quality assessment, and the research has relied principally on the information users' evaluation and has paid insufficient attention to their attitudes and behaviors toward information. Therefore, the authors incorporated users' emotional characteristics into the evaluation of emergency information quality on social media and used them to test the evaluation results so that the results of the information quality assessment not only include the users' explicit attitudes but also their implicit attitudes. This enhances the effectiveness of the information quality assessment system. Finally, through this case study, it was found that an inconsistency exists between user evaluation and user emotional characteristics after major emergencies. The reasons for this phenomenon were explained, and the necessity of integrating user emotional characteristics into information quality assessment was demonstrated. Based on this, the users' emotional characteristics were used as a separate evaluation dimension for assessing the quality of emergency information on social media. Compared with assessing the quality of general information, integrating the user's emotional characteristics into the evaluation index system can lead the evaluation results to include not only the users' cognitive evaluation but also their emotional experience, further enhancing their adaptability.

Details

Library Hi Tech, vol. 41 no. 3
Type: Research Article
ISSN: 0737-8831

Keywords

Article
Publication date: 3 December 2018

Huawei Zhu, Rungting Tu, Wenting Feng and Jiaojiao Xu

Extreme online reviews can have great impacts on consumers’ purchase decisions. The purpose of this paper is to investigate when users are more likely to provide extreme ratings…

Abstract

Purpose

Extreme online reviews can have great impacts on consumers’ purchase decisions. The purpose of this paper is to investigate when users are more likely to provide extreme ratings. The study draws inference from attitude certainty theory and proposes that review extremity is influenced by the interaction of evaluation duration and product/service types: for hedonic products/services, shorter evaluation duration can foster attitude certainty, leading to higher review extremity; in contrast, for utilitarian products/services, longer evaluation duration can increase attitude certainty, resulting in more extreme reviews.

Design/methodology/approach

Three studies were conducted to test the hypotheses: Study 1 is an empirical analysis of 3,000 reviews from an online retailing website; Studies 2 and 3 are two between-subject experiments.

Findings

Results from three studies confirm the hypotheses. Study 1 provides preliminary evidence on how review extremity varies in evaluations of different durations and product/service types. Results from Studies 2 and 3 show that for hedonic products/services, the shorter the evaluation duration, the more likely users are to give extreme ratings; however, for utilitarian products/service, the longer the evaluation duration, the more likely users are to give extreme reviews; and attitude certainty plays a mediating role between evaluation duration and review extremity.

Originality/value

Findings from this study provide understandings on when a fast rather than a slow evaluation can lead to more extreme reviews. The results also highlight the role of users’ attitude certainty in the underlying mechanism.

Details

Online Information Review, vol. 43 no. 5
Type: Research Article
ISSN: 1468-4527

Keywords

Article
Publication date: 16 May 2008

Vassilios P. Aggelidis and Prodromos D. Chatzoglou

It is widely accepted that the use of information and communication technology (ICT) in the healthcare sector offers great potential for improving the quality of services…

2216

Abstract

Purpose

It is widely accepted that the use of information and communication technology (ICT) in the healthcare sector offers great potential for improving the quality of services provided, the efficiency and effectiveness of personnel, and also reducing organizational expenses. This paper seeks to examine various hospital information system (HIS) evaluation methods.

Design/methodology/approach

In this paper a comprehensive search of the literature concerning the evaluation of complex health information systems is conducted and used to generate a synthesis of the literature around evaluation efforts in this field. Three approaches for evaluating hospital information systems are presented – user satisfaction, usage, and economic evaluation.

Findings

The main results are that during the past decade, computers and information systems, as well as their resultant products, have pervaded hospitals worldwide. Unfortunately, methodologies to measure the various impacts of these systems have not evolved at the same pace. To summarize, measurement of users' satisfaction with information systems may be the most effective evaluation method in comparison with the rest of the methods presented.

Practical implications

The methodologies, taxonomies and concepts presented in this paper could benefit researchers and practitioners in the evaluation of HISs.

Originality/value

This review points out the need for more thorough evaluations of HISs that look at a wide range of factors that can affect the relative success or failure of these systems.

Details

EuroMed Journal of Business, vol. 3 no. 1
Type: Research Article
ISSN: 1450-2194

Keywords

Article
Publication date: 15 February 2008

Lili Luo

This paper aims to provide a holistic view of the current practice of chat reference evaluation and to suggest a framework that could help reference practitioners evaluate chat…

3212

Abstract

Purpose

This paper aims to provide a holistic view of the current practice of chat reference evaluation and to suggest a framework that could help reference practitioners evaluate chat reference services in multiple contexts.

Design/methodology/approach

A thorough review of the literature on chat reference evaluation is conducted and the evaluation studies are grouped by their evaluative perspective and measures. Based on the literature review, a framework of perspectives and measures for chat reference evaluation is proposed.

Findings

Chat reference evaluation has incorporated a number of new elements that do not exist in desk reference evaluation. All the evaluation perspectives and measures reported in chat reference literature are incorporated into a final framework except for one variable – “cost‐effectiveness” – which lacks the support of empirical studies in the literature.

Practical implications

The analytical review of the literature provides a holistic view of how chat reference is being measured for its value to both libraries and users, thus furthering the professional understanding of chat reference performance in the library environs. The framework of evaluation perspectives and measures resulting from the literature review is applicable in multiple chat reference contexts and can be customized to serve different evaluation purposes. In turn, this framework gives chat reference evaluators a clear idea of what to look at and how.

Originality/value

This paper fills the need to provide reference practitioners with both a critical view of current chat reference practice, and a tool that could help them design and develop a chat reference evaluation project.

Details

Reference Services Review, vol. 36 no. 1
Type: Research Article
ISSN: 0090-7324

Keywords

1 – 10 of over 69000