Search results

1 – 10 of over 64000
To view the access options for this content please click here
Article
Publication date: 1 June 1992

Patricia L. Rees

Puts forward a framework for carrying out user evaluations ofexpert systems, which is based on user evaluations carried out in alarge British company over a period of…

Abstract

Puts forward a framework for carrying out user evaluations of expert systems, which is based on user evaluations carried out in a large British company over a period of three years. The aim of this framework is to ensure that the aims of the expert system designers and the users of the system are congruent. This in turn will make it more likely that the expert system will be a success. The framework covers aspects such as user type, updating knowledge, training, the value of the system and the human/computer interface.

Details

Industrial Management & Data Systems, vol. 92 no. 6
Type: Research Article
ISSN: 0263-5577

Keywords

To view the access options for this content please click here
Article
Publication date: 7 June 2013

Lynne Hall, Susan Jane Jones, Ruth Aylett, Marc Hall, Sarah Tazzyman, Ana Paiva and Lynne Humphries

This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults.

Abstract

Purpose

This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults.

Design/methodology/approach

In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team.

Findings

The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in‐role game experience.

Practical implications

This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user.

Originality/value

Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in‐game experience that added value to the user experience whilst also yielding relevant results for the development team.

Details

Interactive Technology and Smart Education, vol. 10 no. 2
Type: Research Article
ISSN: 1741-5659

Keywords

To view the access options for this content please click here
Article
Publication date: 4 October 2011

Nadjla Hariri and Yaghoub Norouzi

The present study aims to review the literature concerning Digital Libraries (DLs) and user interfaces in order to identify, determine, and suggest evaluation criteria for…

Abstract

Purpose

The present study aims to review the literature concerning Digital Libraries (DLs) and user interfaces in order to identify, determine, and suggest evaluation criteria for a DLs user interface. Accordingly, this study's objectives are threefold: explore which criteria exert a significant relationship with the DLs user interface; identify a set of criteria that appears to be useful for evaluating DLs user interface; and determine evaluation criteria that have more frequency and occurrence in the related texts reviewed.

Design/methodology/approach

To do it, first, identifying related texts was necessary. Consequently, keywords such as “DLs user interface evaluation”,” DLs user interfaces”, “DLs evaluation”, “DLs usability”, “user interface evaluation”, “DLs research”, “web sites user interface evaluation”, “user interface standards”, and the like have been searched in the web as well as in some leading databases including Emerald, Proquest, SagePub, ScienceDirect, LISA, ERIC, ACM, and Springer. After identifying and accessing more than 100 evaluative works and some related articles, theoretical and empirical, nearly 50 sources were chosen for final examination.

Findings

After reviewing related texts, three major categories are identified: user interface and DLs; DLs and usability; and other studies related to user interface; each one of three identified categories has its own subcategories. Additionally, 22 evaluation criteria for assessing DLs interface have been identified.

Research limitations/implications

The review does not claim to be comprehensive.

Practical implications

Hopefully, criteria such as feedback, ease of use, match between system and the real world, customization, user support, user workload, interaction, compatibility, visibility of system status, user experience, flexibility, and accessibility which have been less considered should be applied more in future, particularly user‐oriented, studies. Furthermore, it is expected that criteria mentioned here could help related bodies pay more attention to the evaluation of EISs, especially DLs interface.

Originality/value

It can be said that this study has contributed to the research into the evaluation of DL interface.

Details

The Electronic Library, vol. 29 no. 5
Type: Research Article
ISSN: 0264-0473

Keywords

To view the access options for this content please click here
Article
Publication date: 1 April 2004

Scott Nicholson

This conceptual piece presents a framework to aid libraries in gaining a more thorough and holistic understanding of their users and services. Through a presentation of…

Abstract

This conceptual piece presents a framework to aid libraries in gaining a more thorough and holistic understanding of their users and services. Through a presentation of the history of library evaluation, a multidimensional matrix of measures is developed that demonstrates the relationship between the topics and perspectives of measurement. These measurements are then combined through evaluation criteria, and then different participants in the library system view those criteria for decision making. By implementing this framework for holistic measurement and cumulative evaluation, library evaluators can gain a more holistic knowledge of the library system and library administrators can be better informed for their decision‐making processes.

Details

Journal of Documentation, vol. 60 no. 2
Type: Research Article
ISSN: 0022-0418

Keywords

To view the access options for this content please click here
Article
Publication date: 14 August 2020

Rajat Kumar Behera, Pradip Kumar Bala and Rashmi Jain

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and…

Abstract

Purpose

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning (ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and establishing a reliable relation with customers and undeniably to grow the business.

Design/methodology/approach

An experimental quantitative research method was conducted in which the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-real-time data set to uncover the best-fitting RE.

Findings

The rule-based automated evaluation of RE has changed the testing landscape, with the removal of longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with high-quality results and can possibly bring a new revolution in the testing practice to start a service line “Machine Learning Testing as a service” (MLTaaS) and the possibility of integrating with DevOps that can specifically help agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.

Research limitations/implications

A small data set was considered for A/B phase study and was captured for ten movies from three theaters operating in a single location in India, and simulation phase study was captured for two movies from three theaters operating from the same location in India. The research was limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.

Practical implications

The best-fitting RE facilitates the business to make personalized recommendations, long-term customer loyalty forecasting, predicting the company's future performance, introducing customers to new products/services and shaping customer's future preferences and behaviors.

Originality/value

The proposed rule-based ML approach named “2-stage locking evaluation” is self-learned, automated by design and largely produces time-bound conclusive result and improved decision-making process. It is the first of a kind to examine the business domain and task of interest. In each stage of the evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution. Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with end-to-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by identifying the missing test metrics coverage.

To view the access options for this content please click here
Article
Publication date: 7 July 2020

Younghee Noh

This study aims to develop an evaluation index to evaluate the economic value among the values of the library and also attempts to measure the economic value of the…

Abstract

Purpose

This study aims to develop an evaluation index to evaluate the economic value among the values of the library and also attempts to measure the economic value of the library by performing a survey of the librarians and users at public libraries. The results of this research would likely encourage the librarians to feel increasingly confident about the library, while students and users, in general, would also likely be able to realize the economic value and presence of the library in more perceptive ways, thereby contributing to the activation of the library.

Design/methodology/approach

This study pertains to the development of an evaluation index for assessing the economic value of the library and, to evaluate the economic value of the library, has derived a preliminary evaluation index by collectively gathering and analyzing the domestic and foreign papers on the value of the library in its first phase. The preliminary evaluation index derived was verified by conducting three phases of Delphi survey by selecting ten experts. The survey questionnaire was developed to measure the economic value based on the final evaluation index derived from this study, and the economic value was measured against the perceptions of librarians and users of the public library.

Findings

The economic value of the library was divided into the four categories of the local economy’s value enhancement, namely, connection with the local community, human resources development, job creation and investment value enhancement for the librarians and users surveyed for assessment purposes. Consequently, the area of connection with the local community turned out to be the highest at 4.15, followed by 4.02 of the investment value improvement, 3.58 for the local economy’s value improvement and 3.50 for the human resources development and job creation, respectively. Furthermore, the respondents demonstrated the highest level of consensus on average on how the resource sharing by libraries has helped to reduce the economic burden for the residents as a matter of social value for the public library while believing that the libraries deliver a high level of return on social investments.

Originality/value

There are not that many studies conducted on the economic influence or the value of public libraries in Korea, and they are merely referenced in part if and when referenced to the overall value of the library. Given that, the research that focuses only on the economic value of the library must be carried out. In this respect, this research has been quite meaningful. The evaluation index developed in this research is likely to become a basic tool that can be applied to public libraries, as well as other types of libraries. Furthermore, the evaluation index developed through this research could be applied to nonprofit organizations, such as libraries, and would likely have a social ripple effect as a research that evaluates and presents the economic value of libraries. Accordingly, in this research, we have analyzed the list presented by the American Library Association and domestic research results, and have also structured the core details and derived the preliminary economic value index. Finally, 4 evaluation areas, 7 evaluation items and 22 evaluation indicators have been developed through the Delphi survey through three phases.

To view the access options for this content please click here
Article
Publication date: 1 November 2006

Sudatta Chowdhury, Monica Landoni and Forbes Gibb

The main objective of this paper is to review work on the usability and impact of digital libraries.

Abstract

Purpose

The main objective of this paper is to review work on the usability and impact of digital libraries.

Design/methodology/approach

Work on the usability and impact of digital libraries is reviewed. Specific studies on the usability and impact of digital libraries in specific domains are also discussed in order to identify general and specific usability and impact measures.

Findings

The usability studies reviewed in this paper show that a number of approaches have been used to assess usability. In addition to the technical aspects of digital library design (e.g. architecture, interfaces and search tools), there are a number of usability issues such as globalisation, localisation, language, culture issues, content and human information behaviour. Digital libraries should, however, be evaluated primarily with respect to their target users, applications and contexts.

Research limitations/implications

Although a digital library evaluation study may have several objectives, ranging from the evaluation of its design and architecture to the evaluation of its usability and its impact on the target users, this paper focuses on usability and impact.

Originality/value

This paper provides insights into the state‐of‐the art in relation to the usability and impact of digital libraries.

Details

Online Information Review, vol. 30 no. 6
Type: Research Article
ISSN: 1468-4527

Keywords

To view the access options for this content please click here
Article
Publication date: 3 December 2018

Huawei Zhu, Rungting Tu, Wenting Feng and Jiaojiao Xu

Extreme online reviews can have great impacts on consumers’ purchase decisions. The purpose of this paper is to investigate when users are more likely to provide extreme…

Abstract

Purpose

Extreme online reviews can have great impacts on consumers’ purchase decisions. The purpose of this paper is to investigate when users are more likely to provide extreme ratings. The study draws inference from attitude certainty theory and proposes that review extremity is influenced by the interaction of evaluation duration and product/service types: for hedonic products/services, shorter evaluation duration can foster attitude certainty, leading to higher review extremity; in contrast, for utilitarian products/services, longer evaluation duration can increase attitude certainty, resulting in more extreme reviews.

Design/methodology/approach

Three studies were conducted to test the hypotheses: Study 1 is an empirical analysis of 3,000 reviews from an online retailing website; Studies 2 and 3 are two between-subject experiments.

Findings

Results from three studies confirm the hypotheses. Study 1 provides preliminary evidence on how review extremity varies in evaluations of different durations and product/service types. Results from Studies 2 and 3 show that for hedonic products/services, the shorter the evaluation duration, the more likely users are to give extreme ratings; however, for utilitarian products/service, the longer the evaluation duration, the more likely users are to give extreme reviews; and attitude certainty plays a mediating role between evaluation duration and review extremity.

Originality/value

Findings from this study provide understandings on when a fast rather than a slow evaluation can lead to more extreme reviews. The results also highlight the role of users’ attitude certainty in the underlying mechanism.

Details

Online Information Review, vol. 43 no. 5
Type: Research Article
ISSN: 1468-4527

Keywords

To view the access options for this content please click here
Article
Publication date: 16 May 2008

Vassilios P. Aggelidis and Prodromos D. Chatzoglou

It is widely accepted that the use of information and communication technology (ICT) in the healthcare sector offers great potential for improving the quality of services…

Abstract

Purpose

It is widely accepted that the use of information and communication technology (ICT) in the healthcare sector offers great potential for improving the quality of services provided, the efficiency and effectiveness of personnel, and also reducing organizational expenses. This paper seeks to examine various hospital information system (HIS) evaluation methods.

Design/methodology/approach

In this paper a comprehensive search of the literature concerning the evaluation of complex health information systems is conducted and used to generate a synthesis of the literature around evaluation efforts in this field. Three approaches for evaluating hospital information systems are presented – user satisfaction, usage, and economic evaluation.

Findings

The main results are that during the past decade, computers and information systems, as well as their resultant products, have pervaded hospitals worldwide. Unfortunately, methodologies to measure the various impacts of these systems have not evolved at the same pace. To summarize, measurement of users' satisfaction with information systems may be the most effective evaluation method in comparison with the rest of the methods presented.

Practical implications

The methodologies, taxonomies and concepts presented in this paper could benefit researchers and practitioners in the evaluation of HISs.

Originality/value

This review points out the need for more thorough evaluations of HISs that look at a wide range of factors that can affect the relative success or failure of these systems.

Details

EuroMed Journal of Business, vol. 3 no. 1
Type: Research Article
ISSN: 1450-2194

Keywords

To view the access options for this content please click here
Article
Publication date: 15 February 2008

Lili Luo

This paper aims to provide a holistic view of the current practice of chat reference evaluation and to suggest a framework that could help reference practitioners evaluate…

Abstract

Purpose

This paper aims to provide a holistic view of the current practice of chat reference evaluation and to suggest a framework that could help reference practitioners evaluate chat reference services in multiple contexts.

Design/methodology/approach

A thorough review of the literature on chat reference evaluation is conducted and the evaluation studies are grouped by their evaluative perspective and measures. Based on the literature review, a framework of perspectives and measures for chat reference evaluation is proposed.

Findings

Chat reference evaluation has incorporated a number of new elements that do not exist in desk reference evaluation. All the evaluation perspectives and measures reported in chat reference literature are incorporated into a final framework except for one variable – “cost‐effectiveness” – which lacks the support of empirical studies in the literature.

Practical implications

The analytical review of the literature provides a holistic view of how chat reference is being measured for its value to both libraries and users, thus furthering the professional understanding of chat reference performance in the library environs. The framework of evaluation perspectives and measures resulting from the literature review is applicable in multiple chat reference contexts and can be customized to serve different evaluation purposes. In turn, this framework gives chat reference evaluators a clear idea of what to look at and how.

Originality/value

This paper fills the need to provide reference practitioners with both a critical view of current chat reference practice, and a tool that could help them design and develop a chat reference evaluation project.

Details

Reference Services Review, vol. 36 no. 1
Type: Research Article
ISSN: 0090-7324

Keywords

1 – 10 of over 64000