Search results

In vol. 6, 1976, of Advances in Librarianship, I published a review about relevance under the same title, without, of course, “Part I” in the title (Saracevic, 1976). [A substantively similar article was published in the Journal of the American Society for Information Science (Saracevic, 1975)]. I did not plan then to have another related review 30 years later—but things happen. The 1976 work “attempted to trace the evolution of thinking on relevance, a key notion in information science, [and] to provide a framework within which the widely dissonant ideas on relevance might be interpreted and related to one another” (ibid.: 338).

The present study aims to review the literature concerning Digital Libraries (DLs) and user interfaces in order to identify, determine, and suggest evaluation criteria for a DLs user interface. Accordingly, this study's objectives are threefold: explore which criteria exert a significant relationship with the DLs user interface; identify a set of criteria that appears to be useful for evaluating DLs user interface; and determine evaluation criteria that have more frequency and occurrence in the related texts reviewed.

To do it, first, identifying related texts was necessary. Consequently, keywords such as “DLs user interface evaluation”,” DLs user interfaces”, “DLs evaluation”, “DLs usability”, “user interface evaluation”, “DLs research”, “web sites user interface evaluation”, “user interface standards”, and the like have been searched in the web as well as in some leading databases including Emerald, Proquest, SagePub, ScienceDirect, LISA, ERIC, ACM, and Springer. After identifying and accessing more than 100 evaluative works and some related articles, theoretical and empirical, nearly 50 sources were chosen for final examination.

After reviewing related texts, three major categories are identified: user interface and DLs; DLs and usability; and other studies related to user interface; each one of three identified categories has its own subcategories. Additionally, 22 evaluation criteria for assessing DLs interface have been identified.

The review does not claim to be comprehensive.

Hopefully, criteria such as feedback, ease of use, match between system and the real world, customization, user support, user workload, interaction, compatibility, visibility of system status, user experience, flexibility, and accessibility which have been less considered should be applied more in future, particularly user‐oriented, studies. Furthermore, it is expected that criteria mentioned here could help related bodies pay more attention to the evaluation of EISs, especially DLs interface.

It can be said that this study has contributed to the research into the evaluation of DL interface.

The purpose of this paper is to report on a user-centred analysis of video digital libraries and to present users’ criteria based on concepts emerging directly from the responses of study participants. As a time-based and multi-channeled format, video warrants considerations for information delivery through interactive tools such as digital libraries.

A survey method was used to collect open-ended responses from participants. Content analysis was performed on the responses, and categories then emerged to form the coding scheme where simple and weighted frequencies were calculated to obtain a ranked set of users’ criteria. Cohen’s κ was 0.87, indicative of high-level of inter-coder reliability. In the end, 81 participants contributed a total of 385 open-ended responses, which were all then analysed and coded.

The emergent coding method and continuous refinements to the coding scheme ultimately produced 28 criteria (subcategories) under four primary categories. Criteria corresponding to “retrieval functionality” of video digital libraries emerged as the highest, or most frequently referenced, primary category, while the “user interface”, “collection qualities”, and “user support” followed, respectively.

The primary research contribution is a baseline for video digital libraries that can be applicable throughout design and evaluation. Future studies can utilise findings presented here as guidance for the initial design stages and/or for assessing systems via targeted evaluations.

While users’ criteria of digital libraries have been analysed from non-visual contexts, they have not been directly assessed as they pertain to users’ experiences and perceptions with video, which is needed as the nature and structure of video can influence users and use.

Going beyond the usual approach of measuring port performance – focusing on the efficiency of port operations – this paper aims to look into shipping lines and other port users’ perceptions on port performance.

The paper develops a framework measuring the perceptions of port users (i.e. shipping companies, shippers, etc.) on port performance. A typology of elements that shape port users’ perceptions has been developed with an eye on capturing the peculiarities of different port markets. Based on this typology, a tool to assess users’ perspectives, and subsequently evaluate, any given port has been developed. The tool provides port authorities the flexibility they need for customized approaches. The developed evaluation mechanism has been tested on a group of European seaports, and the results are presented by this study.

The framework and its pilot application unveil the key parameters that port users take into consideration when evaluating the effectiveness component of port performance. Moreover, the importance and evaluation ratings of specific performance parameters allow for a GAP analysis of the collected data.

The paper advances scholarly and practical discussion on how of port users’ perceptions can be a valuable tool for port performance measurement.

The proposed tool can be a valuable add-on for port authorities to evaluate their performance from the port users’ point of view and take the necessary actions to improve it. Also, the tool can be used for the evaluation of a new process, infrastructure. The evaluation of port users’ perception of port performance can and must be part of a European ports observatory, as it is a set of indicators that clearly reflects the satisfaction of port users by engaging their view on port performance issues, instead of relying almost exclusively on port-generated data.

The paper develops a framework for measuring port user’s perception on port performance, which is flexible and can be applied in any port.

One of the most effective ways information retrieval (IR) systems including Web search engines can improve relevance performance is to provide their users with tools for facilitating query expansion. Search engines such as Google provide users with keyword suggest tools. This paper aims to investigate users’ criteria in relevance judgment regarding Google’s keywords suggest tool and to see how such keywords would lead to more relevant results from the viewpoint of users.

Through a mixed method approach, quantitative and qualitative data were collected from 60 postgraduate students at Ferdowsi University of Mashhad, Iran, using four different instruments (questionnaire, thinking aloud technique, query logs and interviews).

Among other criteria, the “relation between suggested keywords and the information need” (with the mean rate of 3.53 of four) was considered the most important by searchers in selecting suggested keywords for query expansion. Also, the “relation between suggested Keywords and the retrieved items” (with the mean rate of 3.62) was considered the second most important criterion in judging the relevance of the retrieved results. The participants agreed that the suggested keywords by Google improved the retrieval relevance. The content analysis of the participants’ aloud-thinking sessions and the interviews approved such findings.

This research makes a contribution to the need of designers of IR systems regarding the use of add words for query expansion. It also helps librarians how to instruct searchers with expanding their queries to retrieve more relevant results. Another contribution of the study is the identification of a number of new relevance judgment criteria for Web-based environments.

Multi-criteria recommender systems (MC-RSs) allow users to express their preference in multiple aspects. Bayesian flexible mixture model (BFMM) is a model-based RS which extends FMM from single-criterion to MC. However, results of BFMM have a preference on different rating pattern problem. In single-criterion, FMM with decoupled normalization and W’s transposed function try to solve this problem. However, these techniques are applied to each criterion separately. Then, the relationship among criteria will be lost. This paper aims to solve different rating pattern problems and loss of the relationship between criteria.

The proposed method is combining between BFMM and rating conversion. First, mean and variance normalization is applied to make MC ratings of an active user and a neighbor lying on the same plane. After that, a pattern of each user is extracted using principal component analysis (PCA). Next, the pattern is used to convert neighbors’ MC ratings to the active user aspect. After that, converted MC ratings of neighbors are aggregated to be overall ratings using multiple linear regression (MLR). Finally, overall rating of the active user toward the target item is predicted using weighted average on the derived neighbors’ overall ratings where the similarity from BFMM acts as a weight.

The experimental results show that the proposed method where all criteria ratings are converted simultaneously can improve the performance of recommendation.

The proposed method predicts overall rating of the active user by converting MC ratings of each neighbor to the active user aspect at the same time, which can reduce the loss of the relationship between criteria.

The identification of key criteria that influence citizens’ preferences for using m-government services, prioritization of these criteria and making them consistent with the development of m-government services can improve the relationship of the government with citizens and vice versa. The purpose of this study is to investigate and prioritize the criteria that influence the usage of m-government services in the United Arab Emirates (UAE), using the technology acceptance model (TAM) and diffusion of innovation (DOI) theory, along with external variables of perceived security and perceived privacy.

A quantitative approach has been applied using a questionnaire with a nine-point scale for collecting data. The analytic hierarchy process (AHP) approach has been deployed to structure and prioritize citizens’ preferences and select users of the Ministry of Interior (MOI) application.

The results of the data analysis showed that the first criteria, which refer to the intention to use m-government services, were ease of use and perceived security and were given the highest priority, and the criterion of observability was given the lowest priority.

This study integrates the DOI and TAM theories with other external variables such as perceived security and perceived privacy to develop a conceptual framework. AHP has been used to structure and prioritize the main criteria and sub-criteria of m-government services. The results can assist the m-government decision-makers and software developers in focusing on the criteria that should be considered for the development of m-government services.

The purpose of this study is to compare the evaluation of search result lists and documents, in particular evaluation criteria, elements, association between criteria and elements, pre/post and evaluation activities, and the time spent on evaluation.

The study analyzed the data collected from 31 general users through prequestionnaires, think aloud protocols and logs, and post questionnaires. Types of evaluation criteria, elements, associations between criteria and elements, evaluation activities and their associated pre/post activities, and time were analyzed based on open coding.

The study identifies the similarities and differences of list and document evaluation by analyzing 21 evaluation criteria applied, 13 evaluation elements examined, pre/post and evaluation activities performed and time spent. In addition, the authors also explored the time spent in evaluating lists and documents for different types of tasks.

This study helps researchers understand the nature of list and document evaluation. Additionally, this study connects elements that participants examined to criteria they applied, and further reveals problems associated with the lack of integration between list and document evaluation. The findings of this study suggest more elements, especially at list level, be available to support users applying their evaluation criteria. Integration of list and document evaluation and integration of pre, evaluation and post evaluation activities for the interface design is the absolute solution for effective evaluation.

This study fills a gap in current research in relation to the comparison of list and document evaluation.

The purpose of this paper is to examine changes in relevance assessments, specifically the selection of relevance criteria by subjects as they move through the information search process.

The paper examines the relevance criteria choices of 39 subjects in relation to search stage. Subjects were assigned a specific search task in a controlled test. Statistics were collected and analyzed using descriptive statistics and the chi‐square goodness‐of‐fit tests.

The statistically significant findings identified a number of commonly reported relevance criteria, which varied over an information search process for relevant and partially relevant judgments. These results provide statistical confirmations of previous studies, and extend these findings identifying specific criteria for both relevant and partially relevant judgments.

The study only examines a short duration search process and since the convenience sample of subjects were from similar backgrounds and were assigned similar tasks, the study did not explicitly examine the impact of contextual factors such as user experience, background or task in relation to relevance criteria choices.

The paper has implications for the development of search systems which are adaptive and recognize the cognitive changes which occur during the information search process. Examining and identifying relevance criteria beyond topicality and the importance of those criteria to a user can help in the generation of better search queries.

The paper adds more rigorous statistical analysis to the study of relevance criteria and the information search process.

Libraries constitute a highly developing area as they always enrich their offered services in order to satisfy users' expectations. One of the most important factors in customization strategies and improvement of individual services must be user preferences. However, customization requires an in‐depth analysis of user preferences and an evaluation of future behavior. The main objective of this paper is to present a framework for analyzing changes of user preferences in a library.

The paper presents detailed results of two independent user satisfaction surveys conducted in an interval of five years in the Library of the Technical University of Crete and analyzes them using a proposed framework. However, the framework can be applied in every modern library. The analyses are based on non‐parametric statistical techniques and a multicriteria satisfaction analysis method, which is a multicriteria preference disaggregation approach.

Overall, user judgments for both surveys show that 50‐70 percent of users are “very satisfied” or “satisfied”. The percentage of “satisfied” users increased from 2005 to 2010 (almost 18 percent) and “very satisfied” remained almost the same.

Results are mainly focused on the evaluation of potential trends of user preferences. Furthermore, results of a benchmarking analysis are also presented, based on the evolution of satisfaction levels for the quality characteristics of the offered services.

The goal of the presented study and proposed framework and methodology is to help library decision makers track changes to user preferences and improve the provided services according to those preferences.