Search results

The purpose of this study is to compare the evaluation of search result lists and documents, in particular evaluation criteria, elements, association between criteria and elements, pre/post and evaluation activities, and the time spent on evaluation.

The study analyzed the data collected from 31 general users through prequestionnaires, think aloud protocols and logs, and post questionnaires. Types of evaluation criteria, elements, associations between criteria and elements, evaluation activities and their associated pre/post activities, and time were analyzed based on open coding.

The study identifies the similarities and differences of list and document evaluation by analyzing 21 evaluation criteria applied, 13 evaluation elements examined, pre/post and evaluation activities performed and time spent. In addition, the authors also explored the time spent in evaluating lists and documents for different types of tasks.

This study helps researchers understand the nature of list and document evaluation. Additionally, this study connects elements that participants examined to criteria they applied, and further reveals problems associated with the lack of integration between list and document evaluation. The findings of this study suggest more elements, especially at list level, be available to support users applying their evaluation criteria. Integration of list and document evaluation and integration of pre, evaluation and post evaluation activities for the interface design is the absolute solution for effective evaluation.

This study fills a gap in current research in relation to the comparison of list and document evaluation.

The purpose of this paper is to propose a method to have more accurate results in comparing performance of the paired information retrieval (IR) systems with reference to the current method, which is based on the mean effectiveness scores of the systems across a set of identified topics/queries.

Based on the proposed approach, instead of the classic method of using a set of topic scores, the documents level scores are considered as the evaluation unit. These document scores are the defined document’s weight, which play the role of the mean average precision (MAP) score of the systems as a significance test’s statics. The experiments were conducted using the TREC 9 Web track collection.

The p-values generated through the two types of significance tests, namely the Student’s t-test and Mann-Whitney show that by using the document level scores as an evaluation unit, the difference between IR systems is more significant compared with utilizing topic scores.

Utilizing a suitable test collection is a primary prerequisite for IR systems comparative evaluation. However, in addition to reusable test collections, having an accurate statistical testing is a necessity for these evaluations. The findings of this study will assist IR researchers to evaluate their retrieval systems and algorithms more accurately.

A good recommender system helps users find items of interest on the web and can provide recommendations based on user preferences. In contrast to automatic technology‐generated recommender systems, this paper aims to use dynamic expert groups that are automatically formed to recommend domain‐specific documents for general users. In addition, it aims to test several effectiveness measures of rank order to determine if the top‐ranked lists recommended by the experts were reliable.

In the approach, expert groups evaluate web documents to provide a recommender system for general users. The authority and make‐up of the expert group are adjusted through user feedback. The system also uses various measures to gauge the difference between the opinions of experts and those of general users to improve the evaluation effectiveness.

The proposed system is efficient when there is major support from experts and general users. The recommender system is especially effective where there is a limited amount of evaluation data from general users.

This is an original study of how to effectively recommend web documents to users based on the opinions of human experts. Simulation results were provided to show the effectiveness of the dynamic expert group for recommender systems.

The purpose of this research is to study how compliance evaluation becomes performed in practice. Compliance evaluation is a common practice among organizations that need to evaluate their posture against a set of criteria (e.g. a standard, legislative framework and “best practices”). The results of these evaluations have significant importance for organizations, especially in the context of information security and continuity. The author argues that how these evaluations become performed is not merely a “social” activity but shaped by the materiality of the evaluation criteria

The authors adopt a sociomaterial practice-based view to study the compliance evaluation through in situ participant observations from compliance evaluation workshops to evaluate organizational compliance against a information security and business continuity criteria. The empirical material was analyzed to construct vignettes that serve to illustrate the practice of compliance evaluation.

The research analysis shows how the information security and business continuity criteria themselves partake in the compliance evaluations by operating through (ventriloqually) the evaluators on three strata: the material, the textual and the structural. The author also provides a conceptualization of a hybrid agency.

This research contributes to lack of studies on the organizational-level compliance. Further, the research is an original contribution to information security and business continuity management by focusing on the practices of compliance evaluation. Further, the research has theoretical novelty by adopting the ventriloqual agency as a hybrid agency to study the sociomateriality of a phenomenon.

Hypertext documents can and do make large amounts of information readily accessible to ordinary readers, and production of such documents is understandably becoming more widespread. However, good intentions on the part of authors do not automatically result in the production of cost effective and acceptable documents. Evaluation is consequently a key issue — knowledge and use of appropriate evaluative methods helps ensure hypertext documents are correctly designed and presented. The results are increased usage and efficiency, and decreased maintenance costs. Drawing on practical, academic and commercial experience, this paper addresses some of the ways in which a hypertext document may be effectively evaluated.

This study aims to extract knowledge of ancient Chinese scientific and technological documents bibliographic summaries (STDBS) and provide the knowledge graph (KG) comprehensively and systematically. By presenting the relationship among content, discipline, and author, this study focuses on providing services for knowledge discovery of ancient Chinese scientific and technological documents.

This study compiles ancient Chinese STDBS and designs a knowledge mining and graph visualization framework. The authors define the summaries' entities, attributes, and relationships for knowledge representation, use deep learning techniques such as BERT-BiLSTM-CRF models and rules for knowledge extraction, unify the representation of entities for knowledge fusion, and use Neo4j and other visualization techniques for KG construction and application. This study presents the generation, distribution, and evolution of ancient Chinese agricultural scientific and technological knowledge in visualization graphs.

The knowledge mining and graph visualization framework is feasible and effective. The BERT-BiLSTM-CRF model has domain adaptability and accuracy. The knowledge generation of ancient Chinese agricultural scientific and technological documents has distinctive time features. The knowledge distribution is uneven and concentrated, mainly concentrated on C1-Planting and cultivation, C2-Silkworm, and C3-Mulberry and water conservancy. The knowledge evolution is apparent, and differentiation and integration coexist.

This study is the first to visually present the knowledge connotation and association of ancient Chinese STDBS. It solves the problems of the lack of in-depth knowledge mining and connotation visualization of ancient Chinese STDBS.

For reasons of technical convenience, current retrieval algorithms based on probabilistic reasoning are derived from models that assume patrons evaluate documents using a two value relevance scale. This paper extends the theory by describing a model which includes a more general relevance scale. This model permits a re‐examination of the earlier theory as a special case of that developed here and leads to a more satisfying interpretation of the ranking principle of the earlier models.