To read this content please select one of the options below:

A multi-objective approach to determining the usefulness of papers in academic search

Shah Khalid (School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, China) (School of Computer Science, National University of Sciences and Technology, Islamabad, Pakistan)
Shengli Wu (School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, China)
Fang Zhang (School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, China)

Data Technologies and Applications

ISSN: 2514-9288

Article publication date: 18 May 2021

Issue publication date: 11 October 2021

289

Abstract

Purpose

How to provide the most useful papers for searchers is a key issue for academic search engines. A lot of research has been carried out to address this problem. However, when evaluating the effectiveness of an academic search engine, most of the previous investigations assume that the only concern of the user is the relevancy of the paper to the query. The authors believe that the usefulness of a paper is determined not only by its relevance to the query but also by other aspects including its publication age and impact in the research community. This is vital, especially when a large number of papers are relevant to the query.

Design/methodology/approach

This paper proposes a group of metrics to measure the usefulness of a ranked list of papers. When defining these metrics, three factors, including relevance, publication age and impact, are considered at the same time. To accommodate this, the authors propose a framework to rank papers by a combination of their relevance, publication age and impact scores.

Findings

The framework is evaluated with the ACL (Association for Computational Linguistics Anthology Network) dataset. It demonstrates that the proposed ranking algorithm is effective for improving usefulness when two or three aspects of academic papers are considered at the same time, while the relevance of the retrieved papers is slightly down compared with the relevance-only retrieval.

Originality/value

To the best of the authors’ knowledge, the proposed multi-objective academic search framework is the first of its kind that is proposed and evaluated with a group of new evaluation metrics.

Keywords

Citation

Khalid, S., Wu, S. and Zhang, F. (2021), "A multi-objective approach to determining the usefulness of papers in academic search", Data Technologies and Applications, Vol. 55 No. 5, pp. 734-748. https://doi.org/10.1108/DTA-05-2020-0104

Publisher

:

Emerald Publishing Limited

Copyright © 2021, Emerald Publishing Limited

Related articles