Guest editorial: Extraction and evaluation of knowledge entities in the age of artificial intelligence

Citation

Zhang, C., Mayr, P., Lu, W. and Zhang, Y. (2023), "Guest editorial: Extraction and evaluation of knowledge entities in the age of artificial intelligence", Aslib Journal of Information Management, Vol. 75 No. 3, pp. 433-437. https://doi.org/10.1108/AJIM-05-2023-507

1. Introduction

Scientific documents serve as an essential mediator for research achievements and scientific knowledge. With the increasing availability of full-text data and advanced data analytical techniques, bibliometric researchers have started to focus on granular content within scientific documents, transitioning their foci from the external metadata of these documents to the internal knowledge they encompass. In scientific documents, knowledge consists of many interconnected units, known as knowledge entities (Ding et al., 2013).

Ding et al. (2013) classified knowledge entities into macro-level (e.g. author, journal, references), meso-level (e.g. keywords) and micro-level (e.g. dataset, method, biomedical entities) categories. These categories refer to traditional bibliographic entities for evaluation purposes and knowledge entities found within the full-text content, respectively. The extraction and evaluation of knowledge entities can improve existing knowledge services and meet the needs of researchers for rapid and accurate access to scientific knowledge (Ma et al., 2023; Mayr et al., 2014). Scholars can evaluate the scientific, social and economic attributes of knowledge and assess its role in science, technology, innovation and policy. Analyzing dynamic changes over time and identifying geographical differences can provide insights to understand the patterns of knowledge use and dissemination, which can promote knowledge discovery and generate new scientific activities.

The 2nd Workshop on the Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2021) was held online, co-located with the ACM/IEEE Joint Conference on Digital Libraries 2021 (JCDL2021) on September 30, 2021. The EEKE workshop series (https://eekeworkshop.github.io/) aims to engage relevant communities in addressing open problems in the extraction and evaluation of knowledge entities from scientific documents. Participants have contributed tremendous accomplishments in identifying knowledge entities, exploring entity features, analyzing entity relationships, and developing extraction platforms or knowledge bases. The workshop has provided scholars, especially early career researchers, with knowledge recommendations and other knowledge entity-based services (Zhang et al., 2021).

2. Topics in this special issue

This special issue collected articles presented at the EEKE2021 workshop and external submissions. A total of 18 papers were submitted, of which 9 were accepted after a rigorous review process. These 9 articles, contributed by 36 authors from 4 countries (China, USA, South Korea and Thailand), are included in this collection. In this editorial, we provide an overview of these papers and introduce the themes and contributions of this Special Issue. The papers are grouped into three topics: entity extraction and entity relations extraction (3 articles); annotation tools and knowledge entity graph construction (2 articles) and applications of knowledge entities (4 articles).

2.1 Entity extraction and entity relations extraction

Knowledge entities such as concepts, algorithms and methods are included in scientific literature (Ding et al., 2013; Wang et al., 2022). Extracting knowledge entities and entity relationships from scientific literature plays a crucial role in scientific information retrieval, fine-grained scientific information recommendation and scientific evaluation.

To extract “band gap information” from academic papers, Ghosh and Lu (2023) collected 1.44 million titles and abstracts of scholarly articles related to materials science. They filtered the collection to 11,939 articles that may contain core information on materials and their band gap values. The results revealed that the current system can accurately extract information from 51.32% of the articles, partially extract from 36.62% of the articles and incorrectly extract from 12.04% of the articles.

To tackle the issues of insufficient training corpus and the quality control of annotations, Yan et al. (2023) proposed a novel integrated solution for Chinese historical Named Entity Recognition (NER), including automatic entity extraction and man-machine cooperative annotation. This solution is valuable for enhancing the effectiveness of Chinese historical NER and promoting the development of low-resource information extraction.

“Problem-solving” stands out as one of the most fundamental and critical insights of scientific research. Using computer vision as an example, Chen et al. (2023) built a “problem-solving” knowledge graph of scientific domains by extracting four entity relation types, namely problem-solving, problem hierarchy, solution hierarchy and association. They illustrated the utility of the extracted relations in constructing domain knowledge graphs and uncovering historical research trends.

2.2 Annotation tools and the construction of knowledge entity graphs

Knowledge entity annotation and knowledge graph construction are essential tasks for knowledge entity applications, including the development of annotation tools, corpora construction and the creation of knowledge graphs.

To explore the use of entity descriptions and network structures in enhancing knowledge graph completion with a high generalization ability across datasets, Yu et al. (2023) proposed an entity-description augmented knowledge graph completion model (EDA-KGC). The authors conducted extensive experiments on the FB15K, WN18, FB15K-237 and WN18RR datasets to validate the effectiveness of the model.

To address challenges posed by growing volumes of pre-annotated literature and diverse annotations, such as teamwork, quality control and time management, Wang et al. (2023) developed the Bureau for Rapid Annotation Tool (Brat), an annotation collaboration workbench that includes an enhanced semantic constraint system, Vim-like shortcut keys, an annotation filter and a graph-visualizing annotation browser.

2.3 Applications of knowledge entities

Knowledge entities could seed not only granular information retrieval and recommendation (Mayr et al., 2014), intelligent bibliometrics (Zhang et al., 2020), but also new cognitive and practical applications such as the innovative evaluation of scientific document (Liu et al., 2022) and predicting future research directions (Zhang et al., 2023a, 2023b, 2023c).

To explore mental health information entities and the connections between the biomedical, psychological and social domains of bipolar disorder (BD), Timakum et al. (2023) used Reddit posts and full-text papers from PubMed Central to extract BD entities and their relationships in the datasets using a dictionary-based and rule-based approach. The findings indicate that the drug side effects entity was frequently identified in both datasets as a mental health information entity.

Zhang et al. (2023a, 2023b, 2023c) introduced a crucial factor in topic selection, i.e. topic popularity, to investigate its relationship with team performance. The authors used gene/protein entities as proxies for topics and extracted them to monitor the development of topic popularity. By comparing various dimensions of team performance, the study explored the relationship between the phase of selected topic popularity and the academic performance of research teams.

Zeng et al. (2023) utilized the knowledge elements extracted through the Lexicon-LSTM model to measure the interdisciplinary characteristics of Chinese research in library and information science (LIS). They constructed a subject knowledge graph to support the searching and classification of knowledge elements. The results showed that in LIS, the interdisciplinary diversity indicator exhibited an upward trend from 2011 to 2021, while the disciplinary balance and difference indicators showed a downward trend.

Focusing on discovery of topic evolution path and semantic relationship, Zhang et al. (2023a, 2023b, 2023c) identified entities that have the same semantics but different expressions for accurate topic evolution path discovery. They also revealed semantic relationships of topic evolution to better understand what leads to topic evolution. This work provided a new perspective for topic evolution analysis by considering the semantic representation of patent entities.

3. Conclusions

The EEKE workshop series and this Special Issue reinforced the increasing interest of the community in the extraction and evaluation of entities from scientific literature. Endeavors in this field are advancing rapidly, from developing new methods and techniques to their innovative applications in broad practical domains. In today's AI age, three key factors that are driving AI development – namely, data, algorithms and computing power – must work corporately to promote and support one another to achieve success and create value.

Currently, Large-scale pre-trained Language Models (LLMs), such as ChatGPT and GPT-4, have gained widespread popularity due to their broad applications across various industries and fields. There is no doubt that LLMs play an exceptionally important role in scientific documents mining and analysis. Some scholars have tested and evaluated the performance of ChatGPT and GPT-4 in entity extraction (Hu et al., 2023; González-Gallardo et al., 2023) and entity relationship recognition (Rehana et al., 2023) in special domains. With AI's continued advancements and the increasing promotion of open access movements, we can anticipate a more extensive utilization of LLMs, such as Generative Pretrained Transformer (GPT), for extraction and evaluation of knowledge entities from scientific documents in the future.