To read this content please select one of the options below:

Comparing tagging suggestion models on discrete corpora

Bojan Bozic (Department of Computer Science, Technological University Dublin, Dublin, Ireland)
Andre Rios (Department of Computer Science, Technological University Dublin, Dublin, Ireland)
Sarah Jane Delany (Department of Computer Science, Technological University Dublin, Dublin, Ireland)

International Journal of Web Information Systems

ISSN: 1744-0084

Article publication date: 11 May 2020

Issue publication date: 3 June 2020

93

Abstract

Purpose

This paper aims to investigate the methods for the prediction of tags on a textual corpus that describes diverse data sets based on short messages; as an example, the authors demonstrate the usage of methods based on hotel staff inputs in a ticketing system as well as the publicly available StackOverflow corpus. The aim is to improve the tagging process and find the most suitable method for suggesting tags for a new text entry.

Design/methodology/approach

The paper consists of two parts: exploration of existing sample data, which includes statistical analysis and visualisation of the data to provide an overview, and evaluation of tag prediction approaches. The authors have included different approaches from different research fields to cover a broad spectrum of possible solutions. As a result, the authors have tested a machine learning model for multi-label classification (using gradient boosting), a statistical approach (using frequency heuristics) and three similarity-based classification approaches (nearest centroid, k-nearest neighbours (k-NN) and naive Bayes). The experiment that compares the approaches uses recall to measure the quality of results. Finally, the authors provide a recommendation of the modelling approach that produces the best accuracy in terms of tag prediction on the sample data.

Findings

The authors have calculated the performance of each method against the test data set by measuring recall. The authors show recall for each method with different features (except for frequency heuristics, which does not provide the option to add additional features) for the dmbook pro and StackOverflow data sets. k-NN clearly provides the best recall. As k-NN turned out to provide the best results, the authors have performed further experiments with values of k from 1–10. This helped us to observe the impact of the number of neighbours used on the performance and to identify the best value for k.

Originality/value

The value and originality of the paper are given by extensive experiments with several methods from different domains. The authors have used probabilistic methods, such as naive Bayes, statistical methods, such as frequency heuristics, and similarity approaches, such as k-NN. Furthermore, the authors have produced results on an industrial-scale data set that has been provided by a company and used directly in their project, as well as a community-based data set with a large amount of data and dimensionality. The study results can be used to select a model based on diverse corpora for a specific use case, taking into account advantages and disadvantages when applying the model to your data.

Keywords

Acknowledgements

This paper is an extended version of the following conference paper: Bojan Božić, André Ríos and Sarah Jane Delany (2018), Validation of Tagging Suggestion Models for a Hotel Ticketing Corpus. In Proceedings of the 20th International Conference on Information Integration and Web-based Applications and Services (iiWAS2018), Maria Indrawan-Santiago, Eric Pardede, Ivan Luiz Salvadori, Matthias Steinbauer, Ismail Khalil and Gabriele Anderst-Kotsis (Eds.). ACM, New York, NY, USA, 15–23. DOI: https://doi.org/10.1145/3282373.3282386. Furthermore, the authors send their thanks to DmBook Pro [9] who provided a manually tagged data set, which our research was based on.

Citation

Bozic, B., Rios, A. and Delany, S.J. (2020), "Comparing tagging suggestion models on discrete corpora", International Journal of Web Information Systems, Vol. 16 No. 2, pp. 201-221. https://doi.org/10.1108/IJWIS-08-2019-0035

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Emerald Publishing Limited

Related articles