To read the full version of this content please select one of the options below:

Extracting core questions in community question answering based on particle swarm optimization

Ming Li (China University of Petroleum-Beijing, Beijing, China)
Lisheng Chen (China University of Petroleum-Beijing, Beijing, China)
Yingcheng Xu (China National Institute of Standardization, Beijing, China)

Data Technologies and Applications

ISSN: 2514-9288

Article publication date: 22 October 2019

Issue publication date: 22 October 2019

Abstract

Purpose

A large number of questions are posted on community question answering (CQA) websites every day. Providing a set of core questions will ease the question overload problem. These core questions should cover the main content of the original question set. There should be low redundancy within the core questions and a consistent distribution with the original question set. The paper aims to discuss these issues.

Design/methodology/approach

In the paper, a method named QueExt method for extracting core questions is proposed. First, questions are modeled using a biterm topic model. Then, these questions are clustered based on particle swarm optimization (PSO). With the clustering results, the number of core questions to be extracted from each cluster can be determined. Afterwards, the multi-objective PSO algorithm is proposed to extract the core questions. Both PSO algorithms are integrated with operators in genetic algorithms to avoid the local optimum.

Findings

Extensive experiments on real data collected from the famous CQA website Zhihu have been conducted and the experimental results demonstrate the superior performance over other benchmark methods.

Research limitations/implications

The proposed method provides new insight into and enriches research on information overload in CQA. It performs better than other methods in extracting core short text documents, and thus provides a better way to extract core data. The PSO is a novel method used for selecting core questions. The research on the application of the PSO model is expanded. The study also contributes to research on PSO-based clustering. With the integration of K-means++, the key parameter number of clusters is optimized.

Originality/value

The novel core question extraction method in CQA is proposed, which provides a novel and efficient way to alleviate the question overload. The PSO model is extended and novelty used in selecting core questions. The PSO model is integrated with K-means++ method to optimize the number of clusters, which is just the key parameter in text clustering based on PSO. It provides a new way to cluster texts.

Keywords

Acknowledgements

The authors declare no conflict of interest. The research is supported by the National Natural Science Foundation of China (Grant No. 71571191), the Humanity and Social Science Youth Foundation of the Ministry of Education in China (Grant No. 15YJCZH081) and National Natural Science Foundation of China (Grant No. 91646122).

Citation

Li, M., Chen, L. and Xu, Y. (2019), "Extracting core questions in community question answering based on particle swarm optimization", Data Technologies and Applications, Vol. 53 No. 4, pp. 456-483. https://doi.org/10.1108/DTA-02-2019-0025

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Emerald Publishing Limited