Chapter 6 Diversity-Aware Search: New Possibilities and Challenges for Web Search

Abstract

Purpose — Since a couple of years, we are confronted with the phenomenon of information overload. In particular, the web provides a rich source of a variety of information mainly in textual, i.e. unstructured form. Thus, web search faces new challenges that are how to make the user aware of the variety of content available and how to satisfy users best with such manifold content.

Methodology — This variety of content is considered as diversity, i.e. the reflection of a result set's coverage of multiple interpretations of a query. Diversification within web search aims on the one hand at adapting the ranking in a way that the top results are diverse. Increasingly important becomes on the other hand the organization and classification of content within diversification.

Findings — Various approaches to diversification are available or currently focus on research activities. They range from an adapted ranking by means of similarity measures or diversity scores to a comprehensive diversity analysis which determines topics and classifies text according to opinions etc.

Implications — Given the high diversity of web content, approaches for diversification are extremely important. Web search tries to address this problem from different perspectives. For the future, combination with image search result diversification is important. Further, benchmarks and standard data sets for evaluations need to be established to ensure comparability of results from various approaches.

Originality/value — This chapter provides an overview on diversity in web search from two directions: (a) Diversity is introduced with its notions and dimensions. (b) Methods to assess diversity within web search are presented.

Citation

Denecke, K. (2012), "Chapter 6 Diversity-Aware Search: New Possibilities and Challenges for Web Search", Lewandowski, D. (Ed.) Web Search Engine Research (Library and Information Science, Vol. 4), Emerald Group Publishing Limited, Leeds, pp. 139-162. https://doi.org/10.1108/S1876-0562(2012)002012a008