To read the full version of this content please select one of the options below:

A statistical interpretation of term specificity and its application in retrieval

Karen Spärck Jones (Computer Laboratory, University of Cambridge, Cambridge, UK)

Journal of Documentation

ISSN: 0022-0418

Article publication date: 1 October 2004

Abstract

The exhaustivity of document descriptions and the specificity of index terms are usually regarded as independent. It is suggested that specificity should be interpreted statistically, as a function of term use rather than of term meaning. The effects on retrieval of variations in term specificity are examined, experiments with three test collections showing, in particular, that frequently‐occurring terms are required for good overall performance. It is argued that terms should be weighted according to collection frequency, so that matches on less frequent, more specific, terms are of greater value than matches on frequent terms. Results for the test collections show that considerable improvements in performance are obtained with this very simple procedure.

Keywords

Citation

Spärck Jones, K. (2004), "A statistical interpretation of term specificity and its application in retrieval", Journal of Documentation, Vol. 60 No. 5, pp. 493-502. https://doi.org/10.1108/00220410410560573

Publisher

:

Emerald Group Publishing Limited

Copyright © 2004, Emerald Group Publishing Limited