In the catalogue ye go for men: evaluation criteria for information retrieval systems

Abstract

The contrast between the value placed on discriminatory power in discussions of indexing and classification and on the transformation of a query into a set of relevant records dominant in information retrieval research has not yet been fully explored. The value of delivering relevant records in response to a query has been assumed by information retrieval research paradigms otherwise differentiated (the cognitive and the physical). Subsidiary concepts and measures (relevance and precision and recall) have been increasingly subjected to critiques. The founding assumption of the value of delivering relevant records now needs to be questioned. An enhanced capacity for informed choice is advocated as an alternative principle for system evaluation and design. This broadly corresponds to: the exploratory capability discussed in recent information retrieval research; the value of discriminatory power in classification and indexing; Giambattista Vico‘s critique of the unproductivity of Aristotelian methods of categorisation as routes to new knowledge; and, most significantly, to ordinary discourse conceptions of the value of information retrieval systems. The criterion of enhanced choice has a liberating effect, restoring man as an artificer and enabling a continuing dialectic between theory and practice. Techniques developed in classic information retrieval research can be adapted to the new purpose. Finally, the substitution of the principle of enhanced choice exemplifies the development of a true science, in which previous paradigms are absorbed into new as special cases. Ay, in the catalogue ye go for men; As hounds, and greyhounds, mongrels, spaniels, curs, Shoughs, water‐rugs, and demi‐wolves, are clept All by the name of dogs: the valu’d file Distinguishes the swift, the slow, the subtle, The housekeeper, the hunter, every one According to the gift which bounteous Nature Hath in him clos’d; whereby he does receive Particular addition, from the bill That writes them all alike; Shakespeare. Macbeth. c.1606.

Citation

Warner, J. (2000), "In the catalogue ye go for men: evaluation criteria for information retrieval systems", Aslib Proceedings, Vol. 52 No. 2, pp. 76-82. https://doi.org/10.1108/EUM0000000007002