Search results

This paper provides a foundation for a practical way of improving the effectiveness of an automatic retrieval system. Its main concern is with the weighting of index terms as a device for increasing retrieval effectiveness. Previously index terms have been assumed to be independent for the good reason that then a very simple weighting scheme can be used. In reality index terms are most unlikely to be independent. This paper explores one way of removing the independence assumption. Instead the extent of the dependence between index terms is measured and used to construct a non‐linear weighting function. In a practical situation the values of some of the parameters of such a function must be estimated from small samples of documents. So a number of estimation rules are discussed and one in particular is recommended. Finally the feasibility of the computations required for a non‐linear weighting scheme is examined.

My task in this second introductory paper is to attempt a review of the research that has been undertaken in relation to the retrieval of information. It is some twenty years ago that the term ‘information retrieval’ was first used, reputedly by Calvin Mooers, although it was several years later before it became a generally accepted term. It would be absurd to suggest that there was no such thing as information retrieval before the term was coined, but it is useful to consider it as marking a new approach to a problem that had always existed, but which had become very much more serious in consequence of the great increase in scientific and technical literature during and after World War II.

A study has been made of the effect of controlled variations in indexing vocabulary size on retrieval performance using the Cranfield 200 and 1400 test collections. The vocabularies considered are sets of variable‐length character strings chosen from the fronts of document and query terms so as to occur with approximate equifrequency. Sets containing between 120 and 720 members were tested both using an application of the Cluster Hypothesis and in a series of linear associative retrieval experiments. The effectiveness of the smaller sets is low but the larger ones exhibit retrieval characteristics comparable to those of words.

Thirty years or more ago, a favoured question in examination papers for librarians was some variation on the theme ‘Compare the merits of a classified catalogue and an alphabetical subject catalogue’. This was a subject which it was possible to write on or to discuss at great length, advancing a number of theoretical arguments or opinions expressed by pundits, without ever stating a single demonstrable fact. When, in the early 1950's, various people, such as Taube and Mooers, proposed new techniques for indexing, the reaction from the traditionally‐minded was such that it appeared there was to be a repetition of all the old arguments. In an editorial in American Documentation in 1955, Perry expressed a viewpoint, shared by many others, when he wrote:

A model is proposed to explain the retrieval characteristics of an IR system. It is assumed that, for a given system and a given question, there are probabilities of retrieving relevant or non‐relevant documents, but that these probabilities are not necessarily the same for different questions. A Bayesian method is outlined for estimating these probabilities, on the basis of a model relating them. The method is applied successfully to some Cranfield data. Potentialities of the method are discussed.

In selecting the contributions to this special issue, the editors have tried to plot a course that describes the state of the AI field for both the reader unacquainted with AI and for those who are. We have done this by selecting key research papers in the areas of AI that are impacting and will continue to impact libraries and by including sidebars that give context to the research papers. The language of the research papers is not simplified; AI is not a simple field. But neither is it incomprehensible, so if parts of this special issue are found to be difficult, the reader is advised to concentrate on the overall ideas rather than their specific expression.

The purpose of this paper is to clarify the ontological and epistemological basis of classification.

Attention is drawn to a 1785 article on abstraction by Thomas Reid and the contents and theories of the article are explained. The Reid article both provides a sound approach to classification and is interesting historically as it influenced the classification pioneer Charles Ammi Cutter who, in turn, is responsible for much of the modern theory of functional bibliography. Reid's account is supplemented by brief descriptions of fallibilism and fuzziness. An associated view, Aristotelian essentialism is explained and criticized. Some observations are offered on the role of prototypes in classification and on the monothetic‐polythetic distinction.

Reid's theories, suitably embedded in fallibilism and augmented with a respect for truth, provide a sound ontological and epistemological basis for classification.

Reid's essay, together with an appreciation of fallibility and determinate and indeterminate properties, amount to a good basic theoretical foundation for cataloging.

The composition of the Editorial Board has recently undergone a fairly fundamental change as a consequence of the resignation, for a variety of reasons, of four of its members. Mr B. C. Brookes, a long‐serving member of the Board, and until recently its Chairman, Mr J. W. Jolliffe, Mr W. R. Maidment, and Dr R. C. Young all leave us at a time when the Journal enjoys a healthy circulation which can only be a reflection of the esteem in which the readership holds the publication and is sufficient tribute in itself to the valuable contribution which these members have made. We are all very grateful for their help.