A WEIGHTED RANK CORRELATION COEFFICIENT FOR THE COMPARISON OF RELEVANCE JUDGEMENTS

Abstract

The application of conventional rank correlation tests to the comparison of relevance judgements is outlined and their disadvantages are noted. A weighted rank correlation coefficient rw is proposed, which assigns variable weights to exchanges of rank on different parts of the scale. Examples of its computation are given, both for tied and untied ranks, and the values of rw for n=3 and n=4 are tabulated. The chief mathematical properties of the coefficient are investigated.

Citation

KAYE, D. (1973), "A WEIGHTED RANK CORRELATION COEFFICIENT FOR THE COMPARISON OF RELEVANCE JUDGEMENTS", Journal of Documentation, Vol. 29 No. 4, pp. 380-389. https://doi.org/10.1108/eb026564