The development of a method for the evaluation of interactive information retrieval systems

Pia Borlund (Department of IR Theory, The Royal School of Librarianship, Copenhagen, Denmark)
Peter Ingwersen (Department of IR Theory, The Royal School of Librarianship, Copenhagen, Denmark)

Journal of Documentation

ISSN: 0022-0418

Publication date: 1 August 1997

Abstract

The paper describes the ideas and assumptions underlying the development of a new method for the evaluation and testing of interactive information retrieval (IR) systems, and reports on the initial tests of the proposed method. The method is designed to collect different types of empirical data, i.e. cognitive data as well as traditional systems performance data. The method is based on the novel concept of a ‘simulated work task situation’ or scenario and the involvement of real end users. The method is also based on a mixture of simulated and real information needs, and involves a group of test persons as well as assessments made by individual panel members. The relevance assessments are made with reference to the concepts of topical as well as situational relevance. The method takes into account the dynamic nature of information needs which are assumed to develop over time for the same user, a variability which is presumed to be strongly connected to the processes of relevance assessment.

Keywords

Citation

Borlund, P. and Ingwersen, P. (1997), "The development of a method for the evaluation of interactive information retrieval systems", Journal of Documentation, Vol. 53 No. 3, pp. 225-250. https://doi.org/10.1108/EUM0000000007198

Download as .RIS

Publisher

:

MCB UP Ltd

Copyright © 1997, MCB UP Limited

Please note you might not have access to this content

You may be able to access this content by login via Shibboleth, Open Athens or with your Emerald account.
If you would like to contact us about accessing this content, click the button and fill out the form.
To rent this content from Deepdyve, please click the button.