Search results

The purpose of this paper is to understand how users behave and evaluate how systems with users are essential for interactive information retrieval (IIR) research. User study methodology serves as a primary approach to answering IIR research questions. In addition to designing user study procedures, understanding the limitations of varying study designs and discussing solutions to the limitations is also critical for improving the methods and advancing the knowledge in IIR.

Given this unresolved gap, we apply the faceted framework developed by Liu and Shah (2019) in systematically reviewing 131 IIR user studies recently published (2016–2021) on multiple IR and information science venues.

Our study achieve three goals: (1) extracting and synthesizing the reported limitations on multiple aspects of user study (e.g. recruitment, tasks, study procedures, system interfaces, data analysis methods) under associated facets; (2) summarizing the reported solutions to the limitations; (3) clarifying the connections between types of limitations and types of solutions.

The bibliography of user studies can be used by students and junior researchers who are new to user-centered IR studies as references for study design. Our results can facilitate the reflection and improvement on IR research methodology and serve as a checklist for evaluating customized IIR user studies in varying problem spaces.

To our knowledge, this work is the first study that systematically reviews the study limitations and solutions reported by IIR researchers in articles and empirically examines their connections to different study components.

This study aims to identify user perception of different qualities of optical character recognition (OCR) in texts. The purpose of this paper is to study the effect of different quality OCR on users' subjective perception through an interactive information retrieval task with a collection of one digitized historical Finnish newspaper.

This study is based on the simulated work task model used in interactive information retrieval. Thirty-two users made searches to an article collection of Finnish newspaper Uusi Suometar 1869–1918 which consists of ca. 1.45 million autosegmented articles. The article search database had two versions of each article with different quality OCR. Each user performed six pre-formulated and six self-formulated short queries and evaluated subjectively the top 10 results using a graded relevance scale of 0–3. Users were not informed about the OCR quality differences of the otherwise identical articles.

The main result of the study is that improved OCR quality affects subjective user perception of historical newspaper articles positively: higher relevance scores are given to better-quality texts.

To the best of the authors’ knowledge, this simulated interactive work task experiment is the first one showing empirically that users' subjective relevance assessments are affected by a change in the quality of an optically read text.

The purpose of this paper is to report a study of how the test instrument of a simulated work task situation is used in empirical evaluations of interactive information retrieval (IIR) and reported in the research literature. In particular, the author is interested to learn whether the requirements of how to employ simulated work task situations are followed, and whether these requirements call for further highlighting and refinement.

In order to study how simulated work task situations are used, the research literature in question is identified. This is done partly via citation analysis by use of Web of Science®, and partly by systematic search of online repositories. On this basis, 67 individual publications were identified and they constitute the sample of analysis.

The analysis reveals a need for clarifications of how to use simulated work task situations in IIR evaluations. In particular, with respect to the design and creation of realistic simulated work task situations. There is a lack of tailoring of the simulated work task situations to the test participants. Likewise, the requirement to include the test participants’ personal information needs is neglected. Further, there is a need to add and emphasise a requirement to depict the used simulated work task situations when reporting the IIR studies.

Insight about the use of simulated work task situations has implications for test design of IIR studies and hence the knowledge base generated on the basis of such studies.

Simulated work task situations are widely used in IIR studies, and the present study is the first comprehensive study of the intended and unintended use of this test instrument since its introduction in the late 1990’s. The paper addresses the need to carefully design and tailor simulated work task situations to suit the test participants in order to obtain the intended authentic and realistic IIR under study.

One core element of interactive information retrieval (IIR) experiments is the assignment of search tasks. The purpose of this paper is to provide an analytical review of current practice in developing those search tasks to test, observe or control task complexity and difficulty.

Over 100 prior studies of IIR were examined in terms of how each defined task complexity and/or difficulty (or related concepts) and subsequently interpreted those concepts in the development of the assigned search tasks.

Search task complexity is found to include three dimensions: multiplicity of subtasks or steps, multiplicity of facets, and indeterminability. Search task difficulty is based on an interaction between the search task and the attributes of the searcher or the attributes of the search situation. The paper highlights the anomalies in our use of these two concepts, concluding with suggestions for future methodological research related to search task complexity and difficulty.

By analyzing and synthesizing current practices, this paper provides guidance for future experiments in IIR that involve these two constructs.

Keywords search is intuitive, simple to use and convenient. It is also the de facto input interface for textual and multimedia retrieval. However, individuals often perform poorly when faced with exploratory search tasks that are common during learning, resulting in poor quality searches. The purpose of this paper is to examine how adolescent learners search and select videos to support self-learning. The findings allow for the identification of design concepts of video retrieval interface and features that can facilitate better exploratory searches.

Participants were assigned two customized video search tasks. The think-aloud protocol is used to allow participants to verbalize their actions, thoughts and feeling. This approach offered rich insights to the participants’ cognitive processes and considerations when performing the search tasks.

This study identified five themes for exploratory video search behavior: selection of internet resources, query formulation/reformulation, selection of the video(s) for preview, getting acquainted with the video content, and making a decision for the search task. The analysis of these themes led to a number of design concepts, ranging from supporting exploration of topics to better interaction with metadata.

The findings can inform future development of dedicated video retrieval systems interfaces that seeks to facilitate effective exploratory searches by learners.

This study contributes by suggesting design concepts for video retrieval system developers to support exploratory video searches.

The purpose of this paper is to investigate information retrieval (IR) in the context of authentic work tasks (WTs), as compared to traditional experimental IR study designs.

The participants were 22 professionals working in municipal administration, university research and education, and commercial companies. The data comprise 286 WTs and 420 search tasks (STs). The data were collected in natural situations. It includes transaction logs, video recordings, interviews, observation, and daily questionnaires.

The analysis included the effects of WT type and complexity on the number of STs, queries, search keys and types of queries. The findings suggest that simple STs are enough to support most WTs. Complex WTs (vs more simple ones) and intellectual WTs (vs communication, support and editing WTs) include more STs than other WT categories.

Further research should address the problems related to controllability of field studies and enhance the use of realistic WT situations in test-based studies, as well.

The study is an attempt to bring traditional IR studies and realistic research settings closer to each other. Using authentic WTs when studying IR is still rare. The representativeness of the WT/ST types used in interactive IR experiments should be carefully addressed: in the work flow, people seldom consciously recognise separate “STs”. This means that STs may mainly be an academic construct even to the point that studying IR without a decent context does violence to the further understanding of the phenomenon.

The purpose of this paper is to examine the role of task and user’s topic familiarity in the evaluation of information patch (websites).

An experimental study was conducted in a computer laboratory to examine users’ information seeking and foraging behaviour. In total, 160 university students participated in the research. Two types of task instructions, specifically defined and non-specifically defined (general) task types were administered. Mixed methods approach involving both quantitative and qualitative thematic coding were adopted, from the data of the questionnaire surveys and post-experiment interviews.

In the context of task attributes, users who conducted information seeking task with specifically defined instructions, as compared to the non-specifically defined instructions, demonstrated stricter credibility evaluations. Evidence demonstrated the link between topical knowledge and credibility perception. Users with topical knowledge applied critical credibility assessments than users without topical knowledge. Furthermore, the evidential results supported that the level of difficulty and knowledge of the topic or subject matter associated with users’ credibility evaluations. Users who have lesser or no subject knowledge and who experienced difficulty in the information search tended to be less diagnostic in their appraisal of the information patch (website or webpages). Users equipped with topical knowledge and who encountered less difficulty in the search, exhibited higher expectation and evaluative criteria of the information patch.

The constraints of time in the lab experiment, carried out in the presence of and under the observation of the researcher, may affect users’ information seeking behaviour. It would be beneficial to consider users’ information search gratifications and motivations in studying information evaluations and foraging patterns. There is scope to investigate users’ proficiency such as expert or novice, and individual learning styles in assessing information credibility.

Past studies on information evaluation, specifically credibility is often associated with users’ characteristics, source, or contents. This study sheds light on the context of task type, task difficulty and topical knowledge in affecting users’ information judgement.

One of the scarce studies in relating task orientation, task difficulty and topical knowledge to information evaluations.

This study explores the effects of cognitive load on the propensity to reformulate queries during information seeking on the web.

This study employs an experimental design to analyze the effect of manipulations of cognitive load on the propensity for query reformulation between experimental and control groups. In total, three affective components that contribute to cognitive load were manipulated: mental demand, temporal demand and frustration.

A significant difference in the propensity of query reformulation behavior was found between searchers exposed to cognitive load manipulations and searchers who were not exposed. Those exposed to cognitive load manipulations made half as many search query reformulations as searchers not exposed. Furthermore, the National Aeronautical and Space Administration Task Load Index (NASA-TLX) cognitive load scores of searchers who were exposed to the three cognitive load manipulations were higher than those of searchers who were not exposed indicating that the manipulation was effective. Query reformulation behavior did not differ across task types.

The findings suggest that a dual-task method and NASA-TLX assessment serve as good indicators of cognitive load. Because the findings show that cognitive load hinders a searcher's interaction with information search tools, this study provides empirical support for reducing cognitive load when designing information systems or user interfaces.

The purpose of this paper is to present a framework applicable to interactive video retrieval. The objective of the framework is so that it can be applied conceptually for understanding users and use of video digital libraries, and also practically for designing retrieval components like user interfaces.

The framework was developed through a user-centered and analytical approach, and serves as an initial attempt at generalizing how users interact when searching and browsing digital video, throughout different situations, along with the general designs that can be supportive.

The framework is two-fold, yet, together, comprises one set of conceptual findings. The first component of the framework depicts generalized user interactions throughout varying contexts of an interactive video retrieval process, followed by a second component, an illustration of the resulting supportive interface designs or sets of features. Cautions from previous studies not to over generalize the interactive process were heeded.

The implications for such research are based on the understanding that video retrieval will benefit from the advancement of user-centered foundations, which can guide and support design decisions for resources like digital libraries.

The need for this study is rather straightforward: there is currently not enough conceptual research of interactive video retrieval from a user-centered perspective, which contrasts with other areas of information retrieval research where the interaction process has been thoroughly examined for a variety of domains and contexts with implications for different retrieval tools like OPACs, search engines, and article databases.