Search results

Purpose — To develop methodologies to evaluate search engines according to an individual's preference in an easy and reliable manner, and to formulate user-oriented metrics to compare freshness and duplication in search results.

Design/methodology/approach — A personalised evaluation model for comparing search engines is designed as a hierarchy of weighted parameters. These commonly found search engine features and performance measures are given quantitative and qualitative ratings by an individual user. Furthermore, three performance measurement metrics are formulated and presented as histograms for visual inspection. A methodology is introduced to quantitatively compare and recognise the different histogram patterns within the context of search engine performance.

Findings — Precision and recall are the fundamental measures used in many search engine evaluations due to their simplicity, fairness and reliability. Most recent evaluation models are user oriented and focus on relevance issues. Identifiable statistical patterns are found in performance measures of search engines.

Research limitations/implications — The specific parameters used in the evaluation model could be further refined. A larger scale user study would confirm the validity and usefulness of the model. The three performance measures presented give a reasonably informative overview of the characteristics of a search engine. However, additional performance parameters and their resulting statistical patterns would make the methodology more valuable to the users.

Practical implications — The easy-to-use personalised search engine evaluation model can be tailored to an individual's preference and needs simply by changing the weights and modifying the features considered. A user is able to get an idea of the characteristics of a search engine quickly using the quantitative measure of histogram patterns that represent the search performance metrics introduced.

Originality/value — The presented work is considered original as one of the first search engine evaluation models that can be personalised. This enables a Web searcher to choose an appropriate search engine for his/her needs and hence finding the right information in the shortest time with the least effort.

Purpose — The overall quality of an information retrieval system depends on many different aspects of the system and its users' information seeking behaviour, such as the speed of the system, the user interface, the query language and the features provided by the engine. One of the most important aspects is the effectiveness of the retrieval system, i.e. its ability to retrieve items that are relevant to the information need of an end user. This chapter focuses on methods for measuring effectiveness, in particular focusing on recent work that more directly models the utility of an engine to its users.

Methodology/approach — We discuss traditional approaches to effectiveness evaluation based on test collections, then transition to approaches based on test collections along with explicit models of user interaction with search results. We contrast this with approaches for which the user is ‘in the loop’, such as user studies and online evaluations.

Research limitations/implications — If it were possible to model users perfectly, we could directly estimate the utility of a search engine to its users; this would undoubtedly have a transformative effect on information retrieval and web search research. In practice, this goal will never be achievable because users exhibit far too much variability in how they approach the search engine, and furthermore provide valuable feedback that models and simulations cannot provide. Nevertheless, better models of user interaction will help develop better web search engines for a wider variety of tasks more rapidly.

Originality/value of paper — This is the first work that surveys recent work on user model-based evaluation and places it in a context with traditional evaluation based on the Cranfield paradigm.

Purpose – Global cooperation between and within organisations has become essential for successful businesses. For the information management within such an international and necessarily multilingual environment, new challenges arise due to the diversity of the stakeholders and participants as well as due to the heterogeneity of approaches and traditions of information handling.

Design/methodology/approach – Key technologies like search technologies need to be adapted to support content in multiple languages and efficient access to it. Information processes need to be analysed while bearing in mind that problems may arise due to cross-cultural misunderstandings. The diversity requires appropriate treatment and appropriate methods in information systems in order to improve international information flows.

Findings – This chapter identifies some of these challenges and shows how they can be approached from an information science perspective. User-oriented research at the University of Hildesheim in the areas information retrieval, information seeking and human–computer interaction is presented.

Originality/value – Global enterprises and organisations may use this chapter to identify challenges and solutions for adapting their information technology to an international scale. Researchers who work on multilingual information access and intercultural aspects of information systems get an overview on some current research.

Purpose – As researchers need partners to collaborate with, this study aims to provide author recommendation for academic researchers for potential collaboration, conference planning, and compilation of scientific working groups with the help of social information. Hereby the chapter analyzes and compares different similarity metrics in information and computer science.

Methodology/approach – The study uses data from the multidiscipline information services Web of Science and Scopus as well as the social bookmarking service CiteULike to measure author similarity and recommend researchers to unique target researchers. The similarity approach is based on author co-citation, bibliographic coupling of authors and collaborative filtering methods. The developed clusters and graphs are then evaluated by these target researchers.

Findings – The analysis shows, for example, that different methods for social recommendation complement each other and that the researchers evaluated user- and tag-based data from a social bookmarking system positively.

Research limitations/implications – The present study, providing author recommendation for six target physicists, is supposed to be a starting point for further approaches on social academic author recommendation.

Practical implications – The chapter investigates in recommendation methods and similarity algorithm models as basis for an implementation of a social recommendation system for researchers in academics and knowledge-intensive organizations.

Originality/value of chapter – The comparison of different similarity measurements and the user evaluation provide new insights into the construction of social data mining and the investigation of personalized recommendation.

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish higher education, with particular reference to the longitudinal evaluative research of support of teaching and learning (T&L) (the Quality Enhancement Framework or QEF). The chapter will discuss the features of the research process which are shaped by evaluation theory. We adopt a theoretical position on policy research which foregrounds the situated experience of policy as a core research focus. Policy is depicted as being underscored by an implicit theory of change which is used to structure and orientate the research focus. The design of the research is characterised by the involvement of potential users of the research output, with implications on the way in which findings are articulated, presented and ultimately used, along with aspects of the evaluative research design. The case study of the QEF will be contextualised, and the intersection between the design features and theoretical approaches, and the use and usability of research outputs, will be established.

Purpose — The purpose of this chapter is to give an overview of the context of Web search and search engine related research, as well as to introduce the reader to the sections and chapters of the book.

Methodology/approach — We review literature dealing with various aspects of search engines, with special emphasis on emerging areas of Web searching, search engine evaluation going beyond traditional methods and new perspectives on Web searching.

Findings — The approaches to studying Web search engines are manifold. Given the importance of Web search engines for knowledge acquisition, research from different perspectives needs to be integrated into a more cohesive perspective.

Research limitations/implications — The chapter suggests a basis for research in the field and also introduces further research directions.

Originality/value of paper — The chapter gives a concise overview of the topics dealt within the book and also shows directions for researchers interested in Web search engines.

Collaboration is often required for activities that are too complex or difficult to be dealt with by a single individual. Many situations requiring information-seeking activities also call for people to work together. Often the methods, systems, and tools that provide access to information assume that they are used only by individuals working on their tasks alone. This review points to the need to acknowledge the importance of collaboration in information-seeking processes, to study models, and to develop systems that are specifically designed to enable collaborative information seeking (CIS) tasks. This chapter reviews the literature from various domains including library and information science, human–computer interaction, collaborative systems, and information retrieval. Focus of the review is on the extent to which people work together on information seeking tasks and the systems and tools that are available for them to be successful. Since CIS occurs in the broader context of collaboration in general, a review of literature about collaborations is first undertaken to define it and place it into context with related terms such as cooperation and communication. A more focused review of research follows relating CIS to systems that have attempted to support such interactions. Included are identification and synthesis of a number of core issues in the field and how best to evaluate systems and collaborative tools. Key lessons learned from the review are summarized, and gaps in the literature identified to spur future research and study.