Search results

1 – 10 of 17
To view the access options for this content please click here
Article
Publication date: 1 April 2000

C. Oppenheim, A. Morris, C. McKnight and S. Lowley

The literature of the evaluation of Internet search engines is reviewed. Although there have been many studies, there has been little consistency in the way such studies…

Abstract

The literature of the evaluation of Internet search engines is reviewed. Although there have been many studies, there has been little consistency in the way such studies have been carried out. This problem is exacerbated by the fact that recall is virtually impossible to calculate in the fast changing Internet environment, and therefore the traditional Cranfield type of evaluation is not usually possible. A variety of alternative evaluation methods has been suggested to overcome this difficulty. The authors recommend that a standardised set of tools is developed for the evaluation of web search engines so that, in future, comparisons can be made between search engines more effectively, and that variations in performance of any given search engine over time can be tracked. The paper itself does not provide such a standard set of tools, but it investigates the issues and makes preliminary recommendations of the types of tools needed.

Details

Journal of Documentation, vol. 56 no. 2
Type: Research Article
ISSN: 0022-0418

Keywords

To view the access options for this content please click here
Article
Publication date: 1 April 2001

D.C. Veal Doverton

Present and possible future developments in the techniques of document management are reviewed, the major ones being text retrieval and scanning and OCR. Acquisition…

Abstract

Present and possible future developments in the techniques of document management are reviewed, the major ones being text retrieval and scanning and OCR. Acquisition, indexing and thesauri, publishing and dissemination and the document management industry are also addressed. The emerging standards are reviewed and the impact of the Internet is analysed.

Details

Journal of Documentation, vol. 57 no. 2
Type: Research Article
ISSN: 0022-0418

Keywords

To view the access options for this content please click here
Article
Publication date: 1 December 1999

Charles Oppenheim and Karen Selby

The Internet gives access for blind and visually impaired users to previously unobtainable information via Braille or speech synthesis interpretation. This paper looks at…

Abstract

The Internet gives access for blind and visually impaired users to previously unobtainable information via Braille or speech synthesis interpretation. This paper looks at how three search engines, AltaVista, Yahoo! and Infoseek presented their information to a small group of visually impaired and blind users and how accessible individual Internet pages are. Two participants had varying levels of partial sight and two Subjects were blind and solely reliant on speech synthesis output. Subjects were asked for feedback on interface design at various stages of their search and any problems they encountered were noted. The barriers to access that were found appear to come about by lack of knowledge and thought by the page designers themselves. An accessible page does not have to be dull. By adhering to simple guidelines, visually impaired users would be able to access information more effectively than would otherwise be possible. Visually disabled people would also have the same opportunity to access knowledge as their sighted colleagues.

Details

Aslib Proceedings, vol. 51 no. 10
Type: Research Article
ISSN: 0001-253X

Keywords

To view the access options for this content please click here
Article
Publication date: 20 February 2007

Mary L. Robinson and Judith Wusteman

To describe a small‐scale quantitative evaluation of the scholarly information search engine, Google Scholar.

Abstract

Purpose

To describe a small‐scale quantitative evaluation of the scholarly information search engine, Google Scholar.

Design/methodology/approach

Google Scholar's ability to retrieve scholarly information was compared to that of three popular search engines: Ask.com, Google and Yahoo! Test queries were presented to all four search engines and the following measures were used to compare them: precision; Vaughan's Quality of Result Ranking; relative recall; and Vaughan's Ability to Retrieve Top Ranked Pages.

Findings

Significant differences were found in the ability to retrieve top ranked pages between Ask.com and Google and between Ask.com and Google Scholar for scientific queries. No other significant differences were found between the search engines. This may be due to the relatively small sample size of eight queries. Results suggest that, for scientific queries, Google Scholar has the highest precision, relative recall and Ability to Retrieve Top Ranked Pages. However, it achieved the lowest score for these three measures for non‐scientific queries. The best overall score for all four measures was achieved by Google. Vaughan's Quality of Result Ranking found a significant correlation between Google and scientific queries.

Research limitations/implications

As with any search engine evaluation, the results pertain only to performance at the time of the study and must be considered in light of any subsequent changes in the search engine's configuration or functioning. Also, the relatively small sample size limits the scope of the study's findings.

Practical implications

These results suggest that, although Google Scholar may prove useful to those in scientific disciplines, further development is necessary if it is to be useful to the scholarly community in general.

Originality/value

This is a preliminary study in applying the accepted performance measures of precision and recall to Google Scholar. It provides information specialists and users with an objective evaluation of Google Scholar's abilities across both scientific and non‐scientific disciplines and paves the way for a larger study.

Details

Program, vol. 41 no. 1
Type: Research Article
ISSN: 0033-0337

Keywords

To view the access options for this content please click here
Article
Publication date: 30 August 2018

Yiming Zhao, Jin Zhang, Xue Xia and Taowen Le

The purpose of this paper is to evaluate Google question-answering (QA) quality.

Abstract

Purpose

The purpose of this paper is to evaluate Google question-answering (QA) quality.

Design/methodology/approach

Given the large variety and complexity of Google answer boxes in search result pages, existing evaluation criteria for both search engines and QA systems seemed unsuitable. This study developed an evaluation criteria system for the evaluation of Google QA quality by coding and analyzing search results of questions from a representative question set. The study then evaluated Google’s overall QA quality as well as QA quality across four target types and across six question types, using the newly developed criteria system. ANOVA and Tukey tests were used to compare QA quality among different target types and question types.

Findings

It was found that Google provided significantly higher-quality answers to person-related questions than to thing-related, event-related and organization-related questions. Google also provided significantly higher-quality answers to where- questions than to who-, what- and how-questions. The more specific a question is, the higher the QA quality would be.

Research limitations/implications

Suggestions for both search engine users and designers are presented to help enhance user experience and QA quality.

Originality/value

Particularly suitable for search engine QA quality analysis, the newly developed evaluation criteria system expanded and enriched assessment metrics of both search engines and QA systems.

Details

Library Hi Tech, vol. 37 no. 2
Type: Research Article
ISSN: 0737-8831

Keywords

To view the access options for this content please click here
Article
Publication date: 1 December 2004

Alastair G. Smith

This paper explores resource discovery issues relating to New Zealand/Aotearoa information on the WWW in the twenty‐first century. Questions addressed are: How do New…

Abstract

This paper explores resource discovery issues relating to New Zealand/Aotearoa information on the WWW in the twenty‐first century. Questions addressed are: How do New Zealand search engines compare with global search engines for finding information relating to New Zealand? Can search engines find everything that is available on the web? What are effective strategies for finding information relating to New Zealand on the web? What is the quality of NZ information on the web? What can librarians do to make NZ information more accessible on the web? Based on a study, it concludes that neither local nor global search engines are by themselves sufficient, and that to maximize retrieval a variety of engines is necessary. The NZ librarian can play a role in ensuring that NZ information is made both available and accessible. Although the paper discusses the situation in New Zealand, the results and conclusions are applicable to other countries.

Details

The Electronic Library, vol. 22 no. 6
Type: Research Article
ISSN: 0264-0473

Keywords

To view the access options for this content please click here
Article
Publication date: 10 June 2020

Mahdi Zeynali Tazehkandi and Mohsen Nowkarizi

The purpose of this paper is to present a review on the use of the recall metric for evaluating information retrieval systems, especially search engines.

Abstract

Purpose

The purpose of this paper is to present a review on the use of the recall metric for evaluating information retrieval systems, especially search engines.

Design/methodology/approach

This paper investigates different researchers’ views about recall metrics.

Findings

Five different definitions for recall were identified. For the first group, recall refers to completeness, but it does not specify where all the relevant documents are located. For the second group, recall refers to retrieving all the relevant documents from the collection. However, it seems that the term “collection” is ambiguous. For the third group (first approach), collection means the index of search engines and, for the fourth group (second approach), collection refers to the Web. For the fifth group (third approach), ranking of the retrieved documents should also be accounted for in calculating recall.

Practical implications

It can be said that in the first, second and third approaches, the components of the retrieval algorithm, the retrieval algorithm and crawler, and the retrieval algorithm and crawler and ranker, respectively, are evaluated. To determine the effectiveness of search engines for the use of users, it is better to use the third approach in recall measurement.

Originality/value

The value of this paper is to collect, identify and analyse literature that is used in recall. In addition, different views of researchers about recall are identified.

Details

The Electronic Library , vol. 38 no. 3
Type: Research Article
ISSN: 0264-0473

Keywords

To view the access options for this content please click here
Article
Publication date: 6 March 2020

Mahdi Zeynali Tazehkandi and Mohsen Nowkarizi

The purpose was to evaluate the effectiveness of Google (as an international search engine) as well as of Parsijoo, Rismoon, and Yooz (as Persian search engines).

Abstract

Purpose

The purpose was to evaluate the effectiveness of Google (as an international search engine) as well as of Parsijoo, Rismoon, and Yooz (as Persian search engines).

Design/methodology/approach

In this research, Google search engine as an international search engine, and three local ones, Parsijoo, Rismoon, and Yooz, were selected for evaluation. Likewise, 32 subject headings were selected from the Persian Subject Headings List, and then simulated work tasks were assigned based on them. A total of 192 students from Ferdowsi University of Mashhad were asked to search for the information needed for simulated work tasks in the selected search engines, and then to copy the relevant website URLs in the search form.

Findings

The findings indicated that Google, Parsijoo, Rismoon, and Yooz had a significant difference in the precision, recall, and normalized discounted cumulative gain. There was also a significant difference in the effectiveness (average of precision, recall, and NDCG) of these four search engines in the retrieval of the Persian resources.

Practical implications

Users using an efficient search engine will attain more relevant documents, and Google search engine was more efficient in retrieving the Persian resources. It is recommended to use Google as it has a more efficient search.

Originality/value

In this research, for the first time, Google has been compared with local Persian search engines considering the new approach (simulated work tasks).

Details

Library Hi Tech, vol. 39 no. 1
Type: Research Article
ISSN: 0737-8831

Keywords

To view the access options for this content please click here
Article
Publication date: 1 April 2003

Alastair G. Smith

This study evaluates the retrieval of New Zealand information using three local New Zealand search engines, four major global search engines and three metasearch engines…

Abstract

This study evaluates the retrieval of New Zealand information using three local New Zealand search engines, four major global search engines and three metasearch engines. Searches for NZ topics were carried out on all the search engines, and the relative recall calculated. The local search engines did not achieve higher recall than the global search engines or metasearch engines, but no search engine achieved more than 45 percent recall. Despite the theoretical advantage of searching the databases of several individual search engines, metasearch engines did not achieve higher recall. Of relevant pages for the queries, 36 percent were outside the .nz domain. Implications for searching for geographically specific information, and for evaluation of search engines, are discussed.

Details

Online Information Review, vol. 27 no. 2
Type: Research Article
ISSN: 1468-4527

Keywords

To view the access options for this content please click here
Article
Publication date: 9 April 2019

Aabid Hussain, Sumeer Gul, Tariq Ahmad Shah and Sheikh Shueb

The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their…

Abstract

Purpose

The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their image retrieval capability. It is an effort to carry out a Cranfield experiment to know how efficient the commercial giants in the image search are and how efficient an image specific search engine is.

Design/methodology/approach

The keyword search feature of three ISEs – Google images, Yahoo Image Search and Picsearch – was exploited to make search with keyword captions of photos as query terms. Selected top ten images were used to act as a testbed for the study, as images were searched in accordance with features of the test bed. Features to be looked for included size (1200 × 800), format of images (JPEG/JPG) and the rank of the original image retrieved by ISEs under study. To gauge the overall retrieval effectiveness in terms of set standards, only first 50 result hits were checked. Retrieval efficiency of select ISEs were examined with respect to their precision and relative recall.

Findings

Yahoo Image Search outscores Google Images and Picsearch both in terms of precision and relative recall. Regarding other criteria – image size, image format and image rank in search results, Google Images is ahead of others.

Research limitations/implications

The study only takes into consideration basic image search feature, i.e. text-based search.

Practical implications

The study implies that image search engines should focus on relevant descriptions. The study evaluated text-based image retrieval facilities and thereby offers a choice to users to select best among the available ISEs for their use.

Originality/value

The study provides an insight into the effectiveness of the three ISEs. The study is one of the few studies to gauge retrieval effectiveness of ISEs. Study also produced key findings that are important for all ISE users and researchers and the Web image search industry. Findings of the study will also prove useful for search engine companies to improve their services.

Details

The Electronic Library , vol. 37 no. 1
Type: Research Article
ISSN: 0264-0473

Keywords

1 – 10 of 17