Search results

This paper aims to explore the deep Web as a solution for displacement and replacement challenges in libraries, addressing the challenges, benefits, strategies and case studies.

The paper synthesizes existing literature on deep Web integration in libraries, providing a comprehensive analysis of insights from scholarly articles, case studies and expert opinions.

The deep Web grants libraries access to unique content, improving information access, fostering collaboration and enabling personalized content. However, security, privacy, ethics and data protection must be considered.

This paper contributes to the literature by providing a comprehensive examination of deep Web integration in libraries, offering valuable recommendations for navigating the changing landscape and leveraging the deep Web’s potential.

The purpose of this paper is to describe the development of an algorithm for realizing web crawlers that automatically collect dynamically generated webpages from the deep web.

This study proposes and develops an algorithm to collect web information as if the web crawler gathers static webpages by managing script commands as links. The proposed web crawler actually experiments with the algorithm by collecting deep webpages.

Among the findings of this study is that if the actual crawling process provides search results as script pages, the outcome only collects the first page. However, the proposed algorithm can collect deep webpages in this case.

To use a script as a link, a human must first analyze the web document. This study uses the web browser object provided by Microsoft Visual Studio as a script launcher, so it cannot collect deep webpages if the web browser object cannot launch the script, or if the web document contains script errors.

The research results show deep webs are estimated to have 450 to 550 times more information than surface webpages, and it is difficult to collect web documents. However, this algorithm helps to enable deep web collection through script runs.

This study presents a new method to be utilized with script links instead of adopting previous keywords. The proposed algorithm is available as an ordinary URL. From the conducted experiment, analysis of scripts on individual websites is needed to employ them as links.

The purpose of this paper is to decrease the traffic created by search engines’ crawlers and solve the deep web problem using an innovative approach.

A new algorithm was formulated based on best existing algorithms to optimize the existing traffic caused by web crawlers, which is approximately 40 percent of all networking traffic. The crux of this approach is that web servers monitor and log changes and communicate them as an XML file to search engines. The XML file includes the information necessary to generate refreshed pages from existing ones and reference new pages that need to be crawled. Furthermore, the XML file is compressed to decrease its size to the minimum required.

The results of this study have shown that the traffic caused by search engines’ crawlers might be reduced on average by 84 percent when it comes to text content. However, binary content faces many challenges and new algorithms have to be developed to overcome these issues. The proposed approach will certainly mitigate the deep web issue. The XML files for each domain used by search engines might be used by web browsers to refresh their cache and therefore help reduce the traffic generated by normal users. This reduces users’ perceived latency and improves response time to http requests.

The study sheds light on the deficiencies and weaknesses of the algorithms monitoring changes and generating binary files. However, a substantial decrease of traffic is achieved for text-based web content.

The findings of this research can be adopted by web server software and browsers’ developers and search engine companies to reduce the internet traffic caused by crawlers and cut costs.

The exponential growth of web content and other internet-based services such as cloud computing, and social networks has been causing contention on available bandwidth of the internet network. This research provides a much needed approach to keeping traffic in check.

Espouses the Web with regard to the media and all its areas of relevance. Encourages and supports multinational forms of production as new but admits they may be no more sympathetic to social need and democratic practice than previous commercial media. Charts the market and the Web’s changes for commercial business.

The purpose of this article is to provide a critical review of Bergman's study on the deep web. In addition, this study brings a new concept into the discussion, the academic invisible web (AIW). The paper defines the academic invisible web as consisting of all databases and collections relevant to academia but not searchable by the general‐purpose internet search engines. Indexing this part of the invisible web is central to scientific search engines. This paper provides an overview of approaches followed thus far.

Provides a discussion of measures and calculations, estimation based on informetric laws. Also gives a literature review on approaches for uncovering information from the invisible web.

Bergman's size estimate of the invisible web is highly questionable. This paper demonstrates some major errors in the conceptual design of the Bergman paper. A new (raw) size estimate is given.

The precision of this estimate is limited due to a small sample size and lack of reliable data.

This study can show that no single library alone will be able to index the academic invisible web. The study suggests a collaboration to accomplish this task.

Provides library managers and those interested in developing academic search engines with data on the size and attributes of the academic invisible web.

The purpose of this paper is to describe the work of the Office of Scientific and Technical Information (OSTI) in the US Department of Energy Office of Science and OSTI's development of the powerful search engine, WorldWideScience.org. With tools such as Science.gov and WorldWideScience.org, the patron gains access to multiple, geographically dispersed deep web databases and can search all of the constituent sources with a single query.

The paper is both historical and descriptive.

That WorldWideScience.org fills a unique niche in discovering scientific material in an information landscape that includes search engines such as Google and Google Scholar.

This is one of the few papers to describe in depth the important work being done by the US Office of Scientific and Technical Information in the field of search and discovery.

Modern Enterprise Web Application development can exploit third-party software components, both internal and external to the enterprise, that provide access to huge and valuable data sets, tested by millions of users and often available as Web application programming interfaces (APIs). In this context, the developers have to select the right data services and might rely, to this purpose, on advanced techniques, based on functional and non-functional data service descriptive features. This paper focuses on this selection task where data service selection may be difficult because the developer has no control on services, and source reputation could be only partially known.

The proposed framework and methodology are apt to provide advanced search and ranking techniques by considering: lightweight data service descriptions, in terms of (semantic) tags and technical aspects; previously developed aggregations of data services, to use in the selection process of a service the past experiences with the services when used in similar applications; social relationships between developers (social network) and their credibility evaluations. This paper also discusses some experimental results regarding the plan to expand other experiments to check how developers feel using the approach.

In this paper, a data service selection framework that extends and specializes an existing one for Web APIs selection is presented. The revised multi-layered model for data services is discussed and proper metrics relying on it, meant for supporting the selection of data services in a context of Web application design, are introduced. Model and metrics take into account the network of social relationships between developers, to exploit them for estimating the importance that a developer assigns to other developers’ experience.

This research, with respect to the state of the art, focuses attention on developers’ social networks in an enterprise context, integrating the developers’ credibility assessment and implementing the social network-based data service selection on top of a rich framework based on a multi-perspective model for data services.

This article examines the growth of the invisible Web, and efforts underway to make its contents more accessible. Dynamic Web publishing is described. The Open Archives Metadata Harvesting Protocol is reviewed, as are projects related to OCLC’s implementation of the Open Archives Initiative. Recent Dublin Core activities are reported.

Purpose – As the role of technology in libraries has broadened and expanded, tech-savvy librarians and non-librarian technologists are increasingly working side by side in complex digital environments. Little research has explored the key differences between these roles and the implications for the future of the Master of Library Science (MLS) and its variant degrees, particularly as technologists from various backgrounds increasingly enter the information field. This chapter contrasts the technological responsibilities of the two groups to build an understanding of the necessity of the MLS in library-oriented technology work.

Design/Methodology/Approach – Qualitative coding and text mining techniques were used to analyze technology-oriented librarian and non-librarian job advertisements, technology curriculum changes, and surveyed technology interests of current information professionals.

Findings – Findings indicate a clear distinction between librarian and non-librarian technology responsibilities. Librarian positions emphasize web design, data and metadata, technology troubleshooting, and usage of library-oriented software. Non-librarian technologists require programming, database development, and systems administration, with deeper software and systems knowledge. Overlap was noted in the areas of user experience, linked data, and metadata. Several newer trends that information professionals expressed a desire to learn – such as makerspace technologies – were observed to be poorly covered in the technology curriculum, though the MLS curriculum generally covered the tech-savvy librarians’ responsibilities.

Originality/Value – This chapter builds understanding of the current necessity of the MLS in library-oriented technology work, as contrasted against the role of non-librarian technologists, through analysis of a triangulated set of data sources covering employment opportunities, technology curriculum, and librarians’ technology interests.

This study aims to explore a framework for evaluating and comparing two federated search tools (FSTs) using two different retrieval protocols: XML gateways and Z39.50. FSTs are meta-information retrieval systems developed to facilitate the searching of multiple resources through a single search box. FSTs allow searching of heterogeneous platforms, such as bibliographic and full-text databases, online public access catalogues, web search engines and open-access resources.

The proposed framework consists of three phases: the usability testing, retrievability performance assessment and overall comparison. The think-aloud protocol was implemented for usability testing and FSTs retrieval consistency, and precision tests were carried out to assess the retrievability performance for 20 real user queries.

Participants were directed to assign weights for the interface usability and system retrievability importance as indicators for FST evaluation. Results indicated that FSTs retrievability performance was of more importance than the interface usability. Participants assigned an average weight of 62 per cent for the system retrievability and 38 per cent for interface usability. In terms of the usability test, there was no significant difference between the two FSTs, while minor differences were found regarding retrieval consistency and precision at 11-point cut-off recall. The overall evaluation showed that the FST based on the XML gateway rated slightly higher than the FST based on the Z39.50 protocol.

This empirical study faced several limitations. First, the lack of participants’ familiarity with usability testing created the need for a deep awareness and rigorous supervision. Second, the difficulties of empirically assessing participants’ perspectives and future attitudes called for mixing between a formal task and the think-aloud protocol for participants in a real environment. This has been a challenge that faced the collection of the usability data including user behaviour, expectations and other empirical data. Third, the differences between the two FSTs in terms of number of connectors and advanced search techniques required setting rigorous procedures for testing FSTs retrieval consistency and precision.

This paper has practical implications in two dimensions. First, its results could be utilized by FST developers to enhance their product’s performance. Second, the framework could be used by librarians to evaluate FSTs performance and capabilities. The framework enables them to compare between library systems in general and FSTs in particular. In addition to these practical implications, the authors encourage researchers to use and enhance the proposed framework.

Librarians can use the proposed framework to empirically select an FST, involving users in the selection procedures of these information retrieval systems, so that it accords with users’ perspectives and attitudes and serves the community better.

The proposed framework could be considered a benchmark for FST evaluation.