Search results

The purpose of this paper is to analyze the readability and level of word complexity of search engine results pages (SERPs) snippets and associated web pages between Google and Bing.

The authors employed the Readability Test Tool to analyze the readability and word complexity of 3,000 SERPs snippets and 3,000 associated pages in Google and Bing retrieved on 150 search queries issued by middle school children.

A significant difference was found in the readability of SERPs snippets and associated web pages between Google and Bing. A significant difference was also observed in the number of complex words in snippets between the two engines but not in associated web pages. At the engine level, the readability of Google and Bing snippets was significantly higher than associated web pages. The readability of Google SERPs snippets was at a much higher level than those of Bing. The readability of snippets in both engines mismatched with the reading comprehension of children in grades 6–8.

The data corpus may be small. Analysis relied on quantitative measures.

Practitioners and other mediators should mitigate the readability issue in SERPs snippets. Researchers should consider text readability and word complexity simultaneously with other factors to obtain the nuanced understanding of young users’ web information behaviors. Additional theoretical and methodological implications are discussed.

This study measured the readability and the level of word complexity embedded in SERPs snippets and compared them to respective web pages in Google and Bing. Findings provide further evidence of the readability issue of SERPs snippets and the need to solve this issue through system design improvements.

Web search is more and more moving into mobile contexts. However, screen size of mobile devices is limited and search engine result pages face a trade-off between offering informative snippets and optimal use of space. One factor clearly influencing this trade-off is snippet length. The purpose of this paper is to find out what snippet size to use in mobile web search.

For this purpose, an eye-tracking experiment was conducted showing participants search interfaces with snippets of one, three or five lines on a mobile device to analyze 17 dependent variables. In total, 31 participants took part in the study. Each of the participants solved informational and navigational tasks.

Results indicate a strong influence of page fold on scrolling behavior and attention distribution across search results. Regardless of query type, short snippets seem to provide too little information about the result, so that search performance and subjective measures are negatively affected. Long snippets of five lines lead to better performance than medium snippets for navigational queries, but to worse performance for informational queries.

Although space in mobile search is limited, this study shows that longer snippets improve usability and user experience. It further emphasizes that page fold plays a stronger role in mobile than in desktop search for attention distribution.

The purpose of this paper is to examine strengths and limitations that search engines (SEs) exhibit when responding to web search queries associated with the grade school curriculum

The authors employed a simulation-based experimental approach to conduct an in-depth empirical examination of SEs and used web search queries that capture information needs in different search scenarios.

Outcomes from this study highlight that child-oriented SEs are more effective than traditional ones when filtering inappropriate resources, but often fail to retrieve educational materials. All SEs examined offered resources at reading levels higher than that of the target audience and often prioritized resources with popular top-level domain (e.g. “.com”).

Findings have implications for human intervention, search literacy in schools, and the enhancement of existing SEs. Results shed light on the impact on children’s education that result from introducing misconception about SEs when these tools either retrieve no results or offer irrelevant resources, in response to web search queries pertinent to the grade school curriculum.

The authors examined child-oriented and popular SEs retrieval of resources aligning with task objectives and user capabilities–resources that match user reading skills, do not contain hate-speech and sexually-explicit content, are non-opinionated, and are curriculum-relevant. Findings identified limitations of existing SEs (both directly or indirectly supporting young users) and demonstrate the need to improve SE filtering and ranking algorithms.

It is well known that information behaviour can be biased in countless ways and that users of web search engines have difficulty in assessing the credibility of results. Yet, little is known about how search engine result page (SERP) listings are used to judge credibility and in which if any way such judgements are biased. The paper aims to discuss these issues.

Two studies are presented. The first collects data by means of a controlled, web-based user study (N=105). Studying judgements for three controversial topics, the paper examines the extent to which users agree on credibility, the extent to which judgements relate to those applied by objective assessors and to what extent judgements can be predicted by the users’ position on and prior knowledge of the topic. A second, qualitative study (N=9) utilises the same setup; however, transcribed think-aloud protocols provide an understanding of the cues participants use to estimate credibility.

The first study reveals that users are very uncertain when assessing credibility and their impressions often diverge from objective judges who have fact checked the sources. Little evidence is found indicating that judgements are biased by prior beliefs or knowledge, but differences are observed in the accuracy of judgements across topics. Qualitatively analysing think-aloud transcripts from participants think-aloud reveals ten categories of cues, which participants used to determine the credibility of results. Despite short listings, participants utilised diverse cues for the same listings. Even when the same cues were identified and utilised, different participants often interpreted these differently. Example transcripts show how participants reach varying conclusions, illustrate common mistakes made and highlight problems with existing SERP listings.

This study offers a novel perspective on how the credibility of SERP listings is interpreted when assessing search results. Especially striking is how the same short snippets provide diverse informational cues and how these cues can be interpreted differently depending on the user and his or her background. This finding is significant in terms of how search engine results should be presented and opens up the new challenge of discovering technological solutions, which allow users to better judge the credibility of information sources on the web.

College students need more support learning to effectively search for and evaluate online information. Without such skills, students are vulnerable to mis- and disinformation that may appear in their search results, Web browsing and social media feeds. This study investigated four short instructional modules four short instructional modules that were developed to be delivered asynchronously to teach effective approaches to online search and evaluation.

This exploratory study analyzed pre- and post-tests that students in two journalism courses completed before and after the modules. A total of 29 students completed written versions of the pre- and post-tests and 8 students participated in interviews in which they thought aloud while completing the pre- and post-tests. Written and oral responses were analyzed to understand students’ search and evaluation strategies and how, if at all, these shifted from pre- to post-test.

From pre- to post-test, students showed evidence of using strategies that were introduced in the modules to search for and evaluate online content. On the post-test, more students engaged in lateral reading to evaluate unfamiliar websites, used search operators and tools and more critically evaluated elements of the search engine results page.

This study provides initial, positive evidence for the efficacy of embedding short, asynchronous modules in college courses to improve students’ approaches to online searches and evaluations. College students need such support and modules like the ones investigated in this study may be one way to provide it.

Incorporating users’ behavior patterns could help in the ranking process. Different click models (CMs) are introduced to model the sophisticated search-time behavior of users among which commonly used the triple of attractiveness, examination and satisfaction. Inspired by this fact and considering the psychological definitions of these concepts, this paper aims to propose a novel learning to rank by redefining these concepts. The attractiveness and examination factors could be calculated using a limited subset of information retrieval (IR) features by the random forest algorithm, and then they are combined with each other to predicate the satisfaction factor which is considered as the relevance level.

The attractiveness and examination factors of a given document are usually considered as its perceived relevance and the fast scan of its snippet, respectively. Here, attractiveness and examination factors are regarded as the click-count and the investigation rate, respectively. Also, the satisfaction of a document is supposed to be the same as its relevance level for a given query. This idea is supported by the strong correlation between attractiveness-satisfaction and the examination-satisfaction. Applying random forest algorithm, the attractiveness and examination factors are calculated using a very limited set of the primitive features of query-document pairs. Then, by using the ordered weighted averaging operator, these factors are aggregated to estimate the satisfaction.

Experimental results on MSLR-WEB10K and WCL2R data sets show the superiority of this algorithm over the state-of-the-art ranking algorithms in terms of P@n and NDCG criteria. The enhancement is more noticeable in top-ranked items which are reviewed more by the users.

This paper proposes a novel learning to rank based on the redefinition of major building blocks of the CMs which are the attractiveness, examination and satisfactory. It proposes a method to use a very limited number of selected IR features to estimate the attractiveness and examination factors and then combines these factors to predicate the satisfactory which is regarded as the relevance level of a document with respect to a given query.

The purpose of this paper is to evaluate Google question-answering (QA) quality.

Given the large variety and complexity of Google answer boxes in search result pages, existing evaluation criteria for both search engines and QA systems seemed unsuitable. This study developed an evaluation criteria system for the evaluation of Google QA quality by coding and analyzing search results of questions from a representative question set. The study then evaluated Google’s overall QA quality as well as QA quality across four target types and across six question types, using the newly developed criteria system. ANOVA and Tukey tests were used to compare QA quality among different target types and question types.

It was found that Google provided significantly higher-quality answers to person-related questions than to thing-related, event-related and organization-related questions. Google also provided significantly higher-quality answers to where- questions than to who-, what- and how-questions. The more specific a question is, the higher the QA quality would be.

Suggestions for both search engine users and designers are presented to help enhance user experience and QA quality.

Particularly suitable for search engine QA quality analysis, the newly developed evaluation criteria system expanded and enriched assessment metrics of both search engines and QA systems.

The purpose of this paper is to gain a better understanding of the collaborative information seeking (CIS) behaviors of students conducting authentic group work projects, and the features of a collaborative search system that are most useful to these students.

An exploratory study was conducted with 41 participants in ten groups working on an in class, for-credit group project assignment utilizing a collaborative search system. Quantitative and qualitative data were gathered on the everyday search practices of students over the course of the group project, along with quality scores for the sources found.

Results showed that student behavior during their CIS related to the quality of their search outcomes, as the effective and efficient searchers found better quality sources. Students’ pre-task attitudes and experiences toward group work also relate to the quality of their search outcomes. Student feedback demonstrated the importance of making collaborative search tools convenient, lightweight, and easy to use.

These findings may be useful to researchers designing and studying the effectiveness of collaborative search tools, and to instructors planning to incorporate group projects into their classes.

In this paper, the authors document the authentic behaviors and attitudes of students conducting group projects in an classroom setting, and offer specific recommendations for developers of collaborative search systems. These findings provide greater context for CIS research into the collaborative search behaviors of students conducting group work projects.

Good abandonment behavior refers to users obtaining direct answers via search engine results pages (SERPs) without clicking any search result, which occurs commonly in mobile search. This study aims to better understand users' good abandonment behavior and perception, and then construct a good abandonment prediction model for mobile search with improved performance.

In this study, an in situ user mobile search experiment (N = 43) and a crowdsourcing survey (N = 1,379) were conducted. Good abandonment behavior was analyzed from a quantitative perspective, exploring users' search behavior characteristics from four aspects: session and query, SERPs, gestures and eye-tracking data.

Users show less engagement with SERPs in good abandonment, spending less time and using fewer gestures, and they pay more visual attention to answer-like results. It was also found that good abandonment behavior is often related to users' perceived difficulty of the searching tasks and trustworthiness in the search engine. A good abandonment prediction model in mobile search was constructed with a high accuracy (97.14%).

This study is the first to explore eye-tracking characteristics of users' good abandonment behavior in mobile search, and to explore users' perception of their good abandonment behavior. Visual attention features are introduced into good abandonment prediction in mobile search for the first time and proved to be important predictors in the proposed model.