Search results

This study aims to explore a framework for evaluating and comparing two federated search tools (FSTs) using two different retrieval protocols: XML gateways and Z39.50. FSTs are meta-information retrieval systems developed to facilitate the searching of multiple resources through a single search box. FSTs allow searching of heterogeneous platforms, such as bibliographic and full-text databases, online public access catalogues, web search engines and open-access resources.

The proposed framework consists of three phases: the usability testing, retrievability performance assessment and overall comparison. The think-aloud protocol was implemented for usability testing and FSTs retrieval consistency, and precision tests were carried out to assess the retrievability performance for 20 real user queries.

Participants were directed to assign weights for the interface usability and system retrievability importance as indicators for FST evaluation. Results indicated that FSTs retrievability performance was of more importance than the interface usability. Participants assigned an average weight of 62 per cent for the system retrievability and 38 per cent for interface usability. In terms of the usability test, there was no significant difference between the two FSTs, while minor differences were found regarding retrieval consistency and precision at 11-point cut-off recall. The overall evaluation showed that the FST based on the XML gateway rated slightly higher than the FST based on the Z39.50 protocol.

This empirical study faced several limitations. First, the lack of participants’ familiarity with usability testing created the need for a deep awareness and rigorous supervision. Second, the difficulties of empirically assessing participants’ perspectives and future attitudes called for mixing between a formal task and the think-aloud protocol for participants in a real environment. This has been a challenge that faced the collection of the usability data including user behaviour, expectations and other empirical data. Third, the differences between the two FSTs in terms of number of connectors and advanced search techniques required setting rigorous procedures for testing FSTs retrieval consistency and precision.

This paper has practical implications in two dimensions. First, its results could be utilized by FST developers to enhance their product’s performance. Second, the framework could be used by librarians to evaluate FSTs performance and capabilities. The framework enables them to compare between library systems in general and FSTs in particular. In addition to these practical implications, the authors encourage researchers to use and enhance the proposed framework.

Librarians can use the proposed framework to empirically select an FST, involving users in the selection procedures of these information retrieval systems, so that it accords with users’ perspectives and attitudes and serves the community better.

The proposed framework could be considered a benchmark for FST evaluation.

The purpose of this paper is to describe the processes the UCLA Library Website Redesign Team used to develop a new library website responsive to the needs of the broad population of UCLA Library users.

Using a combination of structured analyses of the previous library website, user surveys, a card sort protocol and a think‐aloud protocol, the Website Redesign Team procured sufficient information to meet its redesign goals: established clear site organization and navigation, utilized user‐centered nomenclature, ensured easy access from the library homepage to relevant information, developed a unified institutional visual identity throughout the site, and enabled a content management system.

Standard usability methods such as surveys and the card sort and think‐aloud protocols are essential tools for evaluating and redesigning complex multi‐layered websites. Since the redesign process is not finite, these tools contribute to keeping a website current and responsive to the needs of its users.

This case study provides an example that the Redesign Team hope will empower readers with tools and knowledge that they can use to perform similar tasks in their own environment.

Construction companies are increasingly adopting sensing technologies like laser scanners, making it necessary to upskill the future workforce in this area. However, limited jobsite access hinders experiential learning of laser scanning, necessitating the need for an alternative learning environment. Previously, the authors explored mixed reality (MR) as an alternative learning environment for laser scanning, but to promote seamless learning, such learning environments must be proactive and intelligent. Toward this, the potentials of classification models for detecting user difficulties and learning stages in the MR environment were investigated in this study.

The study adopted machine learning classifiers on eye-tracking data and think-aloud data for detecting learning stages and interaction difficulties during the usability study of laser scanning in the MR environment.

The classification models demonstrated high performance, with neural network classifier showing superior performance (accuracy of 99.9%) during the detection of learning stages and an ensemble showing the highest accuracy of 84.6% for detecting interaction difficulty during laser scanning.

The findings of this study revealed that eye movement data possess significant information about learning stages and interaction difficulties and provide evidence of the potentials of smart MR environments for improved learning experiences in construction education. The research implication further lies in the potential of an intelligent learning environment for providing personalized learning experiences that often culminate in improved learning outcomes. This study further highlights the potential of such an intelligent learning environment in promoting inclusive learning, whereby students with different cognitive capabilities can experience learning tailored to their specific needs irrespective of their individual differences.

The classification models will help detect learners requiring additional support to acquire the necessary technical skills for deploying laser scanners in the construction industry and inform the specific training needs of users to enhance seamless interaction with the learning environment.

The purpose of this study is to explore how individuals with overweight and obesity living in the UK respond to the public health and media messaging surrounding COVID-19 and obesity.

Qualitative interview study with a think aloud protocol. A total of 10 participants self-reported to have overweight, obesity or as actively trying to lose weight were recruited through social media and were asked to think aloud whilst exposed to four sets of public health and media materials describing the link between COVID-19 and obesity. Interviews were conducted over zoom, recorded and transcribed verbatim.

Three primary themes were identified through thematic analysis: “flawed messaging”, “COVID-19 as a teachable moment” and “barriers to change”. Transcending these themes was the notion of balance. Whilst the messaging around COVID-19 and obesity was deemed problematic; for some, it was a teachable moment to facilitate change when their future self and physical health was prioritised. Yet, when focussing on their mental health in the present participants felt more overwhelmed by the barriers and were less likely to take the opportunity to change.

Findings hold implications for public health messaging, highlighting the need for balance between being educational and informative but also supportive, so as to achieve maximum efficacy.

This study offers a novel and useful insight into how the public health and media messaging concerning COVID-19 risk and obesity is perceived by those with overweight and obesity.

In reflective writing, students are encouraged to examine their own setbacks and progress. With a shortage of guidance in how to provide feedback to students on this type of writing, teachers are often left to figure it out on the job. The central hypothesis in this paper is that the lens of reflective practice can help focus teacher efforts and ultimately improve both feedback and instruction. The purpose of this paper is not to produce a universal prescription for assessing reflective writing but rather a protocol for teacher reflective practice that can apply to challenging grading and feedback-giving situations.

Student assessment is a chance for teachers to learn about their students’ abilities and challenges and to provide feedback for improvement. Assessment and grading sessions can also become opportunities for teachers to examine their own instructional and assessment practices. This self-examination process, a cornerstone of reflective practice (Schön, 1984), is challenging, but it may be especially valuable when guidelines for feedback and assessment are hard to come by. Such may be said to be the case in student-centered learning environments such as school Fablabs and makerspaces, where stated goals commonly include cultivating learner self-regulation and resilience. These hard-to-measure constructs are typically assessed through analysis of student reflective journals. This in-depth case study uses mixed-methods to examine how a semester-long intervention affected the grading, feedback and instructional practices of a teacher in a hands-on design classroom. The intervention involved 10 grade-aloud sessions using a computer-based rubric tool (Gradescope) and a culminating card-sorting task. The lens of reflective practice was applied to understanding the teacher’s development of their own reflective capabilities.

During the intervention, the participating teacher grappled with grading and feedback-giving dilemmas which led to clarifications of assessment objectives; changes to instruction; and improved feedback-giving practices, many of which persisted after the intervention. The teacher perceived the intervention as adding both rigor and productive “soul-searching” to their professional practice. Lasting changes in feedback behaviors included a comprehensive rubric and an increase in the frequency, specificity and depth of feedback given to student written work.

Significant prior efforts have been directed separately at the use of reflective practice for teachers, in general, and on the feedback and grading of student process journals. This work combines these lines of inquiry in the reflective classroom assessment protocol, a novel on-the-job professional development opportunity that fosters reflective practice in times of assessment to improve instructional and feedback practices.

Usability testing using think aloud protocols was conducted on the MetaLib interface, a new federated search product soon to be offered by the university libraries. The object of the testing was to find problems users might experience with the MetaLib interface in order to improve site usability.

Based on responses to a demographic questionnaire, a representative sample of eight volunteers, diverse with respect to affiliation, discipline, gender, language, and computer expertise was selected. Using the MetaLib interface and guided by a moderator, participants completed real‐world tasks while verbalizing their thoughts. Participants also completed a questionnaire and answered three open‐ended items. Transcripts provided the data for the study, which were used to determine problems and difficulties with the interface.

The usability testing revealed one critical issue, that is, problems with the login. Other issues included problems with primary and secondary navigation, confusing terminology, and inconsistency with the site design and user expectations.

This was a qualitative study and results should not be used to generalize to a larger population, but rather to identify issues with usability.

The issues identified will provide feedback that will be used to address the usability of the software and the design of training, tutorials, and quick guides.

Results of this study will add to the body of information about the usability of a federated search product and can help others to select and/or design a usable product. This study can also be used for the future improvement of the MetaLib product.

This chapter explores the potential of grounded theory research methods for eliciting threshold concepts. It begins with an overview of threshold concept theory, then reviews current methodological approaches, as well as challenges encountered, when researching threshold concepts. The discussion argues for the suitability of grounded theory for this purpose, using a specific case for illustration. Specific elements of the research design that strengthened the use of grounded theory in the exploration of threshold concepts are described. The case example used is of graduate students and practicing professionals’ learning experiences when acquiring expertise in the online environment. The case is used to demonstrate the grounded theory method’s efficacy for eliciting evidence of transformative learning experiences, leading to implications for improving curriculum design.

If we want to improve managerial cognition, we need to capture the full spectrum of cognitive functions and the complex processes through which they unfold. I propose two very different methods (one older and low-tech, one newer and high-tech) that allow us to observe cognitive functions and processes directly in real time.

The efficient delivery of environmental information to wayfinders in complex environments is a challenge for information designers. Wayfinding tasks can be quite strenuous and frustrating in the visual absence of dedicated wayfinding information. This study aims to explore the behaviour regarding the use of wayfinding information by navigators in complex environments.

An experiment has been conducted in which participants have performed wayfinding tasks in a spatially complex university campus. The participants were instructed to use the think-aloud protocol during the experiment. The behaviour has been recorded using the head-mounted video recorder (GoPro), mobile phone screen (audio\video) recorder and interview. Twelve university students have been selected based on the equal level of spatial ability using the Santa Barbara Sense of Direction scale. Each participant performed three wayfinding tasks to locate the unknown locations inside the campus using a mobile wayfinding application and other information sources.

The results of this study demonstrated significant behavioural preferences in acquiring wayfinding information. Most of the participants synthesised the static and mobile wayfinding information sources, while some preferred only the static ones. Gender differences have also been found for planning and route finding. This study recommends the syntheses of static and mobile wayfinding information for designing an efficient institutional wayfinding system.

The sample size has been kept small because of the qualitative exploration of the wayfinding behaviour regarding the wayfinding information syntheses behaviour. The experiment findings can be further explored with larger data set and controlled behavioural metrics. This study can help understand the user requirements in facilities management for spatially complex institutional environments.

The current findings can be further used to develop a framework for wayfinding information designers to assist them in understanding the current practices and incorporate them for improving institutional wayfinding systems. The management of the offered facilities within an institution can be further improved to make the space more efficient by saving users’ time and efforts.

Information syntheses or symbiosis of environmental information with the beacon-based digital wayfinding system is a new concept. This study explores the potential of such information syntheses for enhancing the legibility of complex institutional environments.

The purpose of this paper is to provide a case study of the usability studies used by the Carnegie Mellon University Libraries during the redesign of their website.

The Libraries used a web‐based survey to determine needs, proceeding to the prototype design, and completing the process with the final design and user testing. Think‐aloud protocols, used to determine the strengths and weaknesses of the final design, asked participants to verbalize their thoughts as they completed a series of tasks.

The results of the protocols indicated several key weaknesses with respect to navigation, screen design and labeling, leading to more revisions and the final release. Testing indicated that color and graphics attract attention; font, labels, and placement increase visibility; chunking and leading with keywords increase readability; and consistency increases usability.

This is a case study and therefore not necessarily representative to the general population of library website design efforts.

This paper describes several methods of gathering feedback during website design or usability testing with an emphasis on think‐aloud protocols.

The techniques used here may be useful to others who are approaching redesign and usability testing of their own sites and interested in creating a user‐centered design.