Search results

During the evaluation of European Capital of Culture (ECoC) Aarhus 2017, the evaluation organisation rethinkIMPACTS 2017 formulated a set of “dilemmas” capturing the main challenges arising during the design of the ECoC evaluation. This functioned as a framework for the evaluation process. This paper aims to present and discuss the relevance of the “Evaluation Dilemmas Model” as subsequently applied to the Galway 2020 ECoC programme evaluation.

The paper takes an empirical approach including auto-ethnography and interview data to document and map the dilemmas involved in undertaking an evaluation in two different European cities. Evolved via a process of practice-based research, the article addresses the development of and the arguments for the dilemmas model and considers its potential for wider applicability in the evaluation of large-scale cultural projects.

The authors conclude that the “Evaluation Dilemmas Model” is a valuable heuristic for considering the endogenous and exogenous issues in cultural evaluation.

The model developed is useful for a wide range of cultural evaluation processes including – but not limited to – European Capitals of Culture.

What has not been addressed in the academic literature is the process of evaluating ECoCs; especially how evaluators often take part in an overall process that is not just about the evaluation but also planning and delivering a project that includes stakeholder management and the development of evaluation criteria, design and methods.

This paper aims to evaluate the Digital Government Collaborative Platform (DGCP), which facilitates collaborations between the citizens and the government to address environmental issues in Sri Lanka. The DGCP is an artifact developed by the value-sensitive design approach.

The DGCP is evaluated following the Framework for Evaluation in Design Science Research (FEDS). In total, 224 citizens participated in the survey based on the User Experience Questionnaire (UEQ) and open questions about human values embedded in the design. Fifteen government officers were interviewed to enhance the evaluation.

The DGCP received positive evaluations from the citizens and government officers. The platform is attractive, novel and pragmatic, also generating hedonic experiences for the citizens. The users believed that human values are reflected in the DGCP. Furthermore, they shared a few suggestions to improve it.

The paper contributes knowledge to evaluating digital government systems, especially in developing countries. The human-value-centered DGCP was evaluated using multiple methods of quantitative (i.e. UEQ Survey) and qualitative (i.e. qualitative interviews with stakeholders) techniques. Furthermore, the systematic process of DGCP evaluation produces a case-based guideline for evaluating related and similar digital government systems using FEDS.

To elaborate the nature of fact-checking in the domain of political information by examining how fact-checkers assess the validity of claims concerning the Russo-Ukrainian conflict and how they support their assessments by drawing on evidence acquired from diverse sources of information.

Descriptive quantitative and qualitative content analysis of 128 reports written by the fact-checkers of Snopes – an established fact-checking organisation – during the period of 24 February 2022 – 28 June, 2023. For the analysis, nine evaluation grounds were identified, most of them inductively from the empirical material. It was examined how the fact-checkers employed such grounds while assessing the validity of claims and how the assessments were bolstered by evidence acquired from information sources such as newspapers.

Of the 128 reports, the share of assessments indicative of the invalidity of the claims was 54.7%, while the share of positive ratings was 26.7%. The share of mixed assessments was 15.6%. In the fact-checking, two evaluation grounds, that is, the correctness of information and verifiability of an event presented in a claim formed the basis for the assessment. Depending on the topic of the claim, grounds such as temporal and spatial compatibility, as well as comparison by similarity and difference occupied a central role. Most popular sources of information offering evidence for the assessments include statements of government representatives, videos and photographs shared in social media, newspapers and television programmes.

As the study concentrated on fact-checking dealing with political information about a specific issue, the findings cannot be extended to concern the fact-checking practices in other contexts.

The study is among the first to characterise how fact-checkers employ evaluation grounds of diverse kind while assessing the validity of political information.

In order to estimate the value of semi-automated subject indexing in operative library catalogues, the study aimed to investigate five different automated implementations of an open source software package on a large set of Swedish union catalogue metadata records, with Dewey Decimal Classification (DDC) as the target classification system. It also aimed to contribute to the body of research on aboutness and related challenges in automated subject indexing and evaluation.

On a sample of over 230,000 records with close to 12,000 distinct DDC classes, an open source tool Annif, developed by the National Library of Finland, was applied in the following implementations: lexical algorithm, support vector classifier, fastText, Omikuji Bonsai and an ensemble approach combing the former four. A qualitative study involving two senior catalogue librarians and three students of library and information studies was also conducted to investigate the value and inter-rater agreement of automatically assigned classes, on a sample of 60 records.

The best results were achieved using the ensemble approach that achieved 66.82% accuracy on the three-digit DDC classification task. The qualitative study confirmed earlier studies reporting low inter-rater agreement but also pointed to the potential value of automatically assigned classes as additional access points in information retrieval.

The paper presents an extensive study of automated classification in an operative library catalogue, accompanied by a qualitative study of automated classes. It demonstrates the value of applying semi-automated indexing in operative information retrieval systems.

The purpose of this paper is to evaluate the impact of leadership development programmes, underpinned by Transformational Learning Theory (TLT).

A corpus-informed analysis was conducted using survey data from 690 participants. Data were collected from participants’ responses to the question “please tell us about the impact of your overall experience”, which culminated in a combined corpus of 75,053 words.

Findings identified patterns of language clustered around the following frequently used word types, namely, confidence; influence; self-awareness; insight; and impact.

This in-depth qualitative evaluation of participants’ feedback has provided insight into how TLT can be applied to develop future health-care leaders. The extent to which learning has had a transformational impact at the individual level, in relation to their perceived ability to influence, holds promise for the wider impact of this group in relation to policy, practice and the promotion of clinical excellence in the future. However, the latter can only be ascertained by undertaking further realist evaluation and longitudinal study to understand the mechanisms by which transformational learning occurs and is successfully translated to influence in practice.

Previous research has expounded traditional leadership theories to guide the practice of health-care leadership development. The paper goes some way to demonstrate the impact of using the principles of TLT within health-care leadership development programmes. The approach taken by The Florence Nightingale Foundation has the potential to generate confident leaders who may be instrumental in creating positive changes across various clinical environments.

Self-regulated learning (SRL) is crucial for successful learning and lifelong learning in today’s rapidly changing world, yet research has shown that many learners need support for SRL. Recently, learning analytics has offered exciting opportunities for better understanding and supporting SRL. However, substantial endeavors are still needed not only to detect learners’ SRL processes but also to incorporate human values, individual needs and goals into the design and development of self-regulated learning analytics (SRLA). This paper aims to examine the challenges that lifelong learners faced in SRL, their needs and desirable features for SRLA.

This study triangulated data collected from three groups of educational stakeholders: focus group discussions with lifelong learners (n = 27); five teacher interviews and four expert evaluations. The groups of two or three learners discussed perceived challenges, support needs and willing-to-share data contextualized in each phase of SRL.

Lifelong learners in professional development programs face challenges in managing their learning time and motivation, and support for time management and motivation can improve their SRL. This paper proposed and evaluated a set of design principles for SRLA.

This paper presents a novel approach for theory-driven participatory design with multistakeholders that involves integrating learners, teachers and experts’ perspectives for designing SRLA. The results of the study will answer the questions of how learners’ voices can be integrated into the design process of SRLA and offer a set the design principles for the future development of SRLA.

This study was conducted to address the methodical shortcomings and high associated cost of understanding the use of new, poorly understood architectural spaces, such as makerspaces. The proposed quantified method of enhancing current post-occupancy evaluation (POE) practices aims to provide architects, engineers and building professionals with accessible and intuitive data that can be used to conduct comparative studies of spatial changes, understand changes over time (such as those resulting from COVID-19) and verify design intentions after construction through a quantified post-occupancy evaluation.

In this study, we demonstrate the use of ultra-wideband (UWB) technology to gather, analyze and visualize quantified data showing interactions between people, spaces and objects. The experiment was conducted in a makerspace over a four-day hackathon event with a team of four actively tracked participants.

The study shows that by moving beyond simply counting people in a space, a more nuanced pattern of interactions can be discovered, documented and analyzed. The ability to automatically visualize findings intuitively in 3D aids architects and visual thinkers to easily grasp the essence of interactions with minimal effort.

By providing a method for better understanding the spatial and temporal interactions between people, objects and spaces, our approach provides valuable feedback in POE. Specifically, our approach aids practitioners in comparing spaces, verifying design intent and speeding up knowledge building when developing new architectural spaces, such as makerspaces.

Adolescent mental health is a global concern. There is an urgent need for creative, multimedia interventions reflecting adolescent culture to promote mental health literacy and well-being. This study aims to assess the impact of a film-based intervention on adolescent mental health literacy, well-being and resilience.

A pretest-posttest intervention with a multi-methods evaluation was used. A convenience sample of ten schools facilitated students aged 15–17 years to engage in an online intervention (film, post-film discussion, well-being Webinar). Participants completed surveys on well-being, resilience, stigma, mental health knowledge and help-seeking. Five teachers who facilitated the intervention participated in post-implementation interviews or provided a written submission. Analysis included paired-t-test and effect size calculation and thematic analysis.

Matched pretest-posttest data were available on 101 participants. There were significant increases in well-being, personal resilience and help-seeking attitudes for personal/emotional problems, and suicidal ideation. Participants’ free-text comments suggested the intervention was well-received, encouraging them to speak more openly about mental health. Teachers similarly endorsed the intervention, especially the focus on resilience.

Intinn shows promise in improving adolescents’ mental health literacy and well-being. Film-based interventions may encourage adolescents to seek professional help for their mental health, thus facilitating early intervention.