Search results

The purpose of this paper is to investigate the potential of altmetric and webometric indicators to aid with funding agencies’ evaluations of their funding schemes.

This paper analyses a range of altmetric and webometric indicators in terms of suitability for funding scheme evaluations, compares them to traditional indicators and reports some statistics derived from a pilot study with Wellcome Trust-associated publications.

Some alternative indicators have advantages to usefully complement scientometric data by reflecting a different type of impact or through being available before citation data.

The empirical part of the results is based on a single case study and does not give statistical evidence for the added value of any of the indicators.

A few selected alternative indicators can be used by funding agencies as part of their funding scheme evaluations if they are processed in ways that enable comparisons between data sets. Their evidence value is only weak, however.

This is the first analysis of altmetrics or webometrics from a funding scheme evaluation perspective.

Many early-career researchers (ECR) are motivated by the prospect of creating knowledge that is useful, not just within but also beyond the academic community. Although research facilities, funders and academic journals praise this eagerness for societal impact, the path toward such contributions is by no means straightforward. In this essay, we address five common concerns faced by ECRs when they strive for societal impact. We discuss the opportunity costs associated with impact work, the fuzziness of current impact measurement, the challenge of incremental results, the actionability of research findings, and the risk of saying something wrong in public. We reflect on these concerns in light of our own experience with impact work and conclude by suggesting a “post-heroic” perspective on impact, whereby seemingly mundane activities are linked in a meaningful way.

The purpose of this paper is to acknowledge that there are bibliometric differences between Social Sciences and Humanities (SSH) vs Science, Technology, Engineering and Mathematics (STEM). It is not so that either SSH or STEM has the right way of doing research or working as a scholarly community. Accordingly, research evaluation is not done properly in one framework based on either a method from SSH or STEM. However, performing research evaluation in two separate frameworks also has disadvantages. One way of scholarly practice may be favored unintentionally in evaluations and in research profiling, which is necessary for job and grant applications.

In the case study, the authors propose a tool where it may be possible, on one hand, to evaluate across disciplines and on the other hand to keep the multifaceted perspective on the disciplines. Case data describe professors at an SSH and a STEM department at Aalborg University. Ten partial indicators are compiled to build a performance web – a multidimensional description – and a one-dimensional ranking of professors at the two departments. The partial indicators are selected in a way that they should cover a broad variety of scholarly practice and differences in data availability.

A tool which can be used both for a one-dimensional ranking of researchers and for a multidimensional description is described in the paper.

Limitations of the study are that panel-based evaluation is left out and that the number of partial indicators is set to 10.

The paper describes a new tool that may be an inspiration for practitioners in research analytics.

The purpose of this paper is to better understand the theory and practice of peer review in open-access mega-journals (OAMJs). OAMJs typically operate a “soundness-only” review policy aiming to evaluate only the rigour of an article, not the novelty or significance of the research or its relevance to a particular community, with these elements being left for “the community to decide” post-publication.

The paper reports the results of interviews with 31 senior publishers and editors representing 16 different organisations, including 10 that publish an OAMJ. Thematic analysis was carried out on the data and an analytical model developed to explicate their significance.

Findings suggest that in reality criteria beyond technical or scientific soundness can and do influence editorial decisions. Deviations from the original OAMJ model are both publisher supported (in the form of requirements for an article to be “worthy” of publication) and practice driven (in the form of some reviewers and editors applying traditional peer review criteria to OAMJ submissions). Also publishers believe post-publication evaluation of novelty, significance and relevance remains problematic.

The study is based on unprecedented access to senior publishers and editors, allowing insight into their strategic and operational priorities. The paper is the first to report in-depth qualitative data relating specifically to soundness-only peer review for OAMJs, shedding new light on the OAMJ phenomenon and helping inform discussion on its future role in scholarly communication. The paper proposes a new model for understanding the OAMJ approach to quality assurance, and how it is different from traditional peer review.

This paper aims to assess whether the current eResearch Knowledge Centre’s (eRKC) research support practices align with researchers’ requirements for achieving their research objectives. The study’s objectives were to assess the current eRKC research support services and to determine which are adequate and which are not in supporting the Human Sciences Research Council (HSRC) researchers.

This study uses interviews as part of the qualitative approach. The researcher chose to use interviews, as some aspects warranted further explanation during the interview. The interviews were scheduled using Zoom’s scheduling assistant. The interviews were semi-structured, guided by a flexible interview procedure and supplemented by follow-up questions, probes and comments. The research life cycle questions guided the interviews. The data obtained were coded and transcribed using MS Excel. The interview data were analysed, using NVivo, according to the themes identified in the research questions and aligned with the theory behind the study. Pre-determined codes were created in line with the six stages of the research life cycle and applied to group the data and extract meaning from each category. Interviewee responses were assigned to groups in line with the stages of the research life cycle.

The current eRKC research support services are aligned with the needs of HSRC researchers and highlight services that could be expanded or promoted more effectively to HSRC researchers. It proposes a new service, data analysis, and suggests that the eRKC could play a more prominent role in research impact, research data management and fostering collaboration with HSRC research divisions.

This study is limited to assessing the eRKC’s support practices at the HSRC in Pretoria, South Africa. A more comprehensive study is needed for HSRC research services, capabilities and capacity.

Assessment of eRKC followed a comprehensive interviewee schedule that followed Raju and Schoombee’s research life cycle model.

Zoom’s scheduling assistant may have generated Zoom fatigue and reduced productivity. Technical issues, losing time, communication gaps and distant time zones may have affected face-to-face interaction.

eRKC research support practices are rare in South Africa and most parts of the world. This study bridges the gap between theory and practice in assessing eRKC research support practices.

Institutional and commercial web profiles that provide biobibliographic information about researchers are used for promotional purposes but also as information sources. In the latter case, the profiles' (re)presentations of researchers may be used to assess whether a researcher can be trusted. The article introduces a conceptual framework of how trust in researchers may be formed based on how the researchers' experiences and achievements are mobilized on the profiles to tell a multifaceted story of the “self.”

The framework is an analytical product which draws on theories of trust as well as on previous research focused on academic web profiles and on researchers' perceptions of trust and credibility. Two dimensions of trust are identified as central to the theoretical construction of trust, namely competence and trustworthiness.

The framework outlines features of profile content and narrative that may influence the assessment of the profile and of the researcher's competence and trustworthiness. The assessment is understood as shaped by the frames of interpretation available to a particular audience.

The framework addresses the lack of a trust perspective in previous research about academic web profiles. It provides an analysis of how potential trust in the researcher may be formed on the profiles. An innovative contribution is the acknowledgement of both qualitative and quantitative indicators of trustworthiness and competence, including the richness of the story told about the “self.”

Open-access mega-journals (OAMJs) represent an increasingly important part of the scholarly communication landscape. OAMJs, such as PLOS ONE, are large scale, broad scope journals that operate an open access business model (normally based on article-processing charges), and which employ a novel form of peer review, focussing on scientific “soundness” and eschewing judgement of novelty or importance. The purpose of this paper is to examine the discourses relating to OAMJs, and their place within scholarly publishing, and considers attitudes towards mega-journals within the academic community.

This paper presents a review of the literature of OAMJs structured around four defining characteristics: scale, disciplinary scope, peer review policy, and economic model. The existing scholarly literature was augmented by searches of more informal outputs, such as blogs and e-mail discussion lists, to capture the debate in its entirety.

While the academic literature relating specifically to OAMJs is relatively sparse, discussion in other fora is detailed and animated, with debates ranging from the sustainability and ethics of the mega-journal model, to the impact of soundness-only peer review on article quality and discoverability, and the potential for OAMJs to represent a paradigm-shifting development in scholarly publishing.

This paper represents the first comprehensive review of the mega-journal phenomenon, drawing not only on the published academic literature, but also grey, professional and informal sources. The paper advances a number of ways in which the role of OAMJs in the scholarly communication environment can be conceptualised.

This study aims to analyse and evaluate the methodology followed by the Times Higher Education Impact Rankings (THE-IR), as well as the coverage obtained and the data offered by this ranking, to determine if its methodology reflects the degree of sustainability of universities, and whether their results are accurate enough to be used as a data source for research and strategic decision-making.

A summative content analysis of the THE-IR methodology was conducted, paying special attention to the macro-structure (university score) and micro-structure (sustainable development goals [SDG] score) levels of the research-related metrics. Then, the data published by THE-IR in the 2019, 2020 and 2021 edition was collected via web scraping. After that, all the data was statistically analysed to find out performance rates, SDGs’ success rates and geographic distributions. Finally, a pairwise comparison of the THE-IR against the Times Higher Education World University Rankings (THE-WUR) was conducted to calculate overlap measures.

Severe inconsistencies in the THE-IR methodology have been found, offering a distorted view of sustainability in higher education institutions, allowing different strategic actions to participate in the ranking (interested, strategic, committed and outperformer universities). The observed growing number of universities from developing countries and the absence of world-class universities reflect an opportunity for less-esteemed institutions, which might have a chance to gain reputation based on their efforts towards sustainability, but from a flawed ranking which should be avoided for decision-making.

University managers can be aware of the THE-IR validity when demanding informed decisions. University ranking researchers and practitioners can access a detailed analysis of the THE-IR to determine its properties as a ranking and use raw data from THE-IR in other studies or reports. Policy makers can use the main findings of this work to avoid misinterpretations when developing public policies related to the evaluation of the contribution of universities to the SDGs. Otherwise, these results can help the ranking publisher to improve some of the inconsistencies found in this study.

Given the global audience of the THE-IR, this work contributes to minimising the distorted vision that the THE-IR projects about sustainability in higher education institutions, and alerts governments, higher education bodies and policy makers to take precautions when making decisions based on this ranking.

To the best of the authors’ knowledge, this contribution is the first providing an analysis of the THE-IR’s methodology. The faults in the methodology, the coverage at the country-level and the overlap between THE-IR and THE-WUR have unveiled the existence of specific strategies in the participation of universities, of interest both for experts in university rankings and SDGs.

The purpose of this paper is to present conceptual definitions for digital object use and reuse. Typically, assessment of digital repository content struggles to go beyond traditional usage metrics such as clicks, views or downloads. This is problematic for galleries, libraries, archives, museums and repositories (GLAMR) practitioners because use assessment does not tell a nuanced story of how users engage with digital content and objects.

This paper reviews prior research and literature aimed at defining use and reuse of digital content in GLAMR contexts and builds off of this group’s previous research to devise a new model for defining use and reuse called the use-reuse matrix.

This paper presents the use-reuse matrix, which visually represents eight categories and numerous examples of use and reuse. Additionally, the paper explores the concept of “permeability” and its bearing on the matrix. It concludes with the next steps for future research and application in the development of the Digital Content Reuse Assessment Framework Toolkit (D-CRAFT).

The authors developed this model and definitions to inform D-CRAFT, an Institute of Museum and Library Services National Leadership Grant project. This toolkit is being developed to help practitioners assess reuse at their own institutions.

To the best of the authors’ knowledge, this paper is one of the first to propose distinct definitions that describe and differentiate between digital object use and reuse in the context of assessing digital collections and data.

International Journal of Operations and Production Management (IJOPM)'s Impact Pathway (IP) section has been launched in 2020 to host short contributions grounded in current managerial practices and/or policy development, challenging established operations and supply chain management (OSCM) knowledge and highlighting innovative and relevant research directions. This commentary reflects on the achievements of the section, delineates the key features of IP papers and stimulates further development.

This commentary provides a brief overview of the IJOPM's IP section, taking stock of the contributions that have been published so far, analysing their topics, methodologies, insights and impact.

The 19 contributions published over the last three years have dealt with a variety of emerging topics, ranging from the COVID-19 response to additive manufacturing, leveraging on key evidence from managerial practice that challenges consolidated knowledge and theory, providing clear research directions as well as managerial and/or policy guidelines.

The commentary reflects on the importance of phenomenon-driven research that seeks to bridge the gap between theory and practice, thus increasing the impact and reach of OSCM research. This is a call for contributions from scholars, business leaders and policymakers to develop further impact-oriented research.