Search results

The competition in the academe has always been tough, but today, the academe seems to be more like an industry than an academic community as academics are evaluated through quantified and economic means.

This article leans on Heidegger’s thoughts on the essence of technology and his ontological view on being to show the dangers that lie in this quantification of researchers and research.

Despite the benefits that information systems (ISs) offer to people and research, it seems that technology has made it possible to objectify researchers and research. This has a negative impact on the academe and should thus be looked into especially by the IS field, which should note the problems that exist in its core. This phenomenon of quantified academics is clearly visible at academic quantification sites, where academics are evaluated using metrics that count their output. It seems that the essence of technology has disturbed the way research is valued by emphasising its quantifiable aspects. The study claims that it is important to look for other ways to evaluate researchers rather than trying to maximise research production, which has led to the flooding of articles that few have the time or interest to read.

This paper offers new insights into the current phenomenon of quantification of academics and underlines the need for critical changes if in order to achieve the academic culture that is desirable for future academics.

This study aims to analyse and evaluate the methodology followed by the Times Higher Education Impact Rankings (THE-IR), as well as the coverage obtained and the data offered by this ranking, to determine if its methodology reflects the degree of sustainability of universities, and whether their results are accurate enough to be used as a data source for research and strategic decision-making.

A summative content analysis of the THE-IR methodology was conducted, paying special attention to the macro-structure (university score) and micro-structure (sustainable development goals [SDG] score) levels of the research-related metrics. Then, the data published by THE-IR in the 2019, 2020 and 2021 edition was collected via web scraping. After that, all the data was statistically analysed to find out performance rates, SDGs’ success rates and geographic distributions. Finally, a pairwise comparison of the THE-IR against the Times Higher Education World University Rankings (THE-WUR) was conducted to calculate overlap measures.

Severe inconsistencies in the THE-IR methodology have been found, offering a distorted view of sustainability in higher education institutions, allowing different strategic actions to participate in the ranking (interested, strategic, committed and outperformer universities). The observed growing number of universities from developing countries and the absence of world-class universities reflect an opportunity for less-esteemed institutions, which might have a chance to gain reputation based on their efforts towards sustainability, but from a flawed ranking which should be avoided for decision-making.

University managers can be aware of the THE-IR validity when demanding informed decisions. University ranking researchers and practitioners can access a detailed analysis of the THE-IR to determine its properties as a ranking and use raw data from THE-IR in other studies or reports. Policy makers can use the main findings of this work to avoid misinterpretations when developing public policies related to the evaluation of the contribution of universities to the SDGs. Otherwise, these results can help the ranking publisher to improve some of the inconsistencies found in this study.

Given the global audience of the THE-IR, this work contributes to minimising the distorted vision that the THE-IR projects about sustainability in higher education institutions, and alerts governments, higher education bodies and policy makers to take precautions when making decisions based on this ranking.

To the best of the authors’ knowledge, this contribution is the first providing an analysis of the THE-IR’s methodology. The faults in the methodology, the coverage at the country-level and the overlap between THE-IR and THE-WUR have unveiled the existence of specific strategies in the participation of universities, of interest both for experts in university rankings and SDGs.

This paper examines the socio-political affordances of metrics in research evaluation and the consequences of epistemic injustice in research practices and recorded knowledge.

First, the use of metrics is examined as a mechanism that promotes competition and social acceleration. Second, it is argued that the use of metrics in a competitive research culture reproduces systemic inequalities and leads to epistemic injustice. The conceptual analysis draws on works of Hartmut Rosa and Miranda Fricker, amongst others.

The use of metrics is largely driven by competition such as university rankings and league tables. Not only that metrics are not designed to enrich academic and research culture, they also suppress the visibility and credibility of works by minorities. As such, metrics perpetuate epistemic injustice in knowledge practices; at the same time, the reliability of metrics for bibliometric and scientometric studies is put into question.

As metrics leverage who can speak and who will be heard, epistemic injustice is reflected in recorded knowledge and what we consider to be information.

This paper contributes to the discussion of metrics beyond bibliometric studies and research evaluation. It argues that metrics-induced competition is antithetical to equality and diversity in research practices.

The purpose of this paper is to acknowledge that there are bibliometric differences between Social Sciences and Humanities (SSH) vs Science, Technology, Engineering and Mathematics (STEM). It is not so that either SSH or STEM has the right way of doing research or working as a scholarly community. Accordingly, research evaluation is not done properly in one framework based on either a method from SSH or STEM. However, performing research evaluation in two separate frameworks also has disadvantages. One way of scholarly practice may be favored unintentionally in evaluations and in research profiling, which is necessary for job and grant applications.

In the case study, the authors propose a tool where it may be possible, on one hand, to evaluate across disciplines and on the other hand to keep the multifaceted perspective on the disciplines. Case data describe professors at an SSH and a STEM department at Aalborg University. Ten partial indicators are compiled to build a performance web – a multidimensional description – and a one-dimensional ranking of professors at the two departments. The partial indicators are selected in a way that they should cover a broad variety of scholarly practice and differences in data availability.

A tool which can be used both for a one-dimensional ranking of researchers and for a multidimensional description is described in the paper.

Limitations of the study are that panel-based evaluation is left out and that the number of partial indicators is set to 10.

The paper describes a new tool that may be an inspiration for practitioners in research analytics.

The purpose of this study is to assess the contributions of graduate research to social innovation and change for learning and improved transdisciplinary practice. Universities, as centers of teaching and research, face high demand from society to address urgent social and environmental challenges. Faculty and students are keen to use their research to contribute to social innovation and sustainable development. As part of the effort to increase societal impact, research approaches are evolving to be more problem-oriented, engaged and transdisciplinary. Therefore, new approaches to research evaluation are also needed to learn whether and how research contributes to social innovation, and those lessons need to be applied by universities to train and support students to do impactful research and foster an impact culture.

This paper uses a theory-based evaluation method to assess the contributions of three completed doctoral research projects. Each study documents the project’s theory of change (ToC) and uses qualitative data (document review, surveys and interviews) to test the ToC. This paper uses a transdisciplinary research (TDR) quality assessment framework (QAF) to analyze each projects’ design and implementation. This paper then draws lessons from the individual case studies and a comparative analysis of the three cases on, namely, effective research design and implementation for social transformation; and training and support for impactful research.

Each project aimed to influence government policy, organizational practice, other research and/or the students’ own professional development. All contributed to many of their intended outcomes, but with varying levels of accomplishment. Projects that were more transdisciplinary had more pronounced outcomes. Process contributions (e.g. capacity-building, relationship-building and empowerment) were as or more important than knowledge contributions. The key recommendations are for: researchers to design intentional research, with an explicit ToC; higher education institutions (HEI) to provide training and support for TDR theory and practice; and HEIs to give more attention to research evaluation.

This is the first application of both the outcome evaluation method and the TDR QAF to graduate student research projects, and one of very few such analyses of research projects. It offers a broader framework for conceptualizing and evaluating research contributions to social change processes. It is intended to stimulate new thinking about research aims, approaches and achievements.

Literature practices represent the process leading up to the citation of a source, and consist of the selection, reading and citing of sources. The purpose of this paper is to explore possible factors that might influence researchers during this process and discover possible consequences of researchers’ citation behaviours.

In this exploratory study, various factors which could influence literature practices were explored via a questionnaire amongst 112 researchers. Participants were first authors of articles published in 2016 in one of five different journals within the disciplines of experimental psychology, educational sciences and social psychology. Academic positions of the participants ranged from PhD student to full professor.

Frequencies and percentages showed that researchers seemed to be influenced in their literature practices by various factors, such as editors suggesting articles and motivation to cite. Additionally, a high percentage of researchers reported taking shortcuts when citing articles (e.g. using secondary citations and reading selectively). Logistic regression did not reveal a clear relationship between academic work experience and research practices.

Seeing that researchers seem to be influenced by a variety of factors in their literature practices, the scientific community might benefit from better citation practices and guidelines in order to provide more structure to the process of literature practices.

This paper provides first insights into researchers’ literature practices. Possible reasons for problems with citation accuracy and replicating research findings are highlighted. Opportunities for further research on the topic of citation behaviours are presented.