Search results

The purpose of this paper is to provide an overview and evaluation of F1000, a publishing outlet and peer review system for research in the biomedical and life sciences.

The review chronicles the rise of F1000 and describes the site’s functionalities and use capabilities.

The findings detail both the strengths and limitations of F1000 and point toward avenues for continued research and development.

This is the first review to provide a substantive evaluation of F1000 for academics to consider when adopting, using and researching the platform.

Peer review has been with humans for a long time. Its effective inception dates back to World War II resulting information overload, which imposed a quantitative and qualitative screening of publications. Peer review was beset by a number of accusations and critics largely from the biases and subjective aspects of the process including the secrecy in which the processes became standard. Advent of the Internet in the early 1990s provided a manner to open peer review up to make it more transparent, less iniquitous, and more objective. This chapter investigates whether this openness led to a more objective manner of judging scientific publications. Three sites are examined: Electronic Transactions on Artificial Intelligence (ETAI), Atmospheric Chemistry and Physics (ACP), and Faculty of 1000 (F1000). These sites practice open peer review wherein reviewers and authors and their reviews and rebuttals are available for all to see. The chapter examines the different steps taken to allow reviewers and authors to interact and how this allows for the entire community to participate. This new prepublication reviewing of papers has to some extent, alleviated the biases that were previously preponderant and, furthermore, seems to give positive results and feedback. Although recent, experiences seem to have elicited scientists’ acceptance because openness allows for a more objective and fair judgment of research and scholarship. Yet, it will undoubtedly lead to new questions which are examined in this chapter.

The purpose of this case study is to investigate the usefulness of altmetrics for measuring the broader impact of research.

This case study is based on a sample of 1,082 the Public Library of Science (PLOS) journal articles recommended in F1000. The data set includes altmetrics which were provided by PLOS. The F1000 data set contains tags on papers which were assigned by experts to characterise them.

The most relevant tag for altmetric research is “good for teaching”, as it is assigned to papers which could be of interest to a wider circle of readers than the peers in a specialised area. One could expect papers with this tag to be mentioned more often on Facebook and Twitter than those without this tag. The results from regression models were able to confirm these expectations: papers with this tag show significantly higher Facebook and Twitter counts than papers without this tag. This clear association could not be seen with Mendeley or Figshare counts (that is with counts from platforms which are chiefly of interest in a scientific context).

The results of the current study indicate that Facebook and Twitter, but not Figshare or Mendeley, might provide an indication of which papers are of interest to a broader circle of readers (and not only for the peers in a specialist area), and could therefore be useful for the measurement of the societal impact of research.

This study aimed to investigate how the enrichment of medical documents' index terms by their comments improves the relevance and novelty of the top-ranked results retrieved by an NLP system.

A semi-experimental pre-test and post-test research was designed to compare NLP-based indexes before and after being expanded by the comment terms. The experiments were conducted on a test collection of 13,957 documents commented by F1000-Prime reviewers. They were indexed at title, abstract, body and full-text levels. In total, 100 seed documents were randomly selected and served as queries. The textual similarity of the documents and queries was calculated using Lucene-more-like-this function and evaluated by the semantic similarity of their MeSH. The results novelty was measured using maximal marginal relevance and evaluated by their MeSH novelties. Normalized discounted cumulative gain was used to compare the basic and expanded indexes' precisions at 10, 20 and 50 top ranks.

The relevance and novelty of the results ranked at the top precision points was improved after expanding the indexes by the comment terms. The finding implies that meta-texts are effective in representing their mother documents, by adding dynamic elements to their rather static contents. It also provides further evidence about the merits of the application of social intelligence and collective wisdom reflected in the actions and reactions of users in tackling the challenges faced by NLP-based systems.

This is the first study to confirm that social comments on scientific papers improve the performance of information systems in terms of relevance and novelty.

The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-05-2022-0283.

The purpose of this study is to contribute to the understanding of how the potential of altmetrics varies around the world by measuring the percentage of articles with non-zero metrics (coverage) for articles published from a developing region (Latin America).

This study uses article metadata from a prominent Latin American journal portal, SciELO, and combines it with altmetrics data from Altmetric.com and with data collected by author-written scripts. The study is primarily descriptive, focusing on coverage levels disaggregated by year, country, subject area, and language.

Coverage levels for most of the social media sources studied was zero or negligible. Only three metrics had coverage levels above 2 per cent – Mendeley, Twitter, and Facebook. Of these, Twitter showed the most significant differences with previous studies. Mendeley coverage levels reach those found by previous studies, but it takes up to two years longer for articles to be saved in the reference manager. For the most recent year, coverage was less than half than what was found in previous studies. The coverage levels of Facebook appear similar (around 3 per cent) to that of previous studies.

The Altmetric.com data used for some of the analyses were collected for a six month period. For other analyses, Altmetric.com data were only available for a single country (Brazil).

The results of this study have implications for the altmetrics research community and for any stakeholders interested in using altmetrics for evaluation. It suggests the need of careful sample selection when wishing to make generalizable claims about altmetrics.

Open-access mega-journals (OAMJs) represent an increasingly important part of the scholarly communication landscape. OAMJs, such as PLOS ONE, are large scale, broad scope journals that operate an open access business model (normally based on article-processing charges), and which employ a novel form of peer review, focussing on scientific “soundness” and eschewing judgement of novelty or importance. The purpose of this paper is to examine the discourses relating to OAMJs, and their place within scholarly publishing, and considers attitudes towards mega-journals within the academic community.

This paper presents a review of the literature of OAMJs structured around four defining characteristics: scale, disciplinary scope, peer review policy, and economic model. The existing scholarly literature was augmented by searches of more informal outputs, such as blogs and e-mail discussion lists, to capture the debate in its entirety.

While the academic literature relating specifically to OAMJs is relatively sparse, discussion in other fora is detailed and animated, with debates ranging from the sustainability and ethics of the mega-journal model, to the impact of soundness-only peer review on article quality and discoverability, and the potential for OAMJs to represent a paradigm-shifting development in scholarly publishing.

This paper represents the first comprehensive review of the mega-journal phenomenon, drawing not only on the published academic literature, but also grey, professional and informal sources. The paper advances a number of ways in which the role of OAMJs in the scholarly communication environment can be conceptualised.

Numerous research tools exist but their usage among researchers across the different phases of the research cycle of scholarly communication remains unknown. This research aims to address this knowledge gap by mapping the research tools frequently used by global researchers against the various phases of the research cycle of scholarly communication.

This research adopts a descriptive research design and conducts a cross-tabulation of secondary data consisting of 20,663 useable responses in a global survey of research tools for scholarly communication. This research also administered a survey to academic experts to classify the research tools according to traditional, modern, innovative and experimental categories.

This research reveals the six phases of the research cycle (i.e. discovery, analysis, writing, publication, outreach and assessment) and the research tools of scholarly communication frequently used by researchers worldwide in each phase as a whole and by roles, disciplines, regions and career stages. Notably, this research indicates that most of the research tools used by researchers are classified as “modern” and “innovative”.

The original insights herein should be useful for both established and early career researchers to gain and share research insights, as well as policymakers and existing and aspiring service providers who wish to improve the utility and usage of research tools for scholarly communication. Notably, this research represents a seminal endeavor at enhancing a global survey (secondary research) using a follow-up expert survey (primary research), which enabled the organization of research tools for scholarly communication into four refined categories. In doing so, this research contributes finer-grained insights that showcase the importance of keeping up with the advancement of technology through the use of modern, innovative and experimental research tools, thereby highlighting the need to go beyond traditional research tools for scholarly communication.

The purpose of this paper is to present the evolution in notions from bibliometrics to altmetrics and confront them taking into consideration specific criteria. The objective of this paper is to present the evolution of research, regarding the above fields, the study of metrics and indicators used, and the strength and weaknesses resulting from the current literature. Furthermore, the authors present the manipulation techniques for both fields as their main weakness, as well as further key points, analyzing the alternative options of bibliometrics and altmetrics.

First, the authors present the evolution of the literature, concerning the specific field and metrics used, following with a brief description of basic indicators related to the field of bibliometrics (journal impact factor (JIF), eigenfactor, article influence score and h-index) discussing their advantages and disadvantages. In the second part, the authors describe altmetrics and present the differences with bibliometrics.

Both bibliometrics and altmetrics remain weak indicators as fraught with disadvantages with manipulation being the greatest of all. Nevertheless, the combination of the two is proposed in order to export safer conclusions on assessing the impact. Regarding the manipulation there is yet not a clean technique to eliminate manipulation. In specific, regarding bibliometrics, the manipulation of indicators refers only to the human factor intervention. The theoretical implication of this study constitutes of collecting the relevant literature regarding scientific indicators.

We must consider the study of new indicators, which combine metrics and methodologies used in both bibliometrics and altmetrics. The theoretical implication of this study constitutes of collecting the relevant literature regarding scientific indicators. Therefore, researchers are encouraged to test the proposed propositions further.

The practical contribution, on the other side, provides scholars with the knowledge of how making their work more accessible, increasing their impact.

The authors add to the originality by providing a framework of the relevant literature for bibliometrics and altmetrics for future researchers. The authors describe altmetrics and present the differences with bibliometrics. The authors conclude the research with the implications of the conducted analysis and the potential directions for future research. Regarding manipulation, the authors provide with the techniques so researchers are aware of the methods in order to protect their academic profile.