Search results

Librarians are challenged by technology transformations in research data metrics to re-position in the evolving cycles of research production, communication and evaluation. They are also are challenged by the new and emerging strategic needs of universities for data-driven research intelligence that provides a comparative edge in the global world of higher education. This paper aims to discover how libraries can support universities in applying international assessment standards by delivering new reference and information services based on data analytics.

This paper focuses on a new reference and information service Research Data Analytics in China that was launched in 2017. There is a discussion of how new university needs have provided the strategic drive for librarians to develop skills.

The Research Data Analytics service development is significant for showcasing a new role for librarians in relation to research data evaluation metrics linked to publication strategies for authors and strategic intelligence for institutions. This has driven the development of the library’s position and influence in strategic research intelligence services.

The future for librarians is about being adventurous. Librarians are experts who will sparkle and shine beyond the bubble of the library walls. They can demonstrate a commitment to supporting university colleagues in professional services and academia to blossom and flourish. Distinctive visual technologies can be adopted for exploring research data which transform research production, communication and evaluation and therefore transform our position and future through technology-enabled innovations.

Any business that opts to adopt a recommender engine (RE) for various potential benefits must choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning (ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and establishing a reliable relation with customers and undeniably to grow the business.

An experimental quantitative research method was conducted in which the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-real-time data set to uncover the best-fitting RE.

The rule-based automated evaluation of RE has changed the testing landscape, with the removal of longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with high-quality results and can possibly bring a new revolution in the testing practice to start a service line “Machine Learning Testing as a service” (MLTaaS) and the possibility of integrating with DevOps that can specifically help agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.

A small data set was considered for A/B phase study and was captured for ten movies from three theaters operating in a single location in India, and simulation phase study was captured for two movies from three theaters operating from the same location in India. The research was limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.

The best-fitting RE facilitates the business to make personalized recommendations, long-term customer loyalty forecasting, predicting the company's future performance, introducing customers to new products/services and shaping customer's future preferences and behaviors.

The proposed rule-based ML approach named “2-stage locking evaluation” is self-learned, automated by design and largely produces time-bound conclusive result and improved decision-making process. It is the first of a kind to examine the business domain and task of interest. In each stage of the evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution. Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with end-to-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by identifying the missing test metrics coverage.

The purpose of this paper is to develop a nuanced framework for evaluating a channel partner’s performance in distribution channel relationships. Given a channel partner’s task environment characteristics (high/low munificence, dynamism and complexity), the study examines which performance metrics (output, activity or capability) are most relevant for evaluating its performance levels effectively.

The study adopts self-administered cross-sectional survey-based research design. Matched data were collected from 252 channel partners – manager relationship dyads. The latent change score (LCS) model within SEM framework provides mean paired-differences of the relevance ratings for each metrics. This was used to assess the empirical validity of the hypothesized relationships.

The study demonstrates the importance of calibrating performance evaluation metrics to a channel partner’s task environment state, made possible by its holistic approach to performance evaluation. Based on an extensive analysis, it shows that no single metric is relevant within all environmental states; rather, it could be dysfunctional, a result that differs from vast majority of the literature.

Investigates individual linkages between task environment dimensions and performance metrics to provide a fuller understanding of these relationships. Also provides a theoretical framework to support further research on the topic.

The study provides managerial guidelines (and extensive graphical analysis) for nuanced and dynamic evaluation of channel partners’ performance that can enable firms to identify and promote their most valuable channel partners and prevent the deterioration of others.

First one to develop and empirically validate a nuanced framework for evaluating performance of exchange partners that operate under diverse task environment states.

The purpose of this paper is to provide a critical discussion of the Leiden Manifesto for libraries already engaged in bibliometric practices. It offers practical recommendations based on the work of the European Association for Research Libraries (LIBER) Working Group on Metrics. This work is in the beginning phase and summarizes literature on the topic, as well as the experiences of the members of the Working Group. The discussion reflects today's growing popularity of (quantitative) research assessment which is seen in enthusiasts introducing new metrics (i.e. altmetrics) and by critics demanding responsible metrics that increase objectivity and equity in evaluations.

This paper is the result of the Working Group on Metrics of the European Association for Research Libraries (LIBER) that critically discussed the practicality of the Leiden Manifesto for libraries.

Full compliance with the Manifesto is time-consuming, expensive and requires a significant increase in bibliometric expertise with respect to both staffing and skill level. Despite these apparent disadvantages, it is recommended that all libraries embrace the Manifesto’s principles. To increase practicality, it is advised that libraries collaborate with researchers, management and other libraries at home and around the world to jointly design and provide services that can be reused within the library community.

Libraries have increasingly been confronted with questions about research assessment, responsible metrics and the role of digital products in evaluations and funding decisions. Although a wide range of recommendations and initiatives are available (e.g. DORA San Francisco Declaration on Research Assessment), many recommendations are not straightforward enough to be implemented from a library perspective. This paper provides assistance for libraries to implement these principles by acknowledging the heterogeneous backgrounds the libraries may stem from.

This study investigates an approach to book metrics for research evaluation that takes into account the complexity of scholarly monographs. This approach is based on work sets – unique scholarly works and their within-work related bibliographic entities – for scholarly monographs in national databases for research output.

This study examines bibliographic records on scholarly monographs acquired from four European databases (VABB in Flanders, Belgium; CROSBI in Croatia; CRISTIN in Norway; COBISS in Slovenia). Following a data enrichment process using metadata from OCLC WorldCat and Amazon Goodreads, the authors identify work sets and the corresponding ISBNs. Next, on the basis of the number of ISBNs per work set and the presence in WorldCat, they design a typology of scholarly monographs: Globally visible single-expression works, Globally visible multi-expression works, Miscellaneous and Globally invisible works.

The findings show that the concept “work set” and the proposed typology can aid the identification of influential scholarly monographs in the social sciences and humanities (i.e. the Globally visible multi-expression works).

In light of the findings, the authors outline requirements for the bibliographic control of scholarly monographs in national databases for research output that facilitate the use of the approach proposed here.

The authors use insights from library and information science (LIS) to construct complexity-sensitive book metrics. In doing so, the authors, on the one hand, propose a solution to a problem in research evaluation and, on the other hand, bring to attention the need for a dialogue between LIS and neighbouring communities that work with bibliographic data.

Much has been analyzed regarding the origins and the impact of rankings and metrics on policies, behaviors, and missions of universities. Surprisingly, little attention has been allocated to describing and analyzing the emergence of metrics as a new action field. This industry, fueled by the “new public management” policy perspectives that operate at the backstage of the contemporary pervasive “regime of excellence,” still remains a black box worth exploring in depth. This paper intends to fill this loophole. It first sets the stage for this new action field by stressing the differences between the policy fields of higher education in the United States and Europe, as a way to understand the specificities of the use of metrics and rankings on both continents. The second part describes the actors of the field, which productive organizations they build, what skills they combine, which products they put on the market, and their shared norms and audiences.

The purpose of this paper is to make an explicit case for the use of data with contextual information as evidence in arts and humanities research evaluations rather than systematic metrics.

A survey of the strengths and limitations of citation-based indicators is combined with evidence about existing uses of wider impact data in the arts and humanities, with particular reference to the 2014 UK Research Excellence Framework.

Data are already used as impact evidence in the arts and humanities but this practice should become more widespread.

Arts and humanities researchers should be encouraged to think creatively about the kinds of data that they may be able to generate in support of the value of their research and should not rely upon standardised metrics.

This paper combines practices emerging in the arts and humanities with research evaluation from a scientometric perspective to generate new recommendations.

This paper aims to understand the nature of citations and metrics in the larger system of knowledge production involving universities, funding agencies, publishers, and indexing and data analytic services.

First, the normative and social constructivist views of citations are reviewed to be understood as co-existing conditions. Second, metrics are examined through the processes of commensuration by tracing the meanings of metrics embedded in various kinds of documents and contexts. Third, the steering effects of citations and metrics on knowledge production are discussed. Finally, the conclusion addresses questions pertaining to the validity and legitimacy of citations as data and their implications for knowledge production and the conception of information.

The normative view of citations is understood as an ideal speech situation; the social constructivist view of citation is recognised in the system of knowledge production where citing motivations are influenced by epistemic, social and political factors. When organisational performances are prioritised and generate system imperatives, motives of competition become dominant in shaping citing behaviour, which can deviate from the norms and values in the academic lifeworld. As a result, citations and metrics become a non-linguistic steering medium rather than evidence of research quality and impact.

This paper contributes to the understanding of the nature of citations and metrics and their implications for the conception of information and knowledge production.

Currently, countries worldwide are struggling with the virus COVID-19 and the severe outbreak it brings. To better benefit from open government health data in the fight against this pandemic, this study developed a framework for assessing open government health data at the dataset level, providing a tool to evaluate current open government health data's quality and usability COVID-19.

Based on the review of the existing quality evaluation methods of open government data, the evaluation metrics and their weights were determined by 15 experts in health through the Delphi method and analytic hierarchy process. The authors tested the framework's applicability using open government health data related to COVID-19 in the US, EU and China.

The results of the test capture the quality difference of the current open government health data. At present, the open government health data in the US, EU and China lacks the necessary metadata. Besides, the number, richness of content and timeliness of open datasets need to be improved.

Unlike the existing open government data quality measurement, this study proposes a more targeted open government data quality evaluation framework that measures open government health data quality on a range of data quality dimensions with a fine-grained measurement approach. This provides a tool for accurate assessment of public health data for correct decision-making and assessment during a pandemic.

Hicks et al. (2015) have formulated the so-called Leiden manifesto, in which they have assembled the ten principles for a meaningful evaluation of research on the basis of bibliometric data. The paper aims to discuss this issue.

In this work the attempt is made to indicate the relevance of the Leiden manifesto for altmetrics.

As shown by the discussion of the ten principles against the background of the knowledge about and the research into altmetrics, the principles also have a great importance for altmetrics and should be taken into account in their application.

Altmetrics is already frequently used in the area of research evaluation. Thus, it is important that the user of altmetrics data knows the relevance of the Leiden manifesto also in this area.