Search results

Organizations often introduce performance measurement systems (PMSs) in order to evaluate the level of their performance, make comparison with competitors, and plan their future activities. Since indicators may affect the behaviour of the monitored system, the design and implementation of a PMS should always include the analysis of the impact it may exert on the organization itself. The aim of this paper is to suggest a methodology to evaluate this impact.

The proposed approach is based on an impact reference model derived from the balanced scorecard (BSC) framework. The different perspectives of the BSC are interpreted as areas of impact within an organization. Structured steps for impact evaluation are described and specific techniques of analysis are introduced.

A series of case studies, together with an analysis of advantages and disadvantages of the proposed method, are presented. Results show that, although many sets of indicators are usually able to meet the role of a PMS, they may exert a different impact on the context they are applied. The proposed methodology results to be a useful instrument for choosing the right set of indicators from the impact point of view. Finally, possible research paths to be undertaken for further developments of the proposed methodology are traced.

The application of the method is based on the assumption that managers charged with the analysis have a profound understanding of the specific contextual factors which may determine a reaction of the organization to a performance indicator or a PMS. Furthermore, at the moment, the methodology does not consider the possibility of interaction among different indicators in producing the impact.

This paper may be used to guide the selection of the most appropriate PMS from the impact point of view. The proposed methodology can be very helpful instrument for an organization involved in the design of new PMSs. It guides the decision maker through the various phases: indicators definition, analysis of their properties, impact analysis, and choice of the set with the preferable impact profile.

The issue of impact has been long debated in literature. Many articles try to analyse the operative and strategic consequences of the introduction of a PMS in an organization. This paper proposes a methodology for a more structured and objective evaluation of the impact of new PMS before introducing it in a firm. This can result in a significant help for manager who have to find the best set of indicators for the performance evaluation of their organization or have to choose between two or more sets of indicator satisfying, in principle, the same representation objective.

The purpose of this paper is to present the results of a study to examine, determine and propose the optimal approach to develop impact assessment indicators for the UK Web Archive.

The paper analyses the existing literature on impact assessment frameworks for digital resources and the types of impact in related fields to set an approach to develop an impact assessment plan. Primarily drawing from the Balanced Value Impact Model, the approach consists of three stages: context setting, indicator development and indicator evaluation.

The development of a set of potential impact assessment indicators for the UK Web Archive shows not only an optimal approach for the development but also recommendations for web archiving organisations.

The research did not carry out follow-up interviews regarding the feedback from UK Web Archive's staff. Adoption of the new set of indicators will further this development.

The staff's duties influence their prioritisation of the indicators, so discussions among partners will be helpful in recognising different perceptions, unnoticed strengths and potential values. A progressive accumulation of assessment and improvements from the current state and small regular evaluations will be also helpful to demonstrate the impact and value to the stakeholders in the future.

This paper proposes a set of 13 potential indicators for the UK Web Archive of which functionality was checked against set quality criteria and tested through semi-structured interviews and survey submissions with the UK Web Archive staff members.

Poor maintenance management leads to non-negligible economic, environmental and social impacts and obstacles to the sustainable manufacturing paradigm. Studies evaluating maintenance impacts on sustainability underline growing interest in the topic, but reports on the industrial field are lacking. Therefore, this paper investigates the industrial environment and the indicators that manufacturing companies use for measuring their maintenance impacts.

In this pilot survey study, several stakeholders of production enterprises in the south of Italy were interviewed to unveil the spread of the measurement of maintenance impacts on sustainability and the indicators used by those companies.

The interview results showed a low level of awareness among stakeholders about maintenance impacts on sustainability. Maintenance stakeholders are mainly focused on technical and economic factors, whereas environmental, quality and safety stakeholders are becoming more aware of maintenance impacts on environmental and social factors. However, both groups need guidelines to define sustainability indicators to assess such impacts.

This exploratory study allowed us to investigate the current situation in industrial organisations and achieve the first variegated and diversified vision of the awareness of company stakeholders on maintenance impacts on the sustainability of several business functions. This paper provides a valuable contribution to “maintenance and sustainability” research area in production contexts and sheds light on non-negligible maintenance impacts on sustainability, providing preliminary insights on the topic and an effective basis for defining future research opportunities. Moreover, this study enables increased awareness among internal and external manufacturing company stakeholders on the role of maintenance in sustainable production.

The purpose of this paper is to introduce altmetric indicators and combine with traditional citation indicators to comprehensively evaluate the impact of academic journals from the perspective of multidimensional and multi-indicator fusion.

The authors take international multidisciplinary journals as an example, combining 14 traditional citation indicators of academic journals and introducing 14 altmetric indicators to build a comprehensive evaluation model of the impact of academic journals (academic impact and societal impact). At the same time, the authors systematically construct a journal evaluation indicator system from three dimensions. Then, the indicators data of three dimensions are evaluated by normalized processing, correlation analysis, reliability and validity analysis, PCA and factor analysis.

Two-dimensional and three-dimensional analyses can exactly provide some useful information for academic journals’ location in the respective coordinate systems. There are strong positive correlations among the measured indicators in the three dimensions, and each indicator has a significant consistency between whole and internal. The correlation coefficient between FD1 and FD2 is 0.888 with a strong positive correlation. It shows that the traditional citation indicators provided by WoS and Scopus database are highly consistent, and they are comparable and alternative in evaluating the academic impact of journals. The correlation coefficients of FD1, FD2 with FD3 are 0.831 and 0.798. There are strong positive correlations among them, which indicate that the evaluation of journals’ societal impact based on altmetrics indicator can be considered as a potential supplement to academic impact evaluation based on citation and to reflect the multidimensional nature of journals impact in an immediate way.

Multidimensional and multi-indicator perspective evaluation can provide references for the selection of impact evaluation indicators and model optimization of academic journals, and also provide new ideas for improving the status of the impact evaluation of academic journals.

The purpose of this paper is to identify the impact indicators of the 2010 Taipei International Flora Exposition from the perspective of residents and to investigate the effects of these impacts on the urban sustainability through the perceptions of Taipei residents after the exposition.

The authors conducted a literature review and consulted experts to identify the impact indicators of the exposition. This process resulted in 17 indicators of economic impact, 12 indicators of environmental impact, and 13 indicators of social impact, and the authors developed a 12-dimension/42-indicator mega-event impact measurement scale. Using a survey of 1,628 host residents after the exposition, the authors performed regression analysis to investigate the effects of these impacts on the urban sustainability.

The results indicated that economic, environmental, and social impacts exerted significant positive effects on urban sustainability. All positive dimensions of economic, environmental, and social impact exerted significant positive effects on urban sustainability, and all negative dimensions exerted nonsignificant effects on urban sustainability.

This study identified the impact indicators of the 2010 Taipei International Flora Exposition and investigated the effects of these impacts on the urban sustainability. The findings of this study indicated that economic, environmental, and social impacts of the exposition exerted significant positive effects on urban sustainability.

The two most used citation impact indicators in the assessment of scientific journals are, nowadays, the impact factor and the h-index. However, both indicators are not field normalized (vary heavily depending on the scientific category). Furthermore, the impact factor is not robust to the presence of articles with a large number of citations, while the h-index depends on the journal size. These limitations are very important when comparing journals of different sizes and categories. The purpose of this paper is to propose an alternative citation impact indicator, based on the percentage of highly cited articles in the journal.

This alternative indicator is empirically compared with the impact factor and the h-index, considering different time windows and citation percentiles (levels of citation for considering an article as highly cited compared to others in the same year and category). The authors use four journal categories (Clarivate Analytics Web of Science) which are quite different according to the publication profiles and citation levels (Information Science & Library Science, Operations Research & Management Science, Ophthalmology, and Physics Condensed Matter).

After analyzing 20 different indicators, depending on the citation percentile and the time window in which citations are counted, the indicator that seems to best homogenize the categories is the one that considers a time window of two years and a citation level of 10 percent.

The percentage of highly cited articles in a journal is field normalized (comparable between scientific categories), independent of the journal size and also robust to the presence of articles with a high number of citations.

The purpose of this paper is to investigate the potential of altmetric and webometric indicators to aid with funding agencies’ evaluations of their funding schemes.

This paper analyses a range of altmetric and webometric indicators in terms of suitability for funding scheme evaluations, compares them to traditional indicators and reports some statistics derived from a pilot study with Wellcome Trust-associated publications.

Some alternative indicators have advantages to usefully complement scientometric data by reflecting a different type of impact or through being available before citation data.

The empirical part of the results is based on a single case study and does not give statistical evidence for the added value of any of the indicators.

A few selected alternative indicators can be used by funding agencies as part of their funding scheme evaluations if they are processed in ways that enable comparisons between data sets. Their evidence value is only weak, however.

This is the first analysis of altmetrics or webometrics from a funding scheme evaluation perspective.

This study aims to illustrate the impact-assessment procedure of low-carbon tourism (LCT) program implemented in a world heritage city and to develop specific indicators toward sustainability.

The impact-assessment framework was indicator-based and designed for creating sustainable tourism (ST) in a case study. A set of indicators in various dimensions was developed and applied, referring to the UNWTO guideline. A mixed method of primary and secondary data collected from different sources included document review, site observation, key informant interview, questionnaire survey and focus-group discussions. Assessment of actual/observed impacts was proceeded based on the data collected from tourists and stakeholders, especially on tourist behaviors and resource consumptions.

The implementation of LCT program in a world heritage city provided impacts in different dimensions and characters. The observed activities were majorly tourism activities in accommodations and recreational places. The indicator initiation is the first development toward sustainability in a case of tourism study in a city destination. Indicators were developed with participation from key stakeholders and covered sustainability and carbon-emission dimensions. Impact-assessment results show a positive theme in less carbon emission, enhanced local income distribution and community capacity. However, the negative impacts include increased amounts of resource consumption and waste generation in visiting sites. The impact matrix works as the map for decision-makers to maximize benefits and manage the cons of the LCT program toward ST principles.

Research methodology, procedure and results on impact assessment with holistic perspectives imply academic contribution and practical benefits for decision-makers regarding ST development. The number of samples and enterprises was limited because of the program implementation period.

The research illustrates the impact-assessment process for an implemented city-based LCT program toward ST, where stakeholder participation was also functioning. A list of indicators was specially designed and can be practically applied for other LCT programs in city destinations. Applying a sustainability impact-assessment framework to the program can provide a clear presentation on how to develop ST.

Despite the general recommendation of using a combination of multiple criteria for research assessment and faculty promotion decisions, the raise of quantitative indicators is generating an emerging trend in Business Schools to use single journal impact factors (IFs) as key (unique) drivers for those relevant school decisions. This paper aims to investigate the effects of using single Web of Science (WoS)-based journal impact metrics when assessing research from two related disciplines: Business and Economics, and its potential impact for the strategic sustainability of a Business School.

This study collected impact indicators data for Business and Economics journals from the Clarivate Web of Science database. We concentrated on the IF indicators, the Eigenfactor and the article influence score (AIS). This study examined the correlations between these indicators and then ranked disciplines and journals using these different impact metrics.

Consistent with previous findings, this study finds positive correlations among these metrics. Then this study ranks the disciplines and journals using each impact metric, finding relevant and substantial differences, depending on the metric used. It is found that using AIS instead of the IF raises the relative ranking of Economics, while Business remains basically with the same rank.

This study contributes to the research assessment literature by adding substantial evidence that given the sensitivity of journal rankings to particular indicators, the selection of a single impact metric for assessing research and hiring/promotion and tenure decisions is risky and too simplistic. This research shows that biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and traditions.

Consistent with the literature, given the sensibility of journal rankings to particular indicators, the selection of a single impact metric for assessing research, assigning research funds and hiring/promotion and tenure decisions is risky and simplistic. However, this research shows that risks and biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and trajectories. The use of multiple criteria is advised for such purposes.

This is an applied work using real data from WoS that addresses a practical case of comparing the use of different journal IFs to rank-related disciplines like Business and Economics, with important implications for faculty tenure and promotion committees and for research funds granting institutions and decision-makers.

This paper aims to propose a tool to assess local public expenditure effectiveness based on a framework of alignment between outputs, outcomes and impacts – the Index of Municipal Expenditure Effectiveness (IMEE). This index is composed of a set of indicators associated with the typology of local expenditure.

The paper describes the methodological approach used in the development of the Index, considering the insights from the literature review and the opinion of a panel of experts. The indicators of outcomes and social impacts that are part of the Index are intentionally aligned with the UN Sustainable Development Goals (SDGs) as they provide an essential guide to assess public value creation in the current context. For simplicity purposes, three main components of municipal expenditure were considered, namely Education, Essential Public Services and Local Public Transportation. The Index is then illustrated through a pilot application, using data from five Portuguese municipalities.

This study argues that measuring the public expenditure effectiveness based on outcome and impact indicators can provide the data needed for local governments to better understand the effects of their activities over time, ultimately assessing their contribution to public value.

Assessing the impact of local spending is important to ensure the best use of public resources. Linking local public expenditures with the 2030 Agenda and the SDGs is a promising avenue to understand up to what extent the application of the public money is contributing to create public value by impacting on citizens’ lives.

The proposed IMEE contributes to addressing a gap identified in public organizations, including local governments, regarding the lack of consideration of expenditure and outcome/impact relationships, and the use of variables to measure long term impacts.