Search results

Pay, tenure and promotion decisions are frequently based upon inferences regarding the value of faculty research. Meanwhile, departmental, college and university reputations are frequently based on perceptions regarding the quality of research being produced by its faculty. Making correct inferences requires accurate measurement of research quality, which is often based upon the journal through which results are shared. This research expands upon the research found elsewhere through its detailed investigation of leading journals in two business disciplines, including examination of four different citation-based measures and four journal characteristics which are exogenous to the quality of any individual piece of research. The paper aims to discuss this issue.

This study assists in the development of an accurate perspective regarding research quality, by studying the popular Journal Citation Reports (JCR) impact factor. A further expansion on the past literature is consideration of three newer journal quality metrics: SCImago Journal Rank (SJR), Source Normalized Impact per Paper (SNIP) and percentage of articles cited. Top-tier journals in finance and information systems are compared to evaluate the consistency of these measures across disciplines. Differences in journal characteristics and their impact on citation-rate based measures of quality are also examined. The potential impact of discipline-based variation in acceptance rate, issue frequency, the time since journal inception and total reviewers are put forth as additional potential exogenous factors that may impact the perception of journal quality. t-Tests are employed for discipline comparisons, while correlation and multiple regression are used for journal characteristic analysis.

There is a significant difference in the JCR impact measures of high-quality finance journals vs high-quality information systems journals, which are correlated with a variety of journal-specific factors including the journal’s acceptance rate and frequency of issue. Information systems journals domination of finance journals persists whether one considers mean, median, minimum or maximum impact factors. SJR measures for finance journals are consistently higher than information systems journals, though the SJR value of any individual journal can be quite volatile. By comparison, the SNIP metric rates premier information systems journals higher. Over 12 percent more of the articles in leading information systems journals are cited during the initial three years.

Logical extensions of this research include examining journals in other business disciplines. One could also evaluate quality measures reaction to variation in journal characteristics (i.e. changes in acceptance rates). Furthermore, one could include other measures of journal quality, including the recently released CiteScore metric. Such research will build on the present research and improve the accuracy of research quality assessment.

To the extent that citation-based research measures and journal-specific factors vary across disciplines as demonstrated by our investigation, discipline-specific traits should be considered adjusted for, when making inferences about the long-term value of recently published research. For instance, finance faculty publishing in journals with JCR readings of 2.0 are in journals that are 53 percent above the discipline’s average, while information systems faculty publishing in journals with JCR readings of 2.0 are in journals that are 18 percent below the discipline’s average. Furthermore, discipline-specific differences in journal characteristics, leading to differences in citation-based quality measures, should be considered when making inferences about the long-term value of recently published research in the process of making recommendations regarding salary adjustments, retention and promotion.

Quantity and quality of research are two hallmarks of leading research institutions. Assessing research quality is very problematic because its definition has changed from being based on review process (i.e. blind refereed), to acceptance rates, to impact factors. Furthermore, the impact factor construct has been a lightning rod of controversy as researchers, administrators and journals themselves argue over which metric to employ. This research is attempting to assess how impact factors and journal characteristics may influence the impact factors, and how these interactions vary business discipline. The research is especially important and relevant to the authors which separately chair departments including finance and information systems faculty, and therefore are in roles requiring assessment of faculty research productivity including quality.

This study is a detailed analysis of bibliographic aspects of the top-tier journals in two quantitative business areas. In addition to the popular JCR, SJR and SNIP measures of performance, the analysis studies the seldom-examined percentage of the article cited metric. A deeper understanding of citation-based measures is obtained though the evaluation of changes in how journals have been rated on these metrics over time. The research shows that there are discipline-related systematic differences in both citation-based research measures and journal-specific factors and that these discipline-specific traits should be considered when making inferences about the long-term value of recently published research. Furthermore, discipline-specific difference in journal characteristics, leading to differences in citation-based quality measures, should be considered when making personnel and remuneration decisions.

The aim of this study is to explore drivers of corporate community involvement (CCI) initiatives and the challenges faced by companies in measuring the social impact of their initiatives in Australia.

The authors conducted semi-structured interviews with various corporate social responsibility (CSR) or CCI managers from Australian companies and their not-for-profit (NFP) partners. The final sample consists of 27 managers from a mix of industries.

The study shows that stakeholder's salience may have an impact on CCI activities, especially in the area of measurements and reporting activities. Moreover, while some companies have attempted to measure the social impact of their initiatives, a large number of companies have not. This is all the more surprising given the recent focus in marketing on accountability and measurement. The results show three challenges: lack of interest, lack of resources and lack of consensus. Subsequently, the authors offer some research propositions to underline these challenges.

This study focuses on CCI which is one of the most visible parts of corporate social responsibility (CSR). It draws on interviews with various managers in charge of companies' CSR or CCI.

This study aims to extend existing research on impact measurement (IM) in social enterprises (SEs) by capturing, comparing and contrasting perceptions of IM in SEs in Australia and India.

A qualitative methodology was used to study five cases each in India and Australia. The SEs were identified using snowball and theoretical sampling, and grounded theory was applied to analyze the data.

Emerging perceptions of IM in both countries are described according to the development of the SE, its perceived impact and IM methods and challenges. Primary differences between India and Australia lie in perceptions of impact and IM, and related tools and processes. Similarities include understanding the importance of IM and the challenges faced. Signaling theory is used to depict how some SEs use IM to signal quality to their stakeholders and how information asymmetry can be reduced by measuring and reporting on IM.

There is limited representation from developed and developing countries, and the snowball and theoretical sampling approaches used to identify SEs have limitations, including limited representation of SEs.

There is presently no standardized method of IM due to common challenges and perceived barriers. It is, therefore, important for SEs to work toward developing their own comprehensive IM methodology that is ingrained in strategy, applied on a regular basis and used to measure collective impact to increase sense of ownership and acceptability for employees and partners.

The paper brings the social entrepreneurs’ perspectives on measuring social impact while comparing these perspectives in one developing and one developed country.

The purpose of this research paper is to identify the non-financial e-procurement performance measures and find out whether these non-financial performance measures are leading indicator of impact on firm financial performance by adoption of e-procurement in terms of reduction in production cost.

The research model has been tested with the data collected from target procurement professionals in India. Structural equation modelling has been used for testing conceptual model hypotheses including mediation. The phantom model approach for testing multiple mediators has deployed.

The present empirical study found that non-financial performance measure of e-procurement, namely, transparency, coordination, efficiency and effectiveness are leading indicators of the impact of e-procurement adoption on production cost. This paper suggests that managers should try to design the e-procurement platform or opt for third party platform which reduces transaction cost to a minimum for enhanced coordination, work on transparency policy with maximum disclosure of information for enhanced transparency and ask for a fast and responsive system for enhanced efficiency and effectiveness.

This study, first time, attempted to identify non-financial performance measures of e-procurement and tried to understand how these intermediate non-financial performance measures impact the firm financial performance. The interdependence of non-financial performance measures has also been explored, and the research model has been developed to empirically examine the interdependence of these financial measures and its impact on production cost.

Earlier research highlights the need for the welfare service sector to measure the impacts of their services. However, it seems that the welfare services lack measures to show their long-term effects and impacts. This paper aims to present a framework to measure the multidimensional impacts of welfare service innovations and report the empirical results from two case studies.

In the first part of the paper, the impact measurement literature is reviewed and a framework for measuring the impacts of welfare services is presented. The empirical part of the paper reports the application of the framework in two cases for measuring the impacts of interventions in welfare services in Finland. The aim of the case studies was to assess and illustrate the usefulness of the framework designed.

The framework proposed in the research may serve as a practical tool for decision-makers for assessing the impacts of different services and service innovations in the welfare service sector. This type of assessment is needed, for example, when new service innovations are designed and budgeted for.

This research introduces a framework for measuring the impacts of welfare services at different levels. In addition, the paper provides information about the measurement process and challenges related to the implementation of impact measurement.

The need for a more structured methodology than currently exists for describing the impact of academic research is widely acknowledged. The most widely used research information standard, CERIF, does not currently allow the encoding of research impact in a structured way: this project devised and tested an extension to CERIF to address this omission. The paper seeks to discuss these points.

The core methodology of the project is a series of extensions to the CERIF model to encode “impact statements”, indicators of impact and measures as evidence for them. These can be linked to persons, organisational units or research outputs. This model is supported by a small semantic taxonomy of indicators and measures. The model was tested by evaluating it against current information environments, and by assessing its compatibility with CERIF and non‐CERIF compliant current research information systems.

Despite some concerns expressed about the validity of reducing qualitative evidence of impact to atomistic measures, and about a general paucity of such data in existing systems, the model tested well against working environments. It offers the potential for reducing workloads and more continuous assessment of research impact within its stakeholder communities.

No substantive methodology for encoding impact statements existed in CERIF prior to this project. In addition, the atomistic, quantifiable approach to describing impact is relatively unexplored in the higher education community, but offers substantial advantages. The work is of relevance to research managers, developers, system designers and metadata specialists.

There is currently a wide range of methods for measuring social impact. Each method uses specific indicators, mainly because of the diverse characteristics of social enterprises (SEs) and the type of impact that is analysed, thus hindering the definition of a single, shared measurement system and, at the same time, prompting the proliferation of countless alternative methods. Many enterprises experience difficulties in selecting the best method to carry out the measurement process correctly. The purpose of this paper is to contribute to filling in conceptual gaps inherent to measuring impact and value creating in the domain of social entrepreneurship (SE), as well as equipping the social entrepreneur with better knowledge of the methodologies available for measuring impact and supporting their decision-making process.

The aims of this paper are, therefore, threefold: to identify the common conditions of how to measure social impact (literature); to analyse how measurement is actually undertaken in practice (process); and to compare the four main methodologies, among the numerous ones, that have been developed to measure the impact generated by SEs so far (methods and comparison). The authors compared four of the most commonly used methodologies in the field of social impact measurement, analysing advantages, disadvantages and application fields. They evaluated whether a method can be considered preferable to others in each case.

The paper demonstrated the high fragmentation that characterised the existing literature concerning the measurement of social impact and the wide range of methodologies used, thus leading to a great confusion in regard to the selection of the most appropriate methodology for the pursuit of one's own ends. This often discourages the undertaking of the measurement process. The analysis used in this paper leads us to conclude that the social return on investment method is more popular than the other three alternatives.

There are significant deficiencies in methodologies adopted, and researchers must use innovative, situated approaches that fit with the SE literature. The authors concluded that for the future, there is a need to do a SLR in a disciplined way. Further research is strongly recommended in this area, to provide more comparative studies of existing methods. It is hoped that enterprises can be directed towards using a limited range of formal methods that can capture the diversity of the various application cases, thus making it possible to compare different situations: a limited range of formal methods that can capture the diversity of the SEs considered and the impacts generated will be promoted.

The authors also want to analyse how the SEs concretely realise the measurement of their impact that often do not use the formal methodologies presented in the literature but rather tools created by the ad hoc companies on the basis of their specific needs.

This paper makes a theoretical contribution to the literature of the theory on social value within the SE field by having regard to how to measure social impact. It partially responds to Choi and Majumdar’s (2014) and Hlady-Rispal and Servantie’s (2016) calls for the development of a theory of measuring social value.

The purpose of this paper is to introduce and evaluate methods for analysing the impacts of work environment changes. New working practices and work environments present the potential to improve both the productivity and the wellbeing of knowledge workers, and more widely, the performance of organisations and the wider society. The flexibility offered by information and communication technology has influenced changes in the physical environment where activity-based offices are becoming the standard. Research offers some evidence on the impacts of work environment changes, but studies examining methods that could be useful in capturing the overall impacts and how to measure them are lacking.

This paper concludes research of the last five years and includes data from several organisations. The paper presents and empirically demonstrates the application of three complementary ways to analyse the impacts of knowledge work redesigns. The methods include: interview framework for modelling the potential of new ways of working (NWoW); questionnaire tool for measuring the subjective knowledge work performance in the NWoW context; and multidimensional performance measurement for measuring the performance impacts at the organisational level.

This paper presents a framework for identifying the productivity potential and measuring the impacts of work environment changes. The paper introduces the empirical examples of three different methods for analysing the impacts of NWoW and discusses the usefulness and challenges of the methods. The results also support the idea of a measurement process and confirm that it suits NWoW context.

The three methods explored in this study can be used in organisations for planning and measuring work environment changes. The paper presents a comprehensive approach to work environment which could help managers to identify and improve the critical points of knowledge work.

Changes in the work environment are huge for knowledge workers, but it is still unclear whether their effects on performance are negative or positive. The value of this paper is that it applies traditional measurement methods to NWoW contexts, and analyses how these could be used in research and management.

A meta-evaluation is an assessment of evaluation practices. Meta-evaluations include assessments of validity and usefulness of two or more studies that focus on the same issues. Every performance audit is grounded explicitly or implicitly in one or more theories of program evaluation. A deep understanding of alternative theories of program evaluation is helpful to gain clarity about sound auditing practices. We present a review of several theories of program evaluation.

This study includes a meta-evaluation of seven government audits on the efficiency and effectiveness of tourism departments and programs. The seven tourism-marketing performance audits are program evaluations for: Missouri, North Carolina, Tennessee, Minnesota, Australia, and two for Hawaii. The majority of these audits are negative performance assessments. Similarly, although these audits are more useful than none at all, the central conclusion of the meta-evaluation is that most of these audit reports are inadequate assessments. These audits are too limited in the issues examined; not sufficiently grounded in relevant evaluation theory and practice; and fail to include recommendations, that if implemented, would result in substantial increases in performance.

The purpose of this paper is to examine the variability of the impact factor (IF) and additional metrics in biomedical journals to provide some clues to the reliability of journal citation indicators.

Having used ISI Journal Citation Reports, from 2005 to 2011, the authors extracted 62 subject categories related to biomedical sciences. The category lists and citation profile for each journal were then downloaded and extracted. Coefficient of variation was applied to estimate the overall variability of the journal citation indicators.

Total citation indicators for 3,411 journals were extracted and examined. The overall variability of IFs and other journal citation measures in basic, clinical or translational, open access or subscription journals decreased while the quality and prestige of those journals developed. Interestingly, journal citation measures produced dissimilar variability trends and thus highlighted the importance of using multiple instead of just one measure in evaluating the performance and influence of biomedical journals. Eigenfactor™, Article's Influence and Cited Half Life proposed as more reliable indicators.

The relative variability of the journal citation measures in biomedical journals would decrease with a development in the impact and quality of journals. Eigenfactor™ and Cited Half Life are suggested as more reliable measures indicating few changes during the study period and across different impact level journals. These findings will be useful for librarians, researchers and decision makers who need to use citation measures as evaluative tools.