Search results

Information and communications technology (ICT) offers enormous opportunities for individuals, businesses and society. The application of ICT is equally important to economic and non-economic activities. Researchers have increasingly focused on the adoption and use of ICT by small and medium enterprises (SMEs) as the economic development of a country is largely dependent on them. Following the success of ICT utilisation in SMEs in developed countries, many developing countries are looking to utilise the potential of the technology to develop SMEs. Past studies have shown that the contribution of ICT to the performance of SMEs is not clear and certain. Thus, it is crucial to determine the effectiveness of ICT in generating firm performance since this has implications for SMEs’ expenditure on the technology. This research examines the diffusion of ICT among SMEs with respect to the typical stages from innovation adoption to post-adoption, by analysing the actual usage of ICT and value creation. The mediating effects of integration and utilisation on SME performance are also studied. Grounded in the innovation diffusion literature, institutional theory and resource-based theory, this study has developed a comprehensive integrated research model focused on the research objectives. Following a positivist research paradigm, this study employs a mixed-method research approach. A preliminary conceptual framework is developed through an extensive literature review and is refined by results from an in-depth field study. During the field study, a total of 11 SME owners or decision-makers were interviewed. The recorded interviews were transcribed and analysed using NVivo 10 to refine the model to develop the research hypotheses. The final research model is composed of 30 first-order and five higher-order constructs which involve both reflective and formative measures. Partial least squares-based structural equation modelling (PLS-SEM) is employed to test the theoretical model with a cross-sectional data set of 282 SMEs in Bangladesh. Survey data were collected using a structured questionnaire issued to SMEs selected by applying a stratified random sampling technique. The structural equation modelling utilises a two-step procedure of data analysis. Prior to estimating the structural model, the measurement model is examined for construct validity of the study variables (i.e. convergent and discriminant validity).

The estimates show cognitive evaluation as an important antecedent for expectation which is shaped primarily by the entrepreneurs’ beliefs (perception) and also influenced by the owners’ innovativeness and culture. Culture further influences expectation. The study finds that facilitating condition, environmental pressure and country readiness are important antecedents of expectation and ICT use. The results also reveal that integration and the degree of ICT utilisation significantly affect SMEs’ performance. Surprisingly, the findings do not reveal any significant impact of ICT usage on performance which apparently suggests the possibility of the ICT productivity paradox. However, the analysis finally proves the non-existence of the paradox by demonstrating the mediating role of ICT integration and degree of utilisation explain the influence of information technology (IT) usage on firm performance which is consistent with the resource-based theory. The results suggest that the use of ICT can enhance SMEs’ performance if the technology is integrated and properly utilised. SME owners or managers, interested stakeholders and policy makers may follow the study’s outcomes and focus on ICT integration and degree of utilisation with a view to attaining superior organisational performance.

This study urges concerned business enterprises and government to look at the environmental and cultural factors with a view to achieving ICT usage success in terms of enhanced firm performance. In particular, improving organisational practices and procedures by eliminating the traditional power distance inside organisations and implementing necessary rules and regulations are important actions for managing environmental and cultural uncertainties. The application of a Bengali user interface may help to ensure the productivity of ICT use by SMEs in Bangladesh. Establishing a favourable national technology infrastructure and legal environment may contribute positively to improving the overall situation. This study also suggests some changes and modifications in the country’s existing policies and strategies. The government and policy makers should undertake mass promotional programs to disseminate information about the various uses of computers and their contribution in developing better organisational performance. Organising specialised training programs for SME capacity building may succeed in attaining the motivation for SMEs to use ICT. Ensuring easy access to the technology by providing loans, grants and subsidies is important. Various stakeholders, partners and related organisations should come forward to support government policies and priorities in order to ensure the productive use of ICT among SMEs which finally will help to foster Bangladesh’s economic development.

High or low quality is as much a result of how care systems are organised as it is a result of individual clinicians’ performance. Failure to introduce new care organisation or quality methods which research shows to be effective is as serious an omission as failing to act on poor clinical performance. Managers make many decisions about policies and organisation which affect the quality of care, but they rarely use evaluation research in making these decisions. Such research is difficult to find, produced using many different types of methods which are difficult for non‐experts to assess, often of poor quality, and difficult to translate to the local setting. However, managers can develop an evaluation informed practice, and make greater use of evaluation research in decisions with high cost or risk implications. The paper explains why the model of evidence‐based medicine is not appropriate, proposes instead a practical four‐step approach, and shows how managers can use evaluation in everyday practice.

This paper aims to address the reporting of validation and evaluation criteria in qualitative management accounting studies, which is a topic of critical debate in qualitative social science research. The objective of this study is to investigate the ways researchers have reported the use of evaluation criteria in qualitative management accounting studies and whether they are associated with certain paradigmatic affiliations.

Building on the work of Eriksson and Kovalainen [Eriksson and Kovalainen (2008) Qualitative Methods in Business Research. London, Sage], the following three approaches are examined: the adoption of classic concepts of validity and reliability, the use of alternative concepts and the abandonment of general evaluation criteria. Content analysis of 212 case and field studies published during 2006 to February 2015 was conducted to be able to offer an analysis of the most recent frontiers of knowledge.

The key empirical results of this study provide partial support for the theoretical expectations. They specify and refine Eriksson and Kovalainen’s (2008) classification system, first, by identifying a new approach to evaluation and validation and, second, by showing mixed results on the paradigmatic consistency in the use of evaluation criteria.

This paper is not necessarily exhaustive or representative of all the evaluation criteria developed; the authors focused on the explicit reporting of criteria only and the findings cannot be generalized. Somewhat different results might have been obtained if other journals, other fields of research or a longer period were considered.

The findings of this study enhance the knowledge of alternative approaches and criteria to validation and evaluation. The findings can aid both in the evaluation of management accounting research and in the selection of appropriate evaluation approaches and criteria.

This paper presents a synthesis of the literature (Table I) and new empirical findings that are potentially useful for both academic scholars and practitioners.

This paper aims to propose a framework for the evaluation of artefacts in Design Science and test it using an exemplar case of a knowledge management system (KMS) developed for an Australian refrigeration manufacturing company.

The research uses Design Science research methodology in a specific case study context. The artefact studied was developed using an ontology based on an engineering design conceptualisation and created using an ontology generator, Protégé. Research data for the evaluation of the framework were collected using a combination of document analysis, interviews, shadowing and observations.

The evaluation framework developed for the research and applied to the KMS specifically built for the company was shown to be useful in determining the efficacy and effectiveness of the research outcomes in terms of usefulness to the company engineers in the technical analysis of their work, and for the CEO and COO as part of their strategic planning for the company. The evaluation framework helped the researcher and the engineers as collaborators to demonstrate the extent of improvement in the design and build processes in the company.

Prior research in both Information System and Design Science has not provided a specific, generalizable, evaluation framework for system developers to use as a guide during the systems development process. This research proposes an evaluation framework which covers all broad aspects of evaluation and efficacy, accepting that evaluation frameworks must be flexible in enabling changes to accommodate variations in the types and purposes of artefacts developed.

The purpose of this study is to explore informetrics researchers' use of social media for academic activities, their attitudes to the applicability of altmetrics in research evaluation, the factors influencing their attitudes, and the main opportunities and weaknesses of using altmetrics.

A survey using a questionnaire was conducted with researchers who participated in the 16th International Conference on Scientometrics and Informetrics ISSI 2017 and a sample of 125 respondents was obtained.

Progressively more researchers are using social media for different types of academic activities. The study found that many factors affect informetrics researchers' attitudes in different application scenarios with respect to research evaluation. Researchers who have studied altmetrics and who began using social media platforms recently or frequently have more positive attitudes. Academic users and social users have statistically significantly disparate attitudes toward altmetrics in different disciplines and different application scenarios.

Our study only focused on 125 informetrics researchers, who participated in ISSI 2017. We mainly used the questionnaire method, but did not conduct in-depth interviews with the researcher's views.

Informetrics researchers are participants in social media and major researchers of altmetrics. Previous research has examined their use of social media, and this study combines this use of social media with their attitudes to altmetrics to explore the value of altmetrics from a particular perspective. The paper also provides suggestions for the application of altmetrics in research evaluation.

The purpose of this paper is to contribute to the body of knowledge on how research evaluation in different national and organisational contexts affects, often in unintended ways, research and publication practices. In particular, it looks at the development of book publication in the social sciences and humanities (SSH) in the Czech Republic since 2004, when a performance-based system of evaluation was introduced, up to the present.

The paper builds upon ethnographic research complemented by the analysis of Czech science policy documents, data available in the governmental database “Information Register of R&D results” and formal and informal interviews with expert evaluators and other stakeholders in the research system. It further draws on the authors’ own experience as scholars, who have also over the years participated in a number of evaluation procedures as peers and experts.

The number of books published by researchers in SSH based at Czech institutions has risen considerably in reaction to the pressure for productivity that is inscribed into the evaluation methodology and has resulted in the rise of in-house publishing by researchers’ own research institution, “fake internationalisation” using foreign low-quality presses as the publication venue, and the development of a culture of orphaned books that have no readers.

In the Czech Republic robust and internationally harmonised bibliometric data regarding books would definitely help to create a form of research evaluation that would stimulate meaningful scholarly book production. At the same time, better-resourced and better-designed peer review evaluation is needed.

This is the first attempt to analyse in detail the conditions and consequences the Czech performance-based research evaluation system has for SSH book publication. The paper demonstrates that often discussed harming of SSH and book-writing in particular by performance-based IF-centred research evaluation does not necessarily manifest in declining numbers of publications. On the contrary, the number of books published may increase at the cost of producing more texts of questionable scholarly quality.

Calls for the development and dissemination of evidence-based programs to support children and families have been increasing for decades, but progress has been slow. This paper aims to argue that a singular focus on evaluation has limited the ways in which science and research is incorporated into program development, and advocate instead for the use of a new concept, “scientific accompaniment,” to expand and guide program development and testing.

A heuristic is provided to guide research–practice teams in assessing the program’s developmental stage and level of evidence.

In an idealized pathway, scientific accompaniment begins early in program development, with ongoing input from both practitioners and researchers, resulting in programs that are both effective and scalable. The heuristic also provides guidance for how to “catch up” on evidence when program development and science utilization are out of sync.

While implementation models provide ideas on improving the use of evidence-based practices, social service programs suffer from a significant lack of research and evaluation. Evaluation resources are typically not used by social service program developers and collaboration with researchers happens late in program development, if at all. There are few resources or models that encourage and guide the use of science and evaluation across program development.

Given the magnitude of the literature, a researcher must be selective of research papers and publications in general. In other words, only papers that meet strict standards of academic integrity and adhere to reliable and credible sources should be referenced. The purpose of this paper is to approach this issue from the prism of scientometrics according to the following research questions: Is it necessary to judge the quality of scientific production? How do we evaluate scientific production? What are the tools to be used in evaluation?

This paper presents a comparative study of scientometric evaluation practices and tools. A systematic literature review is conducted based on articles published in the field of scientometrics between 1951 and 2022. To analyze data, the authors performed three different aspects of analysis: usage analysis based on classification and comparison between the different scientific evaluation practices, type and level analysis based on classifying different scientometric indicators according to their types and application levels and similarity analysis based on studying the correlation between different quantitative metrics to identify similarity between them.

This comparative study leads to classify different scientific evaluation practices into externalist and internalist approaches. The authors categorized the different quantitative metrics according to their types (impact, production and composite indicators), their levels of application (micro, meso and macro) and their use (internalist and externalist). Moreover, the similarity analysis has revealed a high correlation between several scientometric indicators such as author h-index, author publications, citations and journal citations.

The interest in this study lies deeply in identifying the strengths and weaknesses of research groups and guides their actions. This evaluation contributes to the advancement of scientific research and to the motivation of researchers. Moreover, this paper can be applied as a complete in-depth guide to help new researchers select appropriate measurements to evaluate scientific production. The selection of evaluation measures is made according to their types, usage and levels of application. Furthermore, our analysis shows the similarity between the different indicators which can limit the overuse of similar measures.

A meta-evaluation is an assessment of evaluation practices. Meta-evaluations include assessments of validity and usefulness of two or more studies that focus on the same issues. Every performance audit is grounded explicitly or implicitly in one or more theories of program evaluation. A deep understanding of alternative theories of program evaluation is helpful to gain clarity about sound auditing practices. We present a review of several theories of program evaluation.

This study includes a meta-evaluation of seven government audits on the efficiency and effectiveness of tourism departments and programs. The seven tourism-marketing performance audits are program evaluations for: Missouri, North Carolina, Tennessee, Minnesota, Australia, and two for Hawaii. The majority of these audits are negative performance assessments. Similarly, although these audits are more useful than none at all, the central conclusion of the meta-evaluation is that most of these audit reports are inadequate assessments. These audits are too limited in the issues examined; not sufficiently grounded in relevant evaluation theory and practice; and fail to include recommendations, that if implemented, would result in substantial increases in performance.

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish higher education, with particular reference to the longitudinal evaluative research of support of teaching and learning (T&L) (the Quality Enhancement Framework or QEF). The chapter will discuss the features of the research process which are shaped by evaluation theory. We adopt a theoretical position on policy research which foregrounds the situated experience of policy as a core research focus. Policy is depicted as being underscored by an implicit theory of change which is used to structure and orientate the research focus. The design of the research is characterised by the involvement of potential users of the research output, with implications on the way in which findings are articulated, presented and ultimately used, along with aspects of the evaluative research design. The case study of the QEF will be contextualised, and the intersection between the design features and theoretical approaches, and the use and usability of research outputs, will be established.