Search results

Replication is a primary self-correction device in science. In this paper, we have two aims: to examine how and when the results of replications are used in management and organization research and to use the results of this examination to offer guidelines for improving the self-correction process.

Study 1 analyzes co-citation patterns for 135 original-replication pairs to assess the direct impact of replications, specifically examining how often and when a replication study is co-cited with its original. In Study 2, a similar design is employed to measure the indirect impact of replications by assessing how often and when a meta-analysis that includes a replication of the original study is co-cited with the original study.

Study 1 reveals, among other things, that a huge majority (92%) of sources that cite the original study fail to co-cite a replication study, thus calling into question the impact of replications in our field. Study 2 shows that the indirect impact of replications through meta-analyses is likewise minimal. However, our analyses also show that replications published in the same journal that carried the original study and authored by teams including the authors of the original study are more likely to be co-cited, and that articles in higher-ranking journals are more likely to co-cite replications.

We use our results to formulate recommendations that would streamline the self-correction process in management research at the author-, reviewer- and journal-level. Our recommendations would create incentives to make replication attempts more common, while also increasing the likelihood that these attempts are targeted at the most relevant original studies.

This paper replicates four highly cited, classic lab experimental studies in the provision of public goods. The studies consider the impact of marginal per capita return and group size; framing (as donating to or taking from the public good); the role of confusion in the public goods game; and the effectiveness of peer punishment. Considerable attention has focused recently on the problem of publication bias, selective reporting, and the importance of research transparency in social sciences. Replication is at the core of any scientific process and replication studies offer an opportunity to reevaluate, confirm or falsify previous findings. This paper illustrates the value of replication in experimental economics. The experiments were conducted as class projects for a PhD course in experimental economics, and follow exact instructions from the original studies and current standard protocols for lab experiments in economics. Most results show the same pattern as the original studies, but in all cases with smaller treatment effects and lower statistical significance, sometimes falling below accepted levels of significance. In addition, we document a “Texas effect,” with subjects consistently exhibiting higher levels of contributions and lower free-riding than in the original studies. This research offers new evidence on the attenuation effect in replications, well documented in other disciplines and from which experimental economics is not immune. It also opens the discussion over the influence of unobserved heterogeneity in institutional environments and subject pools that can affect lab results.

Purpose – The purpose of this chapter is to examine the frequency of multi-study research packages in the organizational sciences and advocate for their use by detailing strengths and recognizing limitations.

Methodology/approach – Philosophy of science research, focusing on multi-study research packages, is discussed followed by a 20-year review of incidence of these packages in top organizational sciences journals.

Findings – The publication of multi-study research packages have increased over the past 10 years, most notably in micro-level journals.

Social implications – For reasons of validity and generalizability, society benefits if scholars adopt multi-study approaches to knowledge generation and disseminate.

Originality/value of the chapter – This chapter provides the most comprehensive review of multiple-study research packages in the organizational sciences to date, examining publication trends in eight leading micro-and macro-level journals. We also summarize the use of multi-study packages in our own research and offer recommendations for improving the science of replication.

This study aims to explore the degree to which the editorial policies of business and management journals explicitly or implicitly discourage replication studies.

The paper examines differences in editorial policy toward replication studies relative to journal quality, age and sub-discipline area. A total of 600 journals (listed as Q1 and Q2 in Scopus) were selected for the current study.

The results reveal that out of 600 selected journals, only 28 (4.7%) were explicitly open to considering replication studies, while 331 (55.2%) were neutral, being neither explicitly nor implicitly dismissive of replication studies. A further 238 (39.7%) were implicitly dismissive of replication studies, and the remaining 3 (0.5%) journals were explicitly disinterested in considering replication studies for publication. CiteScore and Source Normalized Impact per Paper (SNIP) of neutral journals were significantly lower than those of journals, which were implicitly discouraging replication research. With regard to the journals implicitly discouraging replications (238), journals in the subcategory of business and international management (51) had the highest percentage (21.4%) followed by strategy and management 30 (12.6%) and Organizational Behavior (OB) and Human Resource (HR) 25 (10.5%).

The available literature does not explore the degree to which the editorial policies of business and management journals explicitly or implicitly discourage replication studies. The current study attempts to address this gap in the literature. Given the lack of support for replications among business and management journals, the current paper sets forth the suggested steps which are deemed crucial for moving beyond the replication crisis in the business and management field.

The authors aimed to examine the presence and character of inter- and intra-approaches to replication studies published in five leading marketing journals (Journal of Marketing, Journal of Marketing Research, Journal of Public Policy & Marketing, Marketing Science, Journal of the Academy of Marketing Science) across four decade intervals (i.e. 1980s, 1990s, 2000s and 2010/2011). The research sought answers to three research questions.

Content analysis of a randomly selected sample of 2,717 articles found 128 replicative studies in the journal issues. Comparisons of the replication approaches of the studies address two issues: the criticism that intra-study replication is not true replication as it is inconsistent with the criterion of researcher independence and the reported outcomes of the replicative studies relative to those of the original studies.

Overall, the presence of replications increased over time; however, the increase was attributable primarily to the number of intra-study replications published in two decades, the 2000s and 2010/2011 intervals. Conflicting findings infrequently appeared in the replication studies regardless of approach, indicating the possible existence of confirmation bias in the marketing literature.

Replication in marketing is either improving or stagnant depending on the accepted definition of replication. Of special importance, given the questioning of the intra-study approach as true replicative research, more replicated findings produced by independent researchers are needed to establish theoretical validity of marketing knowledge for use by both marketing academicians and decision makers.

The purpose of this paper is to highlight the role of replications for science, and in particular the knowledge development process.

Descriptive research on the disclosure of sample parameters which are needed for replication was conducted. The analysis includes 2,982 studies from four top-tier marketing journals.

Published parameters are insufficient for replication and, therefore, impede knowledge development.

The paper offers a unique data set for further investigation. In total, 2,982 studies from the defined journals (Journal of Consumer Research, Journal of Marketing, Journal of Marketing Research and Marketing Science) were analyzed. Hereby this paper enables insights into reporting practices of current marketing research and highlights the role of replication research in validating earlier research. It empirically shows, to the authors' best knowledge for the first time, that the insufficient reporting is one of the major reasons for the lack of replications.

The purpose of this study is to see if the affirmative results seen in the pilot study of the positive addiction recovery therapy (PART) programme are replicable and durable given a new cohort of participants. PART is a programme of work designed to improve the recovery and well-being of people in early addiction recovery. Its foundation is in the G-CHIME (growth, connectedness, hope, identity, meaning in life and empowerment) model of addiction recovery. It also uses the values in action character strengths and includes a set of recovery protection techniques.

This study uses a mixed method experimental design, incorporating direct replication and a follow-up study. Measures for recovery capital, well-being and level of flourishing are used to collect pre-, post- and one-month follow-up data from participants. The replication data analysis uses the non-parametric Wilcoxon test, and the follow-up analysis uses the Friedman test with pairwise comparison post hoc analysis. The eligibility criteria ensure participants (n = 35) are all in early addiction recovery, classified as having been abstinent for between three and six months.

This study found a statistically significant improvement in well-being, recovery capital and flourishing on completion of the PART programme. These findings upheld the hypotheses in the pilot study and the successful results reported. It also found these gains to be sustained at a one-month follow-up.

This study endorses the efficacy of the PART programme and its continued use in a clinical setting. It also adds further credibility to adopting a holistic approach when delivering interventions which consider important components of addiction recovery such as those outlined in the G-CHIME model.

This study adds to the existing evidence base endorsing the PART programme and the applied use of the G-CHIME model.

Given that institutional and cultural factors affect the transfer of HRM practices between multinational companies (MNCs) and their subsidiaries, the paper set out to investigate the explanatory mechanism for the replication of transferred HRM practices and the factors likely to influence the choice of transfer mechanisms.

Using an exploratory qualitative approach, 40 interviews were conducted with senior managers and employees in two MNC subsidiaries located in the alcoholic beverage and oil industries in Nigeria.

The findings suggest that organizational culture (OC) was the mechanism for MNC HRM replication and inhibition. In addition to explaining the mechanisms for HRM practice replication, the authors developed a conceptual framework to explain how clan and hierarchical OC influence the extent to which human resource (HR) practices are replicated or inhibited and how institutional and cultural factors influence the choice of OC.

The study uncovered that MNCs deploy hierarchical OC to oversee the replication of transferred practices. Interestingly, the authors found that institutional and cultural environments were key factors that influenced the choice of mechanism for overseeing the replication of HR practices. The conceptual framework can help managers of MNCs to understand how to replicate transferred HRM practices in developing countries.

The overall goal of science is to build a valid and reliable body of knowledge about the functioning of the world and how applying that knowledge can change it. As personnel and human resources management researchers, we aim to contribute to the respective bodies of knowledge to provide both employers and employees with a workable foundation to help with those problems they are confronted with. However, what research on research has consistently demonstrated is that the scientific endeavor possesses existential issues including a substantial lack of (a) solid theory, (b) replicability, (c) reproducibility, (d) proper and generalizable samples, (e) sufficient quality control (i.e., peer review), (f) robust and trustworthy statistical results, (g) availability of research, and (h) sufficient practical implications. In this chapter, we first sing a song of sorrow regarding the current state of the social sciences in general and personnel and human resources management specifically. Then, we investigate potential grievances that might have led to it (i.e., questionable research practices, misplaced incentives), only to end with a verse of hope by outlining an avenue for betterment (i.e., open science and policy changes at multiple levels).

Current publication practices in the scholarly (International) Business and Management community are overwhelmingly anti-Popperian, which fundamentally frustrates the production of scientific progress. This is the result of at least five related biases: the verification, novelty, normal science, evidence, and market biases. As a result, no one is really interested in replicating anything. In this essay, the author extensively argues what he believes is wrong, why that is so, and what we might do about this. The paper aims to discuss these issues.

This is an essay, combining a literature review with polemic argumentation.

Only a tiny fraction of published studies involve a replication effort. Moreover, journal authors, editors, reviewers and readers are not interested in seeing nulls and negatives in print. This replication crisis implies that Popper’s critical falsification principle is actually thrown into the scientific community’s dustbin. Behind the façade of all these so-called new discoveries, false positives abound, as do questionable research practices meant to produce all this allegedly cutting-edge and groundbreaking significant findings. If this dismal state of affairs does not change for the good, (International) Business and Management research is ending up in a deadlock.

A radical cultural change in the scientific community, including (International) Business and Management, is badly needed. It should be in the community’s DNA to engage in the quest for the “truth” – nothing more, nothing less. Such a change must involve all stakeholders: scholars, editors, reviewers, and students, but also funding agencies, research institutes, university presidents, faculty deans, department chairs, journalists, policymakers, and publishers. In the words of Ioannidis (2012, p. 647): “Safeguarding scientific principles is not something to be done once and for all. It is a challenge that needs to be met successfully on a daily basis both by single scientists and the whole scientific establishment.”

Publication practices have to change radically. For instance, editorial policies should dispose of their current overly dominant pro-novelty and pro-positives biases, and explicitly encourage the publication of replication studies, including failed and unsuccessful ones that report null and negative findings.

This is an explicit plea to change the way the scientific research community operates, offering a series of concrete recommendations what to do before it is too late.