(2018), "Prelims", Improving the Marriage of Modeling and Theory for Accurate Forecasts of Outcomes (Advances in Business Marketing and Purchasing, Vol. 25), Emerald Publishing Limited, Bingley, pp. i-xii. https://doi.org/10.1108/S1069-096420180000025011
Emerald Publishing Limited
Copyright © 2018 Emerald Publishing Limited
Half Title Page
IMPROVING THE MARRIAGE OF MODELING AND THEORY FOR ACCURATE FORECASTS OF OUTCOMES
ADVANCES IN BUSINESS MARKETING & PURCHASING
Series Editor: Arch G. Woodside
|Volume 11:||Essays by Distinguished Marketing Scholars of the Society for Marketing Advances|
|Volume 12:||Evaluating Marketing Actions and Outcomes|
|Volume 13:||Managing Product Innovation|
|Volume 14:||Creating and Managing Superior Customer Value|
|Volume 15:||Business-To-Business Brand Management: Theory, Research and Executive Case Study Exercises|
|Volume 16:||Organizational Culture, Business-to-Business Relationships, and Interfirm Networks|
|Volume 17:||Interfirm Networks: Theory, Strategy and Behavior|
|Volume 18:||Business-to-Business Marketing Management: Strategies, Cases, and Solutions|
|Volume 19:||Reflections and Advances in Honor of Dan Nimer|
|Volume 20:||Deep Knowledge of B2B Relationships within and across Borders|
|Volume 21:||Field Guide to Case Study Research in Business-to-Business Marketing and Purchasing|
|Volume 22A:||Sustaining Competitive Advantage via Business Intelligence, Knowledge Management, and System Dynamics|
|Volume 22B:||Sustaining Competitive Advantage via Business Intelligence, Knowledge Management, and System Dynamics|
|Volume 23A:||E-Services Adoption: Processes by Firms in Developing Nations|
|Volume 23B:||E-Services Adoption: Processes by Firms in Developing Nations|
|Volume 24:||Making Tough Decisions Well and Badly: Framing, Deciding, Implementing, Assessing|
ADVANCES IN BUSINESS MARKETING & PURCHASING VOLUME 25
IMPROVING THE MARRIAGE OF MODELING AND THEORY FOR ACCURATE FORECASTS OF OUTCOMES
ARCH G. WOODSIDE
Curtin University, Perth, Australia
United Kingdom – North America – Japan – India – Malaysia – China
Emerald Publishing Limited
Howard House, Wagon Lane, Bingley BD16 1WA, UK
First edition 2018
Copyright © 2018 Emerald Publishing Limited
Reprints and permissions service
No part of this book may be reproduced, stored in a retrieval system, transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without either the prior written permission of the publisher or a licence permitting restricted copying issued in the UK by The Copyright Licensing Agency and in the USA by The Copyright Clearance Center. Any opinions expressed in the chapters are those of the authors. Whilst Emerald makes every effort to ensure the quality and accuracy of its content, Emerald makes no representation implied or otherwise, as to the chapters’ suitability and application and disclaims any warranties, express or implied, to their use.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
ISBN: 978-1-78635-122-7 (Print)
ISBN: 978-1-78635-121-0 (Online)
ISBN: 978-1-78714-891-8 (Epub)
ISSN: 1069-0964 (Series)
List of Contributors
|Huat Bin (Andy) Ang||School of Management, Auckland University of Technology, Auckland, New Zealand|
|Carol M. Megehee||Wall College of Business Administration, Coastal Carolina University, Conway, SC, USA|
|Gábor Nagy||INSEEC, Paris Business School, Paris, France|
|Arch G. Woodside||School of Marketing, Curtin University, Perth, Australia|
Fiss (2007) and Meyer, Tsui, and Hinings (1993) suggest that many of the problems in empirical research on organizational configurations derive from a mismatch between methods and theory. Configurational theory suggests a clean break with the predominant linear paradigm. Rather than implying singular causation and linear relationships, a configurational approach assumes complex causality and nonlinear relationships where “variables found to be causally related in one configuration may be unrelated or even inversely related in another” (Meyer et al., 1993, p. 117). The linear paradigm relies on null hypothesis significance test (NHST) – proposing alternative hypotheses that directional relationships exist (e.g., increases in variable X associates with increases in variably Y).
Are Fiss (2007) and Meyer et al. (1993) correct? If they are correct, why do almost all empirical studies that report forecasting models rely on using symmetric tests (e.g., correlation, multiple regression analysis (MRA), and structural equation modeling (SEM)) and reporting NHST findings? If such analytical tools as symmetric tests and NHST represent bad science, what alternative data analytical tools should researchers use? This volume in Advances in Business Marketing & Purchasing answers these questions and provides examples of using configurational modeling that are somewhat precise outcome tests (SPOTs). The volume in your hands or on your screen suggests that you stop using NHST and symmetric tests such as correlation, MRA, and SEM. The chapters in this volume describe complexity theory tenets and provide examples mostly from the business-to-business strategy, marketing, and purchasing literatures on why and how to build asymmetric models using configurations of antecedent conditions.
Yes, Fiss (2007) and Meyer et al. (1993) are correct on all counts. Additional researchers (Armstrong, 2012; Gigerenzer, 1991; Hubbard, 2016; Ziliak & McCloskey, 2008) – who have carefully reviewed and documented research tools on the usefulness of data analytic methods – reach the same or similar conclusions. Using symmetric tests such as correlation analysis, MRA, and SEM misrepresents the information quality and quantity that a researcher can mine from a data set. Usually the decision by researchers to use symmetric tools and NHST is done automatically, without explicit thinking about the availability and usefulness of asymmetric tools and SPOT. Most researchers propose theories in strategic management, finance, organizational behavior, marketing, and management at the case level but then do symmetric tests on the basis of relationships among variables. Offering case theory and doing variable-relationship testing is the mismatch that Fiss (2007) and Meyer et al. (1993) identify. Both the construction and testing of theory at the case level using asymmetric tests of configurational casual statements is possible and several examples are available (e.g., Frösén, Luoma, Jaakkola, Tikkanen, & Aspara, 2016; McClelland, 1998; Ordanini, Parasuraman, & Rubera, 2014; Wu, Yeh, Huan, & Woodside, 2014).
The dominant practice in the teen years of the 21st century in constructing forecasting models relating to strategic management is to perform MRA and SEM and test resulting models for fit of the predictions of the observations for a dependent variable. However, “Achieving a good fit to observations does not necessarily mean we have found a good model, and choosing the model with the best fit is likely to result in poor predictions. Despite this, Roberts and Pashler (2000) estimated that, in psychology alone, the number of articles relying on a good fit as the only indication of a good model runs into the thousands” (Gigerenzer & Brighton, 2009, p. 118). These studies are examples of shallow analysis that are accurately describable as examples of the rubbish that saddens McCloskey (2002).
The editor-in-chief of at least one journal, Basic and Applied Social Psychology, has now banned the practice of reporting NHST findings as well as confidence intervals from future articles accepted for publication. The NHST and confidence intervals ban announcement by Trafimow and Marks (2015) confirms Hubbard’s (2016) and Ziliak and McCloskey’s troublemaker status in attempting to overthrow bad with good science. This action exemplifies Gigerenzer’s (2004, p. 604) call for courage, “To stop the [NHST] ritual, we also need more guts and nerves. We need some pounds of courage to cease playing along in this embarrassing game. This may cause friction with editors and colleagues, but it will in the end help them to enter the dawn of statistical thinking.” NHST and fit testing-only of regression models are the pervasive practices in articles appearing in all elite and otherwise ranked journals in management and marketing today. As Hubbard (2016) documented, such corrupt theory construction and testing has dominated these literature streams since the early 1960s. Gigerenzer (2008, p. 170) explained that these practices are procedures of bad science, “Statistical packages allow every difference, interaction, or correlation against chance to be tested.” They automatically deliver ratings of “significance” in terms of stars, double stars, and triple stars, encouraging the bad after-the-fact habit. The general problem Feynman (1998) addressed is known as overfitting. Fitting a model to data that is already obtained is not sound hypothesis testing, even if the resulting explained variance, or R2, is impressive. The reason is that one does not know how much noise one has fitted, and the more adjustable parameters one has, the more noise one can fit. Psychologists habitually fit rather than predict, and rarely test a model on new data, such as by cross-validation (Roberts & Pashler, 2000). Fitting per se has the same problems as storytelling after the fact, which leads to a “hindsight bias (Hoffrage, Hertwig, & Gigerenzer, 2000).”
Symmetric testing of statistical significance of directional hypotheses is pervasive in the literature of business marketing and purchasing. Unfortunately, the evidence is abundant that the dominant logic of symmetric testing of directional hypothesis is bad practice and contributes to bad science. Symmetric tests include correlation analysis, the F-test, MRA, and SEM. Researchers perform symmetric tests in most instances with the hope of rejecting null hypotheses. The null hypotheses is a prediction that the relationship between two variables (X and Y) is statistically equal to zero or that the behavior of firms or customers in group A versus group B have beliefs, attitude, and behaviors equal to zero. Tools for symmetric testing appearing in most articles in today’s leading scholarly journals of finance, management, marketing, and psychology include computing correlations (r’s) and b coefficients in multiple regression analyses. The NHST examines whether or not an observed r or b coefficient differs from zero to such an extent that the observed difference is unlikely to have occurred by chance alone (p < .05 or p < .01). The p < .05 indicates that the observed finding would occur less than five times in one hundred if the analysis was done 100 times using the same data collection instruments on separate samples.
Critics of the use of NHSTs describe the severe limitations of NHST. One criticism is that all observed findings in NHST differ statistically different from zero if the sample of cases is very large (n > 5,000). Second, the study of which variables are statistically different from zero and which variable measurements do not differ from zero does not provide information on which configurations of conditions are present that indicate the consistent occurrence of a specific outcome. The primary research focus needs to be on identifying the configurations of ingredients that accurately and consistently predict high performance (or low performance). In reading the chapters in the present ABMP volume, the reader learns how to construct theories of complex configurations of conditions that are sufficient in identifying specific outcomes consistently. “Consistently” refers to the model accurately predicting the same outcome frequently with few, if any, false positives, when testing the model on cases from new samples. Note in reading that a trade-off occurs between accuracy and coverage. Models highly accurate (prediction odds 10 correct to 1 incorrect cases) in forecasting specific cases usually has a greater number of conditions than the accuracy of a simpler model (i.e., a complex statement includes only three conditions that achieves an accuracy of four to one correct to mistaken case identifications). The hope is that the reader reaching the final sentence of this preface is intrigued sufficiently to read the first chapter and then the rest of the volume. Good reading!
Arch G. Woodside
Armstrong (2012) Armstrong, J. S. (2012). Illusions in regression analysis. International Journal of Forecasting, 28, 689–694.
Feynman (1998) Feynman, R. (1998). The Meaning of it all: Thoughts of a citizen-scientist (pp. 80–81). Reading, MA: Perseus Book.
Fiss (2007) Fiss, P. C. (2007). A set-theoretic approach to organizational configurations. Academy of Management Review, 32, 1180–1198.
Frösén, Luoma, Jaakkola, Tikkanen, & Aspara (2016) Frösén, J. , Luoma, J. , Jaakkola, M. , Tikkanen, H. , & Aspara, J. (2016). What counts versus what can be counted: The complex interplay of market orientation and marketing performance measurement. Journal of Marketing, 80, 60–78.
Gigerenzer (1991) Gigerenzer, G. (1991). From tools to theories: A heuristic of discovery in cognitive psychology. Psychological Review, 98, 254–267.
Gigerenzer (2004) Gigerenzer, G. (2004). Mindless statistics. The Journal of Socio-Economics, 33, 587–606.
Gigerenzer (2008) Gigerenzer, G. (2008). Rationality for mortals: How people cope with uncertainty. Oxford, UK: Oxford University Press.
Gigerenzer & Brighton (2009) Gigerenzer, G. , & Brighton, H. (2009). Homo heuristicus: Why biased minds make better inferences. Topics in Cognitive Science, 1, 107–143.
Hoffrage, Hertwig, & Gigerenzer (2000) Hoffrage, U. , Hertwig, R. , & Gigerenzer, G. (2000). Hindsight bias: A byproduct of knowledge updating? Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 566–581.
Hubbard (2016) Hubbard, R. (2016). Corrupt research: The case for reconceptualizing empirical management and social science. Thousand Oaks, CA: Sage.
McClelland (1998) McClelland, D. C. (1998). Identifying competencies with behavioral-event interviews. Psychological Science, 9, 331–339.
McCloskey (2002) McCloskey, D. (2002). The secret sins of economics. Chicago, IL: Prickly Paradigm Press.
Meyer, Tsui, & Hinings (1993) Meyer, A. D. , Tsui, A. S. , & Hinings, C. R. (1993). Configurational approaches to organizational analysis. Academy of Management Journal, 36, 1175–1195.
Ordanini, Parasuraman, & Rubera (2014) Ordanini, A. , Parasuraman, A. , & Rubera, G. (2014). When the recipe is more important than the ingredients: A qualitative comparative analysis (QCA) of service innovation configurations. Journal of Service Research, 17, 134–149.
Roberts & Pashler (2000) Roberts, S. , & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107, 358–367.
Trafimow & Marks (2015) Trafimow, D. , & Marks, E. (2015). Editorial. Basic and Applied Social Psychology, 37, 1–2.
Wu, Yeh, Huan, & Woodside (2014) Wu, P. L. , Yeh, S. S. , Huan, T. C. , & Woodside, A. G. (2014). Applying complexity theory to deepen service dominant logic: Configural analysis of customer experience-and-outcome assessments of professional services for personal transformations. Journal of Business Research, 67, 1647–1670.
Ziliak & McCloskey (2008) Ziliak, S. T. , & McCloskey, D. N. (2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. Ann Arbor, MI: University Michigan Press.
- Embracing the Paradigm Shift from Variable-Based to Case-Based Modeling
- Four-Corner Outcomes in Strategic Management: Successful and Unsuccessful Paddling Down versus Upstream
- Accurately Predicting Precise Outcomes in Business-to-Business Marketing
- Building Generalizable Case-Based Theory in Human Resources Management
- Computing with Words in Modeling Firms’ Paradoxical Performances