Search results

In this interview with Harvard innovation expert Stefan H.Thomke about his latest book, Experimentation Works: The Surprising Power of Business Experiments, he pays tribute to the scientific method and “the engine that has powered” it over the centuries, the “humble experiment.”

Professor Thomke anticipates a burgeoning role for business experimentation, one that it is already playing across the value chain, particularly in leading online companies.

Digital experimentation tools have the potential to revolutionize a company’s R&D, but they can also transform entire industries by shifting experimentation–and thus product innovation–to users and customers.

The ability to access large customer samples, to automatically collect huge amounts of data about user interactions on websites and apps, and to run concurrent experiments gives companies an unprecedented opportunity to evaluate many ideas quickly, with great precision, and at a negligible cost per additional experiment.

Product development is being transformed by rapid experimentation: all aspects of software–including user interfaces, security applications and back-end changes–can now be subjected to A/B tests.

Some organizations are responding to the opportunity of innovation, while others are not. Harvard innovation expert Stefan H. Thomke says a key reason for the slow pace of innovation is that most organizations lack "a culture of experimentation." In this article the author explains the impediments to establishing such a culture. 10;

In his latest book, Experimentation Works: The Surprising Power of Business Experiments, 10;Thomke says, to create a culture of experimentation, firms need “a new model of leadership,” which cultivates curiosity, emphasizes data-informed decisions, experiments ethically, sets grand challenges and establishes systematic training and support for rapid experimentation. Those are all positive steps in the direction of creating a culture of experimentation.

But they won’t be enough because the underlying management model of most big organizations is explicitly aimed at preventing the very kind of experimentation that Thomke is recommending.

In a digital world, technology has made it radically simpler and quicker and cheaper to carry out and evaluate multiple experiments. The whole organization needs to become an organic living network of high-performance teams. In such firms competence resides throughout the organization and that innovation can and must come from anywhere. 10;

An important discussion of how top down managements squelch experimentation and thus throttle innovation.

The purpose of this paper is to revise Burgelman’s idea (1991, 1994) that induced strategic processes is necessarily variation-reducing. In doing so, the authors explore whether major change in a firm’s administrative system can be managed in an evolutionary fashion via induced variation-increasing mechanisms. In particular, the authors focus on a multi-business multinational firm in which different administrative systems were experimented simultaneously as a way to determine which of these systems provided the most conducive context for innovation and capability development.

The authors adopted an inductive perspective and developed a single case research project aimed at documenting the process of experimentation and subsequent selection and adoption of a new administrative system by a large multi-business multinational firm.

The paper’s main contribution is the concept of “induced variation”, understood as intra-organizational variation-increasing mechanisms deliberately created at the top level of the organization to trigger an intra-organizational evolutionary process of management innovation. This finding extends and modifies Burgelman’s discussion of induced and autonomous strategic behavior by showing that induced processes need not necessarily be variation-reducing, but may actually be variation-increasing. Additionally, the authors explain how an evolutionary process aimed at learning about the relative merits of alternative administrative systems through in vivo “reflection in action” (Schön, 1983) unfolds in a complex global organization.

While the work provides several insights on the development of an evolutionary process leading to management innovation, its inductive nature limits its external validity and requires the development of further work for such purpose.

The authors explore the roles of regional organizations in creating new corporate capabilities for the MNC.

The authors show how management capabilities developed in the Latin American context were rolled out to other locations.

The authors' findings confirm that major drastic reorganization initiatives can actually be approached using an evolutionary approach.

The effort in addition to relevance is a major factor for satisfaction and utility of the document to the actual user. The purpose of this paper is to propose a method in generating relevance judgments that incorporate effort without human judges’ involvement. Then the study determines the variation in system rankings due to low effort relevance judgment in evaluating retrieval systems at different depth of evaluation.

Effort-based relevance judgments are generated using a proposed boxplot approach for simple document features, HTML features and readability features. The boxplot approach is a simple yet repeatable approach in classifying documents’ effort while ensuring outlier scores do not skew the grading of the entire set of documents.

The retrieval systems evaluation using low effort relevance judgments has a stronger influence on shallow depth of evaluation compared to deeper depth. It is proved that difference in the system rankings is due to low effort documents and not the number of relevant documents.

Hence, it is crucial to evaluate retrieval systems at shallow depth using low effort relevance judgments.

Health care ecosystems instantiate different innovation trajectories, driven either by science-/techno-push or user-centric rationales. This article focuses on organization intermediaries (OIs), respectively, active in health care ecosystems driven by science- and techno-push versus user-centric innovation processes; it aims at characterizing their operation and intervention modes. The analysis elaborates on network and content brokerage. Innovation also needs to consider various challenges associated with physical vicinity. The authors check whether territorial anchoring plays a role in brokerage, depending on the innovation model.

The article offers an investigation of eight French organizations matching the definition of OIs and active in different areas of health care-related innovation. It follows a qualitative and abductive research protocol adhering to the precepts of grounded theory.

First, the authors show that content and network brokerage specialize in specific activities in each innovation model. On network brokerage, the authors show that OIs foster the development of communities of practice in the science-/techno-push model, while they nurture communities of innovation in the user-centric model. Services materializing content brokerage are typical consequences of activities performed in each model. The second contribution deals with physical vicinity. In the science-/techno-push model, OIs install a physical space (the “internal” dimension) to support the development of communities of practice, while the “external” dimension copes with agglomeration effects. In the user-centric model, OIs deliver services thanks to the “internal” space; communities of innovation create a leverage effect on the physical space to operate their activities that are supported by “external” network effects.

The originality of the article lies in the description of the alternative roles plaid by organization intermediaries in the science-/techno-push versus user-centric approaches of innovation. In these two approaches, (contents and network) brokerage and physical vicinity play different roles.

In a system-based approach, replicating the web would require large test collections, and judging the relevancy of all documents per topic in creating relevance judgment through human assessors is infeasible. Due to the large amount of documents that requires judgment, there are possible errors introduced by human assessors because of disagreements. The paper aims to discuss these issues.

This study explores exponential variation and document ranking methods that generate a reliable set of relevance judgments (pseudo relevance judgments) to reduce human efforts. These methods overcome problems with large amounts of documents for judgment while avoiding human disagreement errors during the judgment process. This study utilizes two key factors: number of occurrences of each document per topic from all the system runs; and document rankings to generate the alternate methods.

The effectiveness of the proposed method is evaluated using the correlation coefficient of ranked systems using mean average precision scores between the original Text REtrieval Conference (TREC) relevance judgments and pseudo relevance judgments. The results suggest that the proposed document ranking method with a pool depth of 100 could be a reliable alternative to reduce human effort and disagreement errors involved in generating TREC-like relevance judgments.

Simple methods proposed in this study show improvement in the correlation coefficient in generating alternate relevance judgment without human assessors while contributing to information retrieval evaluation.

Modern day business is beset with changing operating paradigms. Economies with efficient economic policies and stable political systems are a big draw among the investors. Countries that have opened themselves to world markets and that have good legal systems in place, providing protection to investors have attracted more capital in the process of globalization. As the demand for capital is growing in both the developed and the developing economies, the need to establish good governance practices has gained momentum. Governance practices however, are not uniform across nations. This diversity may be particularly because of the different legal structures and cultural settings adopted by different nations. This paper tries to explore the arguments on convergence and divergence of corporate best practices, keeping in view the various governance models currently in practice. Explaining the rationale behind the emergence of corporate governance as a movement, this paper attempts at discussing the various prevalent systems of governance. In the end an attempt is made to address the challenges to corporate governance in the context of globalization of best practices. Given the cultural settings of different nations it is argued that it would never be possible for corporate laws to converge universally. New models of corporate governance are likely to emerge given the large‐scale experimentation done by transition economies.

The recent rise in online knowledge repositories and use of formalism for structuring knowledge, such as ontologies, has provided necessary conditions for the emergence of tools for generating knowledge assessment. These tools can be used in a context of interactive computer-assisted assessment (CAA) to provide a cost-effective solution for prompt feedback and increased learner’s engagement. The purpose of this paper is to describe and evaluate a tool developed by the authors, which generates test questions from an arbitrary domain ontology, based on sound pedagogical principles encapsulated in Bloom’s taxonomy.

This paper uses design science as a framework for presenting the research. A total of 5,230 questions were generated from 90 different ontologies and 81 randomly selected questions were evaluated by 8 CAA experts. Data were analysed using descriptive statistics and Kruskal–Wallis test for non-parametric analysis of variance.

In total, 69 per cent of generated questions were found to be useable for tests and 33 per cent to be of medium to high difficulty. Significant differences in quality of generated questions were found across different ontologies, strategies for generating distractors and Bloom’s question levels: the questions testing application of knowledge and the questions using semantic strategies were perceived to be of the highest quality.

The paper extends the current work in the area of automated test generation in three important directions: it introduces an open-source, web-based tool available to other researchers for experimentation purposes; it recommends practical guidelines for development of similar tools; and it proposes a set of criteria and standard format for future evaluation of similar systems.