To read this content please select one of the options below:

Organizing for permanent beta: performance measurement before vs performance monitoring after release of digital services

Kim E. van Oorschot (Accounting and Operations Management, BI Norwegian Business School, Oslo, Norway)
Henk A. Akkermans (Tilburg School of Economics and Management, Tilburg University, Tilburg, The Netherlands)
Luk N. Van Wassenhove (Technology and Operations Management, INSEAD, Fontainebleau, France)
Yan Wang (Delft University of Technology, Delft, The Netherlands)

International Journal of Operations & Production Management

ISSN: 0144-3577

Article publication date: 1 February 2022

Issue publication date: 6 March 2023




Due to the complexity of digital services, companies are increasingly forced to offer their services “in permanent beta”, requiring continuous fine-tuning and updating. Complexity makes it extremely difficult to predict when and where the next service disruption will occur. The authors examine what this means for performance measurement in digital service supply chains.


The authors use a mixed-method research design that combines a longitudinal case study of a European digital TV service provider and a system dynamics simulation analysis of that service provider's digital service supply chain.


With increased levels of complexity, traditional performance measurement methods, focused on detection of software bugs before release, become fragile or futile. The authors find that monitoring the performance of the service after release, with fast mitigation when service incidents are discovered, appears to be superior. This involves organizational change when traditional methods, like quality assurance, become less important.

Research limitations/implications

The performance of digital services needs to be monitored by combining automated data collection about the status of the service with data interpretation using human expertise. Investing in human expertise is equally important as investing in automated processes.


The authors draw on unique empirical data collected from a digital service provider's struggle with performance measurement of its service over a period of nine years. The authors use simulations to show the impact of complexity on staff allocation.



The authors acknowledge the support of the people who made their field research possible: Peter Claerhoudt, Peggy Corstens and Theo Wakkermans.


van Oorschot, K.E., Akkermans, H.A., Van Wassenhove, L.N. and Wang, Y. (2023), "Organizing for permanent beta: performance measurement before vs performance monitoring after release of digital services", International Journal of Operations & Production Management, Vol. 43 No. 3, pp. 520-542.



Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles