To read this content please select one of the options below:

Exploring the design elements of open evaluation

Jörg B.A. Haller (Tomorrow Focus AG, Munich, Germany)
Vivek K. Velamuri (Group Innovation and Entrepreneurship, HHL Leipzig Graduate School of Management, Leipzig, Germany)
Dirk Schneckenberg (Strategic Management Department, ESC Rennes School of Business, Rennes, France)
Kathrin M. Möslein (Group Innovation and Entrepreneurship, HHL Leipzig Graduate School of Management, Leipzig, Germany) (School of Business and Economics, University of Erlangen-Nuremberg, Nuremberg, Germany)

Journal of Strategy and Management

ISSN: 1755-425X

Article publication date: 20 February 2017




Firms increasingly integrate a wide range of actors in the early ideation and concept creation phases of innovation processes leading to the collection of a large number of ideas. This creates the challenge of filtering the most promising ideas from a large number of submissions. The use of external stakeholders into the evaluation and selection of submissions (i.e. open evaluation (OE)) might be a viable alternative. The purpose of this paper is to provide a state-of-the-art analysis on how such OE systems are designed and structured.


Since OE is a new phenomenon, an exploratory qualitative research approach is adopted. In all, 122 instances of OE in 90 innovation contest cases are examined for their design elements.


This research reveals that OE systems are configured in many different ways. In total, 32 design elements and their respective parameters are identified and described along the six socio-technical system components of an OE system. This study allows for a comprehensive understanding of what OE is and what factors need to be taken into consideration when designing an OE system.

Practical implications

Scholars and professionals may draw insights on what design choices to make when implementing OE.


The comprehensive analysis performed in this study contributes to research on open and user innovation by examining the concept of OE. In particular, it extends knowledge on design elements of OE systems.



The authors thank the anonymous reviewers who provided constructive feedback during the paper revision process. This paper is part of the dissertation project of the first author Jörg B.A. Haller (see Haller, 2012). An earlier version of this paper was presented at the 9th European Academy of Management Conference (2009), Liverpool, UK.


Haller, J.B.A., Velamuri, V.K., Schneckenberg, D. and Möslein, K.M. (2017), "Exploring the design elements of open evaluation", Journal of Strategy and Management, Vol. 10 No. 1, pp. 40-65.



Emerald Publishing Limited

Copyright © 2017, Emerald Publishing Limited

Related articles