The evolution of project portfolio selection methods: from incremental to radical innovation

Ximena Alejandra Flechas Chaparro (PPGA-FEAUSP, Universidade de Sao Paulo, Sao Paulo, Brazil)
Leonardo Augusto de Vasconcelos Gomes (PPGA-FEAUSP, Universidade de Sao Paulo, Sao Paulo, Brazil)
Paulo Tromboni de Souza Nascimento (PPGA-FEAUSP, Universidade de Sao Paulo, Sao Paulo, Brazil)

Revista de Gestão

ISSN: 2177-8736

Publication date: 15 July 2019

Abstract

Purpose

The purpose of this paper is to identify how project portfolio selection (PPS) methods have evolved and which approaches are more suitable for radical innovation projects. This paper addressed the following research question: how have the selection approaches evolved to better fit within radical innovation conditions? The current literature offers a number of selection approaches with different and, in some cases, conflicting nature. Therefore, there is a lack of understanding regarding when and how to use these approaches in order to select a specific type of innovation projects (from incremental to more radical ones).

Design/methodology/approach

Given the nature of the research question, the authors perform a systematic literature review method and analyze 48 portfolio selection approaches. The authors then classified and characterized these articles in order to identify techniques, tools, required data and types of examined projects, among other aspects.

Findings

The authors identify four key features related to the selection of radical innovation projects: dynamism, interdependency management, uncertainty treatment and required input data. Based on the content analysis, the authors identified that approaches based on different sources and nature of data are more appropriated for uncertain conditions, such as behavioral methods, information gap theory, real options and integrated approaches.

Originality/value

The research provides a comprehensive framework about PPS methods and how they have been evolving over time. This portfolio selection framework considers the particular aspects of incremental and radical innovation projects. The authors hope that the framework contributes to reinvigorating the literature on selection approaches for innovation projects.

Keywords

Citation

Flechas Chaparro, X., de Vasconcelos Gomes, L. and Tromboni de Souza Nascimento, P. (2019), "The evolution of project portfolio selection methods: from incremental to radical innovation", Revista de Gestão, Vol. 26 No. 3, pp. 212-236. https://doi.org/10.1108/REGE-10-2018-0096

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Ximena Alejandra Flechas Chaparro, Leonardo Augusto de Vasconcelos Gomes and Paulo Tromboni de Souza Nascimento

License

Published in Revista de Gestão. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

There are two ways for a business to succeed at new products: doing projects right, and doing the right projects.

(Cooper, Edgett, & Kleinschmidt 2002)

Project portfolio selection (PPS) is an iterative process whereby managers select projects from available proposals and current projects, in order to meet organizational objectives (Archer & Ghasemzadeh, 1999). This process is crucial to maintain the competitive advantage, enabling the company to focus on the most relevant and strategic projects. However, firms struggle in managing this process. PPS’ complexity relies on two main facts. First, the information required to determine whether a project could be successful or not is extremely difficult – if not impossible – to know (Baker, 1974). And second, because the project and environment conditions are too volatile, strategic goals might change, promoting high levels of uncertainty (Rice, O’Connor, & Pierantozzi, 2008).

Different methods and approaches have been proposed to address the PPS (Bard, Balachandra, & Kaufmann, 1988; Cooper, 1993; Regan, Ben-Haim, Wilson, Lundberg, Andelman, & Burgman, 2005). Current literature offers a number of increasingly sophisticated project selection approaches, including multi-criteria decision methods, scoring-based and simulation techniques, among others (Zhang, Yang, Dou, & Jiang, 2016). However, the current literature presents an “interesting dichotomy”: although there are an increasing number of frameworks supporting PPS, the adoption by firms and its impact remains limited. In addition, there is a lack of theoretical foundations for these frameworks (Kavadias & Chao, 2008).

Other scholars have also identified similar problems in PPS literature (Beaujon, Marin, & McDonald, 2001; Cooper, Edgett, & Kleinschmidt, 1999; Kavadias & Chao, 2008). Indeed, years of research provided a fragmented and disperse knowledge on PPS, increasing the difficulty of convergence and accumulation of knowledge. To some extent, the existence of this fragmentation might also impact the adoption by firms. The development and adoption of PPS are more difficult for radical innovation, in which managers face high uncertainties in structuring and selecting projects. According to Brasil, Salerno and Gomes (2018), inappropriate selection methods might restrain radical innovation proposals, since, typically, the PPS process has been centered on the assessment of ex ante information, such as estimated costs, profits or market share, that only can be confirmed after launching the project (Zhang et al., 2016). In sum, there are two major concerns in PPS literature, which can render the consolidation of the field more difficult. First, there is a lack of a comprehensive framework regarding how the PPS approaches evolved. And second, there is a lack of understanding of how these different approaches might be associated with different degrees of innovation.

To reduce these two major gaps, the following research question drives our research:

RQ1.

How have the PPS approaches evolved to better fit within radical innovation?

Employing a systematic literature review, we aim to identify which approaches are more suitable for radical innovation projects and what aspects are crucial to manage radical innovation portfolios. Our goal is to go beyond the last review on PPS methods, dated from more than 10 years ago (Cooper et al., 1999), capitalizing on more recent works. Addressing this research question is fundamental to theory and practice. First, there is a lack of understanding regarding when and how to use these approaches to select the specific level (degree) of the innovation project (from incremental to more radical ones). Likewise, without a proper method, firms might not be able to select the opportunities related to radical innovations (Brasil, Salerno, & Gomes, 2018). Using a kind of method that fits all approaches, firms might fail in differentiating the valuation and selection of incremental and radical innovation. Finally, it is also important to determine whether newer PPS methods meet the particular features of radical innovation projects in order to provide a guide for future research.

This paper provides important insights that advance the knowledge on PPS literature. First, we show how PPS evolved. This is especially important to new scholars in the field, who might face difficulties in building a big picture of how this field is structured. Second, we identified the main characteristics of different PPS approaches. This finding provides a rich portrait regarding the approaches, helping scholars and practitioners to identify when and how to use these approaches according to the innovation level. Finally, we provide a theoretical framework in which we associate these approaches according to the innovation level.

This study is structured in five sections as follows. Section 2 presents the methodological procedures including the documents sample selection and the analysis of the methods. Section 3 discusses the PPS approaches, describing each of the 12 taxonomies identified. In Section 4, we discuss the key issues for radical innovation portfolio management and which approaches are more suitable to this end. In addition, we elaborated an integrated portfolio selection method for radical innovation framework, based on the main characteristics and advantages of the approaches. Finally, Section 5 is concluded with the final considerations and future research directions.

2. Methodology

To obtain an overview of the evolution of PPS, a systematic literature review was performed. This methodology enforces explicit methods to perform a thorough research and analysis, underlining how a determined field has evolved, and enabling the identification of consensus or divergence among studies and theories (Briner & Denyer, 2010).

2.1 Sample selection procedure

The collecting methods’ process started through the search on the Web of Science database, which provides different searching and browsing options to obtain higher accuracy during the research. We applied the keywords: “Project Portfolio Selection,” in order to gather the broadest number of related works. The search returned 175 documents; this sample was refined selecting only the pertinent predefined categories as follows: “management,” “operations research management,” “engineering industrial” and “business.” Additionally, we focused our research only on “article,” because this type of documents is peer-reviewed – an important indicator of quality. We then reached out 75 documents.

Another selecting parameter was the inclusion of articles whose purpose was the proposition or improvement of method or approach for PPS. We read again the titles, abstracts and keywords. Based on this process, 35 articles related to “Project Portfolio Selection” were obtained. Notwithstanding, four documents referenced by Cooper, Edgett, and Kleinschmidt (1999) were added and other works were included too because they are recurrently cited by the studies in our sample. In total, 48 articles were selected.

2.2 Methods analysis procedure

Once the sample of articles was established, the first stage was to read them identifying the relevant points and the main contributions including proposed frameworks, techniques, tools, examples of application and required data. Based on Cooper et al.’s (1999) portfolio selection methods classification, we defined 12 taxonomy categories to classify the PPS approaches. The categories are: financial, probabilistic, options pricing, strategic, scoring, combinatorial optimization, behavioral, mapping approaches, real options, integrated methods, information gap theory and scenario-based approach. Table I shows the number of documents per year and methods categories. Combinatorial optimization (15) and integrated methods (13) have the greatest number of approaches and also represent the categories more explored in the last 10 years (ten and five documents, respectively).

Afterward, we carefully examined each PPS approach. We then identified four key requirements that PPS methods must meet in order to address radical innovation projects: dynamism, interdependencies, uncertainties and required input data. Based on these analyses, we proposed an integrated portfolio selection method for radical innovation (Section 5).

3. Project portfolio selection methods

According to Cooper et al. (1999), portfolio selection methods have progressively changed since the first methods emerged in 1960. At that time, methods were strongly focused on mathematical techniques. Over time, because the business and technology environment became more complex, more sophisticated methods emerged. Scholars identified that some of these approaches consist of prescribed solutions (one fits all), which might not fit with the vast variety of project and firms’ conditions (Chien, 2002; Baker, 1974). Furthermore, these prescribed aggregate methods raise a paradigm wherein a given set of projects (usually competing for limited resources) must be appraised in order to choose the ones which will maximize profits. Although these methods might be useful for more incremental innovation, an increasing research stream suggested that prescribed methods are not sufficient to analyze radical innovation projects due to the large numbers of unpredictable variables and knowledge gaps (e.g. Baker, 1974; Archer & Ghasemzadeh, 1999; Ghapanchi, Tavana, Khakbaz, & Low, 2012). In the following sections, we present 12 methods for PPS.

3.1 Financial methods

In these methods, the most important selection criterion is profitability and the objective is to establish a portfolio that maximizes expected returns (Bard et al., 1988). This is one of the most widely adopted methods by managers for screening projects (Chien, 2002; Dutra, Ribeiro, & Carvalho, 2014), and basically, are utilized to assess projects individually. Two documents were identified within this category (Table II).

3.2 Probabilistic methods

Among the most popular probabilistic methods are Monte Carlo simulation and Bayesian theory. These methods analyze different scenarios producing distributions of possible outcome values. Some methods are stochastic, using random values. Archer and Ghasemzadeh (1999) indicate that simulation techniques are more effective to analyze risk (Table III).

3.3 Option pricing theory

Option pricing uses mathematical models to calculate how demand varies at different prices’ levels. These models work with three critical pricing elements: pricing strategy, both buyer and seller value of the product, and techniques and elements that affect profitability (Galai & Masulis, 1976). According to Chien (2002), mathematical models are slightly used by managers because these models tend to do not consider the diversity of projects and criteria (Table IV).

3.4 Strategic methods

Since selecting the right projects is extremely critical, not just to maintain the competitive position, but also to achieve financial goals (Cooper et al., 2002), authors have been exploring selection methods driven basically by strategic objectives. Zeynalzadeh and Ghajari (2011) raise particular awareness about the hierarchy of strategy (i.e. organization, portfolio and project strategy) to be considered during the process of project selection. Strategic Buckets is one of the most widespread methods, in which managers create budget buckets where projects are categorized and prioritized (Cooper et al., 2002). These methods involve both external and internal considerations of the firm (Archer & Ghasemzadeh, 1999), enabling a focused perspective from the entire context (Table V).

3.5 Scoring methods

Basically, projects are screened based on a set of different requirements and aspects. Projects are eliminated if they do not meet the requirements and selected if they meet most part of these requirements (Archer & Ghasemzadeh, 1999). Scoring methods are a quantified extension of a checklist and rely on subjective ratings of managers. This fact presents a weakness because such ratings and weights are arbitrarily determined (Cooper, 1981), and this is one of the reasons why scoring methods could work better with objective-centered approaches. This method is also popular in practice because of its simplicity of execution (Hall & Nauda, 1990) (Table VI).

3.6 Combinatorial optimization

This category includes methods such as multi-criteria decision-making methods (MCDM), data envelopment analysis and analytical hierarchy approaches (AHP). According to Razi and Shariat (2017), MCDM are based on the utility theory and assist the human constraints to deal with profits and loss. In AHP models, criteria and alternatives are arranged in hierarchical structures in at least three levels, in order to prioritize the criteria within each level (Amiri, 2010).

According to Archer and Ghasemzadeh (1999), these methods are complex and difficult to apply, and also require a lot of information. Another constraint stems from the use of Technique for Order of Preference by Similarity to Ideal Solution and crisp values which are not recommended for handling uncertainties since these techniques work with single numeric values (Amiri, 2010). However, robust and specialized assessing algorithms may be useful for early screening when there is a large number of potential projects (Beaujon et al., 2001; Abbassi, Ashrafi, & Tashnizi, 2013). The approach proposed by Amiri (2010) incorporates fuzzy numbers, enabling managers to introduce interval judgments improving handling uncertainty issues (Table VII).

3.7 Behavioral methods

These methods aim to create a consensus based on experts and managers’ opinions to decide which projects should be undertaken. For Cooper et al. (1999), this taxonomy is particularly useful during the initial stages of the project selection, in which, virtually, these opinions might be the single input data available (Table VIII).

3.8 Mapping approaches

In this taxonomy, projects are plotted against various parameters in form of matrices or bubble diagram in which it is possible to visualize the strategic position of the firm’s product (Archer & Ghasemzadeh, 1999). Mapping methods have gained wide popularity among companies. For instance, Linton, Walsh and Morabito (2002) mentioned firms, such as 3M, Lucent Technologies or Procter and Gamble, which have implemented such mapping method (Table IX).

3.9 Real options

These methods incorporate learning models to resolve uncertainties through the experience gained over time. Differently from traditional economic analysis (e.g. discounted cash flow), the real options model is a fluctuating investment decision and assumes that the strategy may evolve depending on the real characteristics of the environment (Mun, 2005). It also helps managers to define a portfolio (which projects and when they should be run) among several seemingly economic outcomes and projects. Brasil et al. (2018) mention some barriers to adopt real options in PPS by firms: mathematical complexity, non-intuitive outcomes, and difficulties to handle unknown uncertainties, among others (Table X).

3.10 Integrated methods

Integrated methods are those in which two or more different taxonomies are employed in order to guide the portfolio selection process (Table XI).

3.11 Information gap theory

This method assists the decision-making process in situations under several knowledge gaps. It works with a mathematical process model, performance requirement and uncertainty model (Regan et al., 2005). Through the information gap model, analysts can identify what they believe to be true and important regarding the system, enabling the evaluation of robustness (identifying possible bad outcomes) and opportunity (windfall chances).

Differently from traditional selection methods, which are based on maximum expected benefits, in this method, the focus is on the identification of knowledge gaps, uncertainties and risks, in order to determine the least vague portfolio (Table XII).

3.12 Scenario-based approach

One of the main concerns of this approach is to address exogenous uncertainties, which are defined by Liesiö and Salo (2012, p. 162) as missing data or information “which are not influenced by the projects” (e.g. macroeconomic developments or rate of industry growth). This approach proposes to deal with exogenous uncertainties through the scenarios technique. Scenario planning enables a company to imagine possible futures through exploring “the joint impact of various uncertainties” (Shoemaker, 1995, p. 26). For Shoemaker (1995, p. 27), scenarios “attempt to capture the richness and range of possibilities, stimulating decision makers to consider changes they would otherwise ignore.”

The methods rely on the assumption that, through mathematical models and functions, it is possible to complete the information provided by experts, to build a scenario and identify synergies and probabilities. Scenario-based methods require specific and sophisticated knowledge to operate, thus, its applicability and diffusion are restricted (Archer & Ghasemzadeh, 1999) (Table XIII).

4. Results and discussion

4.1 Portfolio selection for radical innovation projects

In this section, we develop a discussion by bridging incremental and radical innovation levels and PPS approaches. First, integrated methods (i.e. those that enable managers to employ different techniques and different input data) are more convenient to adopt in both cases – incremental and radical innovations (Archer & Ghasemzadeh, 1999; Cooper et al., 1999; Dutra et al., 2014). This is also especially relevant for radical innovation, since no single method has all the features required to respond adequately under uncertain circumstances. Coldrick, Longhurst, Ivey, and Hannis (2005) recognize that optimization methods are powerful, but if combined with scoring, behavioral or mapping methods, they would be more beneficial. Second, qualitative judgment is more appropriate to deal with radical innovation (Archer & Ghasemzadeh, 1999; Cooper, 1981; Fahrni & Spätig, 1990). For instance, mapping or scoring methods are very useful to identify which projects to cut-off regarding the number of uncertainties and risks. Quantitative methods seem to be more suitable for handling risk (Fahrni & Spätig, 1990), rather than uncertainty in which the probabilities are “unknowable” (Brasil et al., 2018). However, some techniques employed by quantitative methods might handle uncertainties, such as is the case of fuzzy theory (combinatorial optimization) or volatility metrics (real options) (Jafarzadeh, Akbari, & Abedin, 2018; Wang & Hwang, 2007).

Another important aspect to consider when handling uncertain projects is the method’s ability to update knowledge, enabling managers to learn through the experience gained. Real options and probabilistic methods that implement Bayesian network enable the aggregation of information at each stage of the process (Chin, Tang, Yang, Wong, & Wang, 2009). Finally, an interesting technique of trade-off between immunity to uncertainty and aspirations is the information gap theory model (Regan et al., 2005). This method focuses on the identification of knowledge gaps, uncertainties and risks in order to determine the tolerance for undesirable outcomes.

Based on the literature review and the considerations presented above, we identified four key requirements that PPS methods must meet in order to address radical innovation projects. First, Dynamism, is defined as the characteristic that enables managers to permanently adjust the portfolio according to the evolution of the firm’s guidelines (Kavadias & Chao, 2008, p. 139). Second, Interdependencies, is regarded as the interrelations among projects and their impacts on portfolio performance (i.e. synergy or incompatibilities); several authors suggest that the identification of interdependencies is crucial for innovation projects (Verma & Sinha, 2002; Chien, 2002; Archer & Ghasemzadeh, 1999). Third, Uncertainties, is perceived as situations where neither the outcomes nor the odds could be known (Sommer, Loch, & Pich, 2008). The concept of uncertainty is central to radical innovation. Leifer, O’Connor, and Rice (2001) explain that radical innovation processes involve unprecedented situations and features plenty of technical and market uncertainties (p. 103). Finally, Required input data that refer to the kind of data required by the PPS method. Some authors (e.g. Baker, 1974; Fahrni & Spätig, 1990) consider that financial input data (e.g. estimated costs, profit margin, investment, sales, etc.) imprecise because information will never be precisely available. For Ghasemzadeh and Archer (2000), qualitative information is more recommended for assessing radical innovation projects.

Regarding these four key issues and based on the information provided in the studies, we built a framework (Figure 1). To do so, the approaches identified were analyzed by considering the adequacy between the approaches and the four requirements – we developed a scoring model in which each approach received a point or more if such approach is suitable to one characteristic or more (see Table AI). According to this assessment, behavioral methods and information gap theory are the two approaches more suited for radical innovation, followed by real options and integrated methods. The methods less appropriate for radical innovation are financial methods, options-pricing theory and scoring methods; these approaches are more suitable for incremental innovation (Fahrni & Spätig, 1990).

4.2 Integrated portfolio selection method for radical innovation framework

After identifying the advantages and disadvantages of each approach, and identifying the methods more recommended to radical innovation projects, we developed a framework for supporting the PPS selection method according to the innovation level. This framework might help managers to choose and to combine different methods among the alternatives according to a specific innovation level (i.e. radical or incremental innovation).

The framework comprises three phases: ideation phase, which includes the strategic goals, resource constraints and project guidelines; the phase ends with the analysis of the characteristics of the project in which the innovation level is identified. The second phase is the PPS process, where the projects are screened using one or more approaches according to the innovation level. In the last phase, decision makers define the project portfolio based on the recommendations of the selection approaches; this phase also comprises the portfolio adjustments stage, in which managers might include, modify or discard projects in accordance with the environment and the strategic goals. Figure 2 presents the framework suggested.

5. Conclusions and future research

In this paper, we addressed the question of how the selection approaches have evolved to better fit within radical innovation conditions by conducting a systematic literature review. Based on this research approach, we showed how the project selection methods evolved. We present a comprehensive characterization of these methods. We have also identified four key aspects to manage radical innovation portfolios: dynamism, interdependency management, uncertainty treatment and required input data. Based on these findings, we propose a framework for supporting the PPS process according to the innovation level. Moreover, we suggest which PPS approach is more appropriate to incremental and radical innovation projects.

Of course, there is an important opportunity for further studies. We have not discussed in details about how to test our framework. Further studies might test our framework in different industry settings. Also, scholars might test if the methods related to incremental and radical innovation provide superior performance (in relation to the configuration of the portfolio). Scholars might test the best set of methods according to the project’s features (incremental and radical innovation). Finally, scholars might explore empirically how different combinations of methods are more recommended taking into account different industry settings and other features.

Figures

Portfolio selection methods: from incremental to radical innovation

Figure 1

Portfolio selection methods: from incremental to radical innovation

Integrated portfolio selection method for radical innovation framework

Figure 2

Integrated portfolio selection method for radical innovation framework

Number of documents per year and methods categories

Total 1974 1976 1981 1985 1988 1990 1992 1999 2000 2001 2002 2003 2004 2005 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Financial methods 2 1 1
Probabilistic models 1 1
Options-pricing theory 1 1
Strategic models 4 1 1 1 1
Scoring models 3 1 1 1
Comb. optimization 15 1 1 1 1 1 1 1 1 2 1 2 2
Behavioral models 1 1
Mapping approaches 2 1 1 1
Real options 3 1 1
Integrated models 13 1 2 1 2 1 1 1 1 1 2
Info. gap theory 1 1
Scenario-based approach 2 1 1
Total 48 1 1 1 1 1 2 1 2 2 3 4 1 1 3 2 1 1 1 2 2 2 1 3 2 5 2

Methods’ analysis: financial methods

Author (year) Summary Techniques Projects Data required
Bard, Balachandra, and Kaufmann (1988) Projects are assessed in two stages: screened against critical factors and with the other projects. This is achieved through mathematical programming technique that maximizes expected returns Nonlinear integer programming
NPV. Zero-one decision
Decision support system (DSS)
Small computer firm specializing in peripheral equipment Exogenous critical factors (regulations)
Project-related variables
Financial Data
Lal (1974) Projects are selected according to the cost and benefits and several financial portfolio selection techniques IRR
Marginal social cost
Marginal social value
NPV
Value of marginal product
Pareto optimality
Food industry
Trade companies
Several industrial firms
Expected benefits
Financial data
Production and external information (trade)

Methods’ analysis: probabilistic methods

Author (year) Summary Techniques Projects Data required
Chin, Tang, Yang, Wong, and Wang (2009) This methodology assesses new product development risk by Bayesian network with a systematic probability generation and using an NPD project risk network and uncertain variables Directed acyclic graph (DAG)
Conditional probability table (CPT)
Bayesian network
NPD project risk network
Pair-wise comparison matrix
Multinational flashlight manufacturer Project-related variables
Available resources
Conditional probabilities
Expected benefits
Financial data
Strategic goals

Methods’ analysis: option pricing theory

Author (year) Summary Techniques Projects Data required
Galai and Masulis (1976) This method combines the option pricing model (OPM) with the capital asset pricing model (CAPM) to decide about future investments Black-scholes
Capital asset pricing model CAPM
Linear function
Economic return (NPV, IRR, ROI, RAI, PBP and EV)
Cash flow models
European-type options
Two private sector companies Expected benefits
Financial data
Production and external information (trade)

Methods’ analysis: strategic methods

Author (year) Summary Techniques Projects Data required
Cooper et al. (2002) Managers define different resources buckets where projects are sorted and prioritized according to these goals: value maximization, balance (long vs short time, or high vs low risk), strategic direction and the right number of projects Strategic buckets
Familiarity matrix
Several large industrial firms Strategic goals
Product lines
Project type
Scoring criteria
Expected benefits
Financial data
Fahrni and Spätig (1990) The guidelines are concentrated on the most critical problems, the degree of quantification of relevant factors, interdependencies, consideration of single or multiple objectives and degree of risk and uncertainty Decision trees
Checklists
Analytical hierarchy process (AHP)
Economic return (NPV, IRR and ROI)
Simulation
Linear and integer optimization
Mathematical programming
Several Swiss firms Project-related variables
Interdependencies
Expected benefits
Scoring criteria
Zeynalzadeh and Ghajari (2011) The framework considers the firm’s strategy, single project success, portfolio balance and interdependencies. The diamond technique is used to analyze the novelty, technology, complexity and pace Diamond technique
Scoring system
Not informed Project-related variables
Interdependencies
Strategic goals
Financial data
Project success criteria
Project competitive advantage
Jafarzadeh, Tareghian, Rahbarnia, and Ghanbari (2015) This method uses reinvesting strategy within a flexible time horizon. In order to maximize gains, the model aims to optimize the combination of projects, time horizon and schedule of projects Linear and integer optimization
Functions and algorithms
Nonlinear integer programming
Illustrative example Project-related and decision variables
Strategic goals
Time horizon Interdependencies

Methods’ analysis: scoring methods

Author (year) Summary Techniques Projects Data required
Cooper (1981) This method considers the compatibility of resources, the novelty of the project, market competitiveness, technical complexity, competitive strength and economic advantage of the product. It also includes multiple regression analysis to determine the degree of success or failure Checklists
Multiple regression analysis
Cross-split-half method
Test of predictability
Several large industrial firms Project-related variables
Expected benefits
Strategic goals
Available resources
Hall and Nauda (1990) An interactive process for R&D selection based on the judgment of key business and technical elements: this approach emphasizes both corporate strategic planning and customer needs Peer-review/Expert scoring process
Experts/Market/Customer Surveys
Department of Defense Aerospace Contractor Project-related variables
Expected benefits
Available resources
Comparison criteria
Strategic goals
Henriksen and Traynor (1999) PPS method based on the ranking of relevance, risk, reasonableness and return. This technique implements an index algorithm which produces a measure of project value as a function of both merit and cost Scoring algorithm
Peer-review scoring process
Stakeholder inquiry
Delphi technique
Linear mapping
Value index equation
Decision support system (DSS)
Federal Research Laboratory Project-related variables
Expected benefits
Comparison criteria
Strategic goals

Methods’ analysis: combinatorial optimization

Author (year) Summary Techniques Projects Data required
Amiri (2010) This method uses six criteria to compare investment alternatives through an AHP and fuzzy TOPSIS techniques Analytical hierarchy process (AHP)
Fuzzy TOPSIS
Portfolio matrices
Delphi technique
National Iranian Oil Company Project-related variables
Expected benefits
Strategic goals
Financial data
Beaujon, Marin and McDonald (2001) The mathematical model derives from a multidimensional knapsack problem and applies specialized algorithms Mixed-integer programming (MIP)
Linear programming (LP)
Manual solution Heuristics
Sensitivity Analysis
General Motors R&D Center Strategic goals
Available resources
Project-related variables
Expected benefits
Financial data
Doerner, Gutjahr, Hartl, Strauss, and Stummer (2004) The Ant Colony approach imitates the ants’ behavior to create the shortest and most efficient paths. This method enables the identification of the best configuration for the project portfolio Multi-objective combinatorial optimization
Pareto Ant Colony Optimization Algorithm
Quadtree data structure
Pareto Simulated Annealing
Non-Dominated Sorting Genetic Algorithm (NSGA)
Illustrative example Project-related variables
Interdependencies
Expected benefits
Financial data
Strategic goals
Available Resources
Synergies among projects
Loch and Kavadias (2002) The method uses marginal analysis to optimally allocate the scarce budget. It considers multiple interacting factors: interdependencies, common resource pool, increasing or decreasing returns, carry-over of the investment benefit over multiple periods, uncertainty and risks Risk-return matrices
Market segment balances
Functions and algorithms
Weighted average cost of capital (WACC)
Illustrative example based on the diamond industry Project-related variables
Available Resources
Expected benefits
Financial data
Interdependencies
Synergies among projects
Mandakovic and Souder (1985) A hierarchical zero-one model of the R&D budgeting process. The authors suggest a flexible or “in-house” objective function that leaves an open option for the users Multiple-criteria objective functions
Economic Return (NPV, IRR, RA, RR and EV)
Ad hoc objective function
Decomposable interactive algorithm
Hierarchical framework
Random number generator
Illustrative example Project-related variables
Risk coefficient
Expected benefits
Probability of success
Financial data
Available Resources
Weighted contribution per project
Mohanty, Agarwal, Choudhury, and Tiwari (2005) The proposed approach simultaneously addresses the issue of combining both monetary and non-monetary aspects using fuzzy set theory to overcome the vagueness in the preferences Pair-wise comparison matrix
Triangular fuzzy numbers
Analytical network process (ANP)
Fuzzy cost analysis
Fuzzy set theory
Market/customer surveys
Feasibility studies
Fuzzy ANP framework
Cash flow models
Supermatrix
Iron and steel industry in South East Asia Project-related variables
Interdependencies
Expected benefits
Financial data
Strategic goals
Available Resources
Past projects information
Exogenous critical factors
Abbassi, Ashrafi, and Tashnizi (2013) The authors proposed a 0-1 nonlinear mathematical programming method for balancing portfolio following the Markowitz portfolio logic Cross-entropy algorithm
Functions and algorithms
Nonlinear integer programming
Illustrative example Interdependencies
Strategic goals
Project-related variables
Resource constraints
Financial data
Exogenous critical factors
Technological plan
Razi and Shariat (2017) This method consists of hybrid artificial neural network algorithms based on the gray relational analysis (GRA), decision tree and regressions Grey Relational Algorithm
C&R tree (CART) VIKOR
Fuzzy TOPSIS
Decision matrix
Expert opinion questionnaire
Sensitivity analysis
Iran Power Plant Repair Company Strategic goals
Project-related variables
Evaluation criteria and weighting
Ghapanchi, Tavana, Khakbaz, and Low (2012) The model is a mathematical programming approach used to compare the inputs and outputs of decision-making units (DMU) Data envelopment analysis (DEA)
Fuzzy DEA (FDEA)
Illustrative example: information system/information technology (IS/IT) projects Interdependencies
Strategic goals
Evaluation criteria and weighting
Project costs
Contribution margins
Fuzzy risk
Pérez, Gómez, Caballero, and Liern (2018) The method enables the identification of synergies, incompatibilities and other constraints through mathematical programs. It uses fuzzy parameters and adopts several graphics to communicate the results Fuzzy parameters
Mathematical programming
Illustrative example and a real case in a Spanish state university Project-related variables
Financial data
Strategic goals
Political constraints
Available resources
Past projects information
Time horizon
Liesiö, Mild, and Salo (2008) This method supports PPS through multiple evaluation criteria, advanced portfolio-level benefit-cost analysis and interdependencies identification Multi-objective zero-one linear programming
Interval-valued objective function coefficients
Dynamic programming algorithms
Portfolio-level benefit-cost analysis
Multi-objective zero/one linear programming (MOZOLP)
Illustrative example: technology company Evaluation criteria and weighting
Project-related variables
Financial data
Strategic goals
Resource constraints
Hall, Zhuoyu, Qi, and Sim (2015) This approach uses riskiness indexes, integrates correlations and synergies. The selection is centered on target based choice instead of maximizing returns Underperformance riskiness index (URI)
Entropic underperformance riskiness index (EURI)
Benders decomposition techniques
Functions and algorithms
Sensitivity analysis
Illustrative example (200 projects) Projects-related variables
Multiple objectives and constraints
Uncertainties
Interdependencies
Tavana, Khalili-Damghani, and Abtahi (2013) This method is a hybrid fuzzy multidimensional multiple-choice Knap-sack problem (MMKP) that helps to define which project is the most indicated under certain circumstances of limited resources Efficient Epsilon-Constrain (EEC) method
Multi-objective evolutionary algorithm
Data envelopment analysis (DEA)
Fuzzy set theory
Case study: local bank (East Virginia, EEUU.) Resource constraints
Projects-related variables
Strategic goals
Past projects information
Jafarzadeh, Akbari, and Abedin (2018) Aiming to address simultaneously prioritization, uncertainty and interdependencies, the authors develop a model based on three stages: modeling the problem, house of quality (HOQ) and evaluation of maximal portfolio Data envelopment analysis (DEA)
Fuzzy set theory
Quality function development (QFD)
Crisp values
HOQ matrix
Numerical example of a large-scale
Telecommunication Company
Projects-related variables
Strategic goals
Customer requirements
Interdependencies
Risk and dependencies among projects
Financial data
Kalashnikov, Benita, López-Ramos, and Hernández-Luna (2017) This optimization method considers interdependencies among projects, resource limitations and addresses the problem as a Mixed-integer quadratically constrained programming problem Lean Six Sigma
0-1 bi-objective quadratic programming problem
CPLEX
Functions and algorithms
Mixed-integer quadratically constrained programming (MIQCP)
Illustrative example Number of potential projects and related variables
Strategic goals
Resource constraints
Financial data
Interdependencies
Time required by projects

Methods’ analysis: behavioral methods

Author (year) Summary Techniques Projects Data required
Han and Diekmann (2001) Based on the CIA (cross-impact analysis) technique and critical conditions, the interactions among the projects are predicted in order to identify uncertainties and data gaps, enabling decision makers to test combinations to obtain the best project portfolio Cross-impact analysis
Go/No-go decision analysis model
Decision makers group
International government projects Critical conditions
Project-related variables
Rate impact of the decision strategies
Interdependencies

Methods’ analysis: mapping approaches

Author (year) Summary Techniques Projects Data required
Freedman and Desi (2011) The DICE framework assesses the risks and probabilities of success of projects. It is based on these factors: duration of the project, the integrity of the team, commitment to change and the effort of stakeholders. This chart distributes the projects into three areas: comfort, risk and uncertainty BCG’s DICE framework
BCG analysis
Statistical trend analysis
Power plant Project-related variables
Scoring criteria
Expected benefits
Financial data
Customer data
Technical data
Strategic goals
Wheelwright and Clark (2003) This method helps to determine which new projects must be adopted and when, how the project set evolves over time and what role each project should play in the overall development effort. This method is useful to identify the gaps, to balance the portfolio and to reveal where development capabilities should be strengthened Aggregate project plan
Bubble diagram
Project sequence chart
PreQuip, scientific instruments company Project-related variables
Project types
Expected benefits
Strategic goals
Resources required per project

Methods’ analysis: real options

Author (year) Summary Techniques Projects Data required
Luehrman (1998) In this method, the author explores how option pricing can be used to improve decision making about the sequence and timing of a portfolio of strategic investments. Through the option space, managers can select projects according to the value metrics (value-to-cost and volatility) Option pricing
Option space
value-to-cost metric (NPVq)
Volatility metric
Cash flow models
Nested call options
Illustrative example. WeatherIze Corporation (fabrics) Project-related variables
Available resources
Expected benefits
Financial data
Interdependencies
Planning periods
Wang and Hwang (2007) A fuzzy set approach for R&D portfolio selection by using real options valuation model. The objective of this method is to maximize the total benefits of the R&D investment portfolio Fuzzy set theory
Zero-one and probabilistic integer programming
Crisp mathematical model
Option pricing
Fuzzy compound option
Triangular fuzzy numbers
Qualitative possibility theory
Geske compound option
Volatility metric
Newton-Raphson algorithm
An illustrative example from the pharmaceutical industry Project-related variables
Available resources
Expected benefits
Planning periods
Strategic goals
Montajabiha, Khamseh, and Afshar-Nadjafi (2017) The PPS approach applies n-fold compound real options valuations giving flexibility to managers to decide which projects to select. In this method, uncertainty is treated as a range, eliminating the need for statistical distributions Robust combinatorial optimization algorithm
N-fold compound option model
Sensitivity analysis
Functions and algorithms
Robust mixed-integer programming (MIP)
An illustrative example in the pharmaceutical industry Project-related variables
Budget restrictions
Estimated financial data
Past projects information
Strategic goals

Methods’ analysis: integrated methods

Author (year) Summary Techniques Projects Data required Integrated taxonomies
Archer and Ghasemzadeh (1999) A staged and integrated framework where users are able to select the most suitable techniques for each stage. The approach is divided into three main phases: strategic considerations, individual project evaluation, and portfolio selection. It also has a stage of portfolio adjustment whereby users can make some changes Economic return (NPV, IRR and ROI)
CAPM (capital asset pricing model)
Work breakdown structure (WBS)
Monte Carlo simulation
Bayesian statistical theory
Market research
Ad hoc approaches based on scoring
Q-sort
Portfolio matrices
Experimental tests Project-related variables
Scoring criteria
Expected Benefits
Strategic goals
Financial Data
Interdependencies
Strategic
Financial
Scoring
Mapping
Combinatorial optimization
Probabilistic
Behavioral
Chien (2002) The approach has three phases: identification of objectives of the portfolio, construction of scales to measure portfolio attributes, and measurement of alternative portfolios. The model focuses on the identification and exploitation of the interrelations among projects Work breakdown structure (WBS)
Value/utility functions
MAV/MAU models
Linear programming (LP)
An illustrative example based on the project: American stop smoking intervention study by The National Cancer Institute Project-related variables
Scoring criteria
Expected Benefits
Financial data
Strategic goals
Interdependencies
Scoring
Combinatorial optimization
Behavioral
Coldrick, Longhurst, Ivey, and Hannis (2005) This staged method is an extension of Lockwood’s (1999) work in which the selection process is divided into project evaluation (single projects), project selection (examining a range of projects), and application to the different types of research Checklists
Delphi technique
Scoring table layout
Risk/reward assessment
Bubble diagram
Risk analysis
Economic return (NPV, IRR, CBA, DCF and PVI)
Gas turbine project Project-related variables
Scoring criteria
Expected Benefits
Strategic goals
Financial data
Combinatorial optimization
Scoring
Behavioral
Mapping
Financial
Cooper, Edgett, and Kleinschmidt (2000) In this evolved version of the Stage Gate System®, the two-parts decision gates (Go/Kill) were incorporated, and also a formal stage where projects are force-ranked against one another Strategy buckets approach
Bubble diagrams
Prioritized scored list
Economic return (NPV, IRR and EV)
Checklists
Several large industrial firms Project-related variables
Scoring criteria
Expected benefits
Financial data
Strategic goals
Strategic
Financial
Scoring
Mapping
Dutra, Ribeiro, and Carvalho (2014) An economic-probabilistic model that combines financial and probabilistic approaches to consider uncertainties, incorporate multiple criteria, and to adjust the method in response to changes in the business environment Economic return (NPV, IRR, RA, RR and EV)
PayBack period
Monte Carlo
Decision Support System (DSS)
Triangular distribution
Power distribution company Project-related variables
Scoring criteria
Expected benefits
Financial data
Strategic goals
Financial
Probabilistic
Ghasemzadeh and Archer (2000) An evolution of the integrated framework for project portfolio selection (1999). The authors set up the project analysis and selection system (PASS), an organized framework for portfolio selection through a decision support system (DSS) Economic return (NPV, IRR, ROI and RAI)
Analytical hierarchy process (AHP)
Decision support system (DSS)
Group support system (GSS)
Expert choice
Integer linear programming
Ad hoc approaches based on scoring
Gantt Chart
Portfolio matrices
Experimental tests with two high-tech companies Project-related variables
Scoring criteria
Interdependencies
Expected benefits
Strategic goals
Financial data
Strategic
Financial
Scoring
Mapping
Combinatorial optimization
Probabilistic
Behavioral
Linton, Walsh, and Morabito (2002) This dual method uses a multi-criteria decision-making model and a graphical decision support system that enables the integration of objective and subjective methods for analysis and selection. This approach is ideal for reducing a large number of projects Data envelopment analysis (DEA)
Value creation model (VCM)
Parabox representation
Nicheworks depiction
Advanced technology division of bell laboratories Project-related variables
Stage of the intellectual property lifecycle
Expected benefits
Financial data
Strategic goals
Scoring
Mapping
Combinatorial optimization
Loch, Pich, Terwiesch, and Urbschat (2001) This methodology comprises the identification of target contributions, project expectations, and needs. Projects are selected by mixed-integer linear programming. This approach focuses on qualitative data such as the criteria, project contributions and gaps Matrix of evaluation criteria
Mixed-integer programming (MIP)
Linear programming (LP)
Checklists
R&D at BMW Project-related variables
Interdependencies
Scoring criteria
Expected benefits
Financial data
Strategic goals
Scoring
Mapping
Combinatorial optimization
Medaglia, Graves, and Ringuest (2007) An evolutionary method which includes multiple objectives, project interdependencies and linear structure for resource constraints. The method is ideal for partially funded projects, multiple objectives, uncertainties, project interdependencies, and linearly-constrained resources Parameter space investigation method (PSI)
Black Box approach
Pareto solution
Multi-objective evolutionary algorithms (MOEA)
Monte Carlo simulation
Experimental tests Project-related variables
Scoring criteria
Expected benefits
Financial data
Interdependencies
Combinatorial optimization
Probabilistic
Jeng and Huang (2015) The method consists of an integrated approach based on a hybrid MCDM, modified Delphi, a DEMATEL method and an analytic network process. The objective is to overcome the complexity of interdependences and feedbacks between criteria and alternatives Modified Delphi method
Need solution differentiation and benefits (NSDB) framework
DEMATEL method ANP
National research institute Strategic goals
Evaluation criteria
Potential customer requirements
Combinatorial optimization
Strategic
Behavioral
Xidonas, Doukas, Mavrotas, and Pechak (2016) This approach incorporates environmental criteria and uses a bi-objective programming model. In order to handle uncertainties, the model uses the iterative trichotomic approach (ITA) combined with multi-objective problems and Monte Carlo simulations Multi-objective integer programming (MOIP)
Pareto solution
Monte Carlo simulation
Functions and algorithms Stochastic multi-objective acceptability analysis (SMAA)
40 large firms Financial data
Projects-related variables
Multiple objectives and constraints
Conditional probabilities
Expected Benefits
Uncertainties
Combinatorial optimization
Probabilistic
Ei-Kholany and Abdelsalam (2017) This method combines a multi-objective binary cuckoo search (MOCS) algorithm with Monte Carlo simulation to support the PPS decision. Uncertainties are handled via Monte Carlo simulation to translate risk into quantitative data Monte Carlo simulation
MOCS
Fuzzy parameters
Mathematical programming
Functions and algorithms
Meta-heuristic algorithm
Binary cuckoo search
Pareto solution
Illustrative example (40 projects) Criteria and performance values
Project-related variables
Multiple-objective parameters
Combinatorial optimization
Probabilistic
Hu and Szmerekovsky (2017) On the basis of limited resources and the Newsvendor problem perspective, the authors develop a set of procedures and heuristics to provide managerial guidelines in the portfolio selection process. The aim of the model is to find the ideal projects, avoiding cost overruns and budgetary slack Functions and algorithms
Sensitivity analysis
Evolutionary solution method (Excel solver)
Heuristics based on projects ratio and profit ratio
Numerical example Number of potential projects and related variables
Strategic goals
Resource constraints
Financial data
Budget by project
Combinatorial optimization
Strategic

Methods’ analysis: information gap theory

Author (year) Summary Techniques Projects Data required
Regan, Ben-Haim, Wilson, Lundberg, Andelman, and Burgman (2005) This method uses information gap theory to propagate uncertainties, by establishing a tolerance threshold to rank management options and to cut off pernicious options. This approach includes three elements: a mathematical process model, a performance requirement and a model for uncertainty Robustness Function
Decision table (utilities and probabilities)
EV
Robustness curves
Decision problem for conservation area of the Sumatran Rhino Project-related variables
Positive and negative expected outcomes

Methods’ analysis: scenario-based approach

Author (year) Summary Techniques Projects Data required
Liesiö and Salo (2012) This method offers interactive decision support for rejection and selection of projects, which uses scenarios probabilities and utility functions. The method uses MOZOLP to identify the non-dominated portfolios which satisfy criteria such as resource and feasibility constraints Multi-objective zero/one linear programming (MOZOLP)
Functions and algorithms
Experts opinion survey
Risk utility function
Tornado plots
Conditional value at risk (CVaR)
An illustrative example at the International Institute for Applied Systems Analysis (IIASA) Project-related variables
Experts’ estimates (risk preferences/feasible scenarios)
Time horizon
Zhang, Yang, Dou, and Jiang (2016) The authors proposed a scenario-based approach to select army engineering and manufacturing development projects under incomplete scenario information and interdependency constraints. The method is based on a multi-objective programming model Scenario-based programming
Functions and algorithms
Pair-wise comparison matrix
Hidden Markov models (HMMs)
An illustrative example of army engineering and manufacturing development project Experts’ estimates (risk preferences/feasible scenarios)
Financial data
Project-related variables

Detailed values of the innovation adequacy assessment

Document Method Dynamism Interdependencies Input data Uncertainty Total Mean per method
Bard et al. (1988) Financial methods 0 0 0 0 0 0.0
Lal (1974) Financial methods 0 0 0 0 0
Chin et al. (2009) Probabilistic Models 0 0 0 1 1 1.0
Galai and Masulis (1976) Opt. pricing theory 0 0 0 0 0 0.0
Cooper and Kleinschmidt (2002) Strategic models 1 0 0 0 1 2.0
Fahrni and Spätig (1990) Strategic models 1 1 0 0 2
Zeynalzadeh and Ghajari (2011) Strategic models 1 1 0 0 2
Jafarzadeh et al. (2015) Strategic models 1 1 1 0 3
Cooper (1981) Scoring models 0 0 0 0 0 0.0
Hall and Nauda (1990) Scoring models 0 0 0 0 0
Henriksen and Traynor (1999) Scoring models 0 0 0 0 0
Amiri (2010) Combinatorial opt. 0 0 0 0 0 0.9
Beaujon et al. (2001) Combinatorial opt. 0 0 0 0 0
Doerner et al. (2004) Combinatorial opt. 0 1 0 1 2
Loch and Kavadias (2002) Combinatorial opt. 0 1 0 0 1
Mandakovic and Souder (1985) Combinatorial opt. 1 0 0 0 1
Mohanty et al. (2005) Combinatorial opt. 0 0 0 0 0
Abbassi et al. (2013) Combinatorial opt. 0 1 0 0 1
Razi and Shariat (2017) Combinatorial opt. 0 0 0 0 0
Ghapanchi et al. (2012) Combinatorial opt. 0 1 0 0 1
Pérez et al. (2018) Combinatorial opt. 0 1 0 0 1
Liesiö et al. (2008) Combinatorial opt. 0 1 0 1 2
Hall et al. (2015) Combinatorial opt. 0 1 0 0 1
Tavana et al. (2013) Combinatorial opt. 0 0 0 0 0
Jafarzadeh et al. (2018) Combinatorial opt. 0 1 0 1 2
Kalashnikov et al. (2017) Combinatorial opt. 0 1 0 0 1
Han and Diekmann (2001) Behavioral models 1 1 1 1 4 4.0
Freedman and Desi (2011) Mapping app. 1 0 0 0 1 2.0
Wheelwright and Clark (2003) Mapping app. 1 0 1 1 3
Luehrman (1998) Real options 1 1 1 0 3 2.3
Wang and Hwang (2007) Real options 1 0 0 1 2
Montajabiha et al. (2017) Real options 1 0 0 1 2
Archer and Ghasemzadeh (1999) Integrated models 1 1 1 1 4 2.1
Chien (2002) Integrated models 0 1 1 1 3
Coldrick et al. (2005) Integrated models 1 0 0 1 2
Cooper et al. (2000) Integrated models 0 0 0 0 0
Dutra et al. (2014) Integrated models 1 0 0 1 2
Ghasemzadeh and Archer (2000) Integrated models 1 1 1 1 4
Linton et al. (2002) Integrated models 1 1 0 1 3
Loch et al. (2001) Integrated models 1 1 0 0 2
Medaglia et al. (2007) Integrated models 1 1 0 1 3
Jeng and Huang (2015) Integrated models 1 1 0 0 2
Xidonas et al. (2016) Integrated models 0 0 0 1 1
Ei-Kholany and Abdelsalam (2017) Integrated models 0 0 0 1 1
Hu and Szmerekovsky (2017) Integrated models 0 0 0 0 0
Regan et al. (2005) Info. gap theory 1 1 1 1 4 4.0
Liesiö and Salo (2012) Scenario-based 1 0 1 0 2 2.0
Zhang et al. (2016) Scenario-based 1 1 0 0 2

Appendix

Table AI

References

Abbassi, M., Ashrafi, M., & Tashnizi, E. (2013). Selecting balanced portfolios of R&D projects with interdependencies: a cross-entropy based methodology. Technovation, 34(1), 5463.

Amiri, M. P. (2010). Project selection for oil-fields development by using the AHP and fuzzy TOPSIS methods. Expert Systems with Applications, 37(9), 62146224.

Archer, N. P., & Ghasemzadeh, F. (1999). An integrated framework for project portfolio selection. International Journal of Project Management, 17(4), 2072016.

Baker, N. R. (1974). R&D Project selection models: an assessment. IEEE Transactions on Engineering Management, 21(4), 165170.

Bard, J. F., Balachandra, R., & Kaufmann, P. E. (1988). An interactive approach to R&D project selection and termination. IEEE Transactions on Engineering Management, 35(3), 139146.

Beaujon, G. J., Marin, S. P., & McDonald, G. C. (2001). Balancing and optimizing a portfolio of R&D projects. Naval Research Logistics, 48(1), 1840.

Brasil, V., Salerno, M., & Gomes, L. A. V. (2018, July–September). Valuation of innovation projects with high uncertainty: reasons behind the search of real options. Journal of Engineering and Technology Management, 49, 109122.

Briner, R. B., & Denyer, D. (2010), Systematic review and evidence synthesis as a practice and scholarship tool. In D. Rousseau (Ed.), Handbook of Evidence-Based Management: Companies, Classrooms and Research (pp. 112129). Oxford: Oxford University Press.

Chien, C. (2002). A portfolio-evaluation framework for selecting R&D projects. R&D Management, 32(4), 359368.

Chin, K., Tang, D., Yang, J., Wong, S., & Wang, H. (2009). Assessing new product development project risk by Bayesian network with a systematic probability generation methodology. Expert Systems with Applications, 36(6), 98799890.

Coldrick, S., Longhurst, P., Ivey, P., & Hannis, J. (2005). An R&D options selection model for investment decisions. Technovation, 25(3), 185193.

Cooper, R. G. (1981). An empirically derived new product project selection model. IEEE Transactions on Engineering Management, 28(3), 5461.

Cooper, R. G. (1993). Winning at New Products: Accelerating the Process from Idea to Launch. MA: Addison-Wesley.

Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (1999). New product portfolio management: practices and performance. Journal of Production Innovation Management, 16(4), 333351.

Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2000). New problems, new solutions: making portfolio management more effective. Research-Technology Management, 43(2), 1833.

Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2002). Portfolio management-fundamental to new product success. In P. Belliveau, A. Griffin, & S. Somermeyer (Eds), PDMA ToolBook for New Product Development (pp. 331347). New York, NY: John Wiley & Sons.

Doerner, K., Gutjahr, W. J., Hartl, R., Strauss, C., & Stummer, C. (2004). Pareto ant colony optimization: a metaheuristic approach to multiobjective portfolio selection. Annals of Operations Research, 131(1-4), 7999.

Dutra, C., Ribeiro, J. L., & Carvalho, M. (2014). An economic-probabilistic model for project selection and prioritization. International Journal of Project Management, 32(6), 10421055.

Ei-Kholany, M. M. S., & Abdelsalam, H. M. (2017). Multi-objective binary cuckoo search for constrained project portfolio selection under uncertainty. European Journal of Industrial Engineering, 11(6), 818853.

Fahrni, P., & Spätig, M. (1990). An application-oriented guide to R&D project selection and evaluation methods. R&D Management, 20(2), 155171.

Freedman, M., & Desi, R. (2011). Large-Project Management, A Blueprint for Success. The Boston Consulting Group. Available from: www.bcg.com/documents/file76656.pdf

Galai, D., & Masulis, R. W. (1976). The option pricing model and the risk factor of stock. Journal of Financial Economics, 3(1-2), 5381.

Ghapanchi, A., Tavana, M., Khakbaz, M., & Low, G. (2012). A methodology for selecting portfolios of projects with interactions and under uncertainty. International Journal of Project Management, 30(7), 791803.

Ghasemzadeh, F., & Archer, N. P. (2000). Project portfolio selection through decision support. Decision Support Systems, 29(1), 7388.

Hall, D. L., & Nauda, A. (1990). An interactive approach for selecting IR&D projects. IEEE Transactions on Engineering Management, 37(2), 126133.

Hall, N. G., Zhuoyu, D., Qi, J., & Sim, M. (2015). Managing underperformance risk in project portfolio selection. Operations Research, 63(3), 660675.

Han, S. H., & Diekmann, J. E. (2001). Approaches for making risk-based go/no-go decision for international projects. Journal of Construction Engineering and Management, 127(4), 3003008.

Henriksen, A. D., & Traynor, A. J. (1999). A practical R&D project-selection scoring tool. IEEE Transactions on Engineering Management, 46(2), 158170.

Hu, Q., & Szmerekovsky, J. (2017). Project portfolio selection: a newsvendor approach. Decision Sciences, 48(1), 176199.

Jafarzadeh, M., Akbari, P., & Abedin, B. (2018, November). A methodology for project portfolio selection under criteria prioritisation, uncertainty and projects interdependency – combination of fuzzy QFD and DEA. Expert Systems with Applications, 110, 237249.

Jafarzadeh, M., Tareghian, H.R., Rahbarnia, F., & Ghanbari, R. (2015). Optimal selection of project portfolios using reinvestment strategy within a flexible time horizon. European Journal of Operational Research, 243(2), 658664.

Jeng, D., & Huang, K. (2015). Strategic project portfolio selection for national research institutes. Journal of Business Research, 68(11), 23052311.

Kalashnikov, V., Benita, F., López-Ramos, F., & Hernández-Luna, A. (2017, April). Bi-objective project portfolio selection in Lean Six Sigma. International Journal of Production Economics, 186, 8188.

Kavadias, S., & Chao, R. O. (2008), “Resource allocation and new product development portfolio management. In C. H. Loch, & S. Kavadias (Eds.), Handbook of New Product Development Management (pp. 135–163). Oxford: Butterworht-Heinemann.

Lal, D. (1974). Methods of project analysis: A review, New York, NY: International Bank For Reconstruction and Development.

Leifer, R., O’Connor, G., & Rice, M. (2001). Implementing radical innovation in mature firms: the role of hubs. The Academy of Management Executive, 15(3), 102113.

Liesiö, J., & Salo, A. (2012). Scenario-based portfolio selection of investment projects with incomplete probability and utility information. European Journal of Operational Research, 217(1), 162172.

Liesiö, J., Mild, P., & Salo, A. (2008). Robust portfolio modeling with incomplete cost information and project interdependencies. European Journal of Operational Research, 190(3), 679695.

Linton, J. D., Walsh, S. T., & Morabito, J. (2002). Analysis, ranking and selection of R&D projects in a portfolio. R&D Management, 32(2), 139148.

Loch, C. H., & Kavadias, S. (2002). Dynamic portfolio selection of NPD programs using marginal returns. Management Science, 48(10), 12271241.

Loch, C. H., Pich, M. T., Terwiesch, C., & Urbschat, M. (2001). Selecting R&D projects at BMW: a case study of adopting mathematical programming models. IEEE Transactions on Engineering Management, 48(1), 7080.

Lockwood, C. (1999). Comparison of average-passage equation closures through simulation of single and multi-row axial compressors; the limitations of using a commercial CFD code (PhD thesis). Cranfield University.

Luehrman, T. (1998). Strategy as a portfolio of real options. Harvard Business Review (pp. 89–99). September–October.

Mandakovic, T., & Souder, W. E. (1985). A flexible hierarchical model for project selection and budget allocation. R&D Management, 15(1), 2329.

Medaglia, A. L., Graves, S. B., & Ringuest, J. L. (2007). A multiobjective evolutionary approach for linearly constrained project selection under uncertainty. European Journal of Operational Research, 179(3), 869894.

Mohanty, R. P., Agarwal, R., Choudhury, A. K., & Tiwari, M. K. (2005). A fuzzy ANP-based approach to R&D project selection: a case study. International Journal of Production Research, 43(24), 51995216.

Montajabiha, M., Khamseh, A., & Afshar-Nadjafi, B. (2017). A robust algorithm for project portfolio selection problem using real options valuation. International Journal of Managing Projects in Business, 10(2), 386403.

Mun, J. (2005). Real Options Analysis: Tools and Techniques. NJ: Wiley.

Pérez, F., Gómez, T., Caballero, R., & Liern, V. (2018, June). Project portfolio selection and planning with fuzzy constraints. Technological Forecasting & Social Change, 131, 117129.

Razi, F. F., & Shariat, S. H. (2017). A hybrid grey based artificial neural network and C&R tree for project portfolio selection. Benchmarking: An International Journal, 24(3), 651665.

Regan, H., Ben-Haim, Y., Wilson, W., Lundberg, P., Andelman, S., & Burgman, M. (2005). Robust decision-making under severe uncertainty for conservation management. Ecological Applications, 15(4), 14711477.

Rice, M. P., O’Connor, G. C., & Pierantozzi, R. (2008). Implementing a learning plan to counter project uncertainty. MIT Sloan Management Review, 49(2), 5462.

Shoemaker, P. (1995). Scenario planning: a tool for strategic thinking. MIT Sloan Management Review, 36(2), 2540.

Sommer, S. C., Loch, C. H., & Pich, M. T. (2008). Project risk management in new product development. In C. H. Loch, & S. Kavadias (Eds), Handbook of New Product Development Management (pp. 439465). Oxford: Butterworht-Heinemann.

Tavana, M., Khalili-Damghani, K., & Abtahi, A. (2013). A fuzzy multidimensional multiple-choice knapsack model for project portfolio selection using an evolutionary algorithm. Annals of Operations Research, 206(1), 449483.

Verma, D., & Sinha, K. (2002). Toward a theory of project interdependencies in high tech R&D environments. Journal of Operations Management, 20(5), 451468.

Wang, J., & Hwang, W. (2007). A fuzzy set approach for R&D portfolio selection using a real options valuation model. The International Journal of Management Science, 35(3), 247257.

Wheelwright, S.C., & Clark, K.B. (2003). Creating project plan to focus product development. Harvard Business Review (pp. 1–14). September–October.

Xidonas, P., Doukas, H., Mavrotas, G., & Pechak, O. (2016). Environmental corporate responsibility for investments evaluation: an alternative multi-objective programming model. Annals of Operations Research, 247(2), 395413.

Zeynalzadeh, R., & Ghajari, A. (2011). A framework for project portfolio selection with risk reduction approach. African Journal of Business Management, 5(26), 1047410482.

Zhang, P., Yang, K., Dou, Y., & Jiang, J. (2016). Scenario-based approach for project portfolio selection in army engineering and manufacturing development. Journal of Systems Engineering and Electronics, 27(1), 166176.

Further reading

Howell, D., Whindahl, C., & Seidel, R. (2010). A project contingency framework based on uncertainty and its consequences. International Journal of Project Management, 28(3), 256264.

Lynn, G. S., Morone, J. G., & Paulson, A. S. (1996). Marketing and discontinuous innovation: the probe and learn process. California Management Review, 38(3), 837.

Pich, M. T., Loch, C. H., & De Meyer, A. (2002). On uncertainty, ambiguity, and complexity in project management. Management Science, 48(8), 10081023.

Shane, S., & Ulrich, C. (2004, February). Technological innovation, product development, and entrepreneurship in management science. Management Science, 50, 133154.

Corresponding author

Ximena Alejandra Flechas Chaparro can be contacted at: xaflechas@gmail.com
Associate Editor: Felipe Lara