Integrating performance measurement, system dynamics, and problem-solving methods

Abstract

Purpose

Transport logistics systems in companies with additional public service roles are complex and could benefit from new approaches to performance management. Existing approaches tend to be fragmented; thus, the purpose of this paper is to integrate balanced performance measures, a dynamics model, and the problem-solving method into a new model.

Design/methodology/approach

An integrated framework is developed by reviewing literature and synthesising attributes of performance measurement systems, system dynamics and problem-solving methods. The framework is then applied to a multiple-role company’s sea transportation system. The study uses statistical methods to identify performance indicators, management interviews with document study to develop a dynamics model, and simulation methods to formulate an improvement plan.

Findings

The performance measurement design stage allowed for the identification of balanced, aligned performance indicators, while the system dynamics model illuminated the impact of the system components’ interrelationships on performance output. The problem-solving method allowed for analysis of system performance, identification of constraints and formulation of a performance improvement plan.

Practical implications

This framework can help transport logistics system stakeholders in multiple-role companies avoid silo thinking, misaligned performance objectives, local optima and short-term solutions.

Originality/value

This study contributes to the existing body of research by introducing a novel framework integrating performance measurement, system dynamics and the problem-solving method. It also addresses a theoretical gap by showing how interconnecting components of sea transportation systems affect transport logistics performance.

Citation

Irfani, D.P., Wibisono, D. and Basri, M.H. (2020), "Integrating performance measurement, system dynamics, and problem-solving methods", International Journal of Productivity and Performance Management, Vol. 69 No. 5, pp. 939-961. https://doi.org/10.1108/IJPPM-12-2018-0456

1. Introduction

Transport logistics performance in multiple-role companies involves very complex systems. First, there are multiple parties involved – shippers, consignees, transport providers and so forth – whose interests often conflict (Lai, 2002). Furthermore, financial and social systems may also have conflicting goals (Almonte et al., 2017). In addition, multiple-role companies are accountable to state or municipal administrative authorities; they must conform to political party, professional, societal and legislative rules; and they must adapt to any changes in regulation (Christensen and Laegreid, 2007; Swiatczak et al., 2015). Bai and Xu (2005) explain that the measurement of managerial performance in multiple-role companies is complicated and that a gap still exists, internationally, in this research area.

This complexity may pose difficulties for decision-makers. First, decision-makers may struggle to select the most appropriate performance indicators from a long list of possibilities (Khan and Wibisono, 2008). Poorly chosen indicators may create a myopic view, leading to local optima or misaligned solutions. Poor indicators may also mislead involved parties into focusing only on short-term improvement efforts (Neely et al., 1995). In addition, conflicting financial and social targets can encourage silo thinking, which in and of itself can hamper system performance. As a result, decision-makers may fail to understand the true drivers of system performance, that is, what is at the root of poor performance and how to address it (Santos et al., 2001).

To avoid these pitfalls, a contextual performance management approach is needed. Cuthbertson and Piotrowicz (2011) describe performance management as a context-dependent process, tailored to specific requirements. Similarly, Lye (2004) suggests that designing performance management systems require an examination of the rich interdependencies between contextual factors and performance measures. In this case, to avoid misaligned performance, performance indicators must be selected carefully. To manage the trade-off between financial and reliability issues, dynamic linkages between performance indicators must be visible to all relevant parties. This visibility drives collaboration in the system, which, in turn, catalyses improvement initiatives whose optimisation is systemwide rather than local. To identify a systemic problem and formulate performance improvement plans, problem-solving methods can be utilised. These three dimensions together are required for comprehensive management of transport logistics system performance in multiple-role companies.

However, the number of performance management frameworks that integrate the three dimensions – performance measurement design, system thinking and problem solving – is very limited. Instead, current models tend to be fragmented, overall. To design performance measurement systems, logistics system decision-makers may adapt existing performance measurement frameworks, such as the balanced scorecard (BSC) of Kaplan and Norton (1992); the performance prism of Neely et al. (2002); knowledge-based performance measurement systems of Wibisono (2003); the supply chain operation reference (SCOR) framework; Lean Six Sigma Logistics (Goldsby and Martichenko, 2005); and others. But these frameworks can only guide the design of simple performance measurement systems, and their ability to explain dynamic relationships among system components is limited. Atkinson et al. (1997) suggest that BSC is intrinsically static. Similarly, Brown (2000) criticises BSC for failing to address interrelationships among variables and for its inability to predict the impact of lagging indicators on leading ones. Additionally, Norreklit (2000) also criticises BSC for neglecting causality in the relationships between different measures. Akkermans and Oorschot (2002) find fault with BSC’s strategic map, pointing out its unidirectionality, simplicity and failure to separate cause and effect in time.

Among several available models, system dynamics is the one that allows for visualisation of the dynamic relationships between system components in a non-linear way. In addition, system dynamics permits simulations to demonstrate the potential impact of a particular policy on system behaviour. Despite the fact that some of its features do address the needs of transport logistics systems, system dynamics has some limitations. Its main focus is on understanding system behaviour, and it does not provide explicit, practical guidance for how decision-makers can improve a system’s performance. On the other hand, problem-solving approach like Theory of constraint (TOC) can be used to analyse complex problems and develop solutions (Mabin and Balderston, 2000). In sum, there are existing frameworks to help decision-makers formulate performance indicators, model dynamic relationships between system components, and improve performance. Nonetheless, these frameworks are fragmented, and more integrated ones are currently not available.

To help transport logistics system decision-makers in multiple-role companies more effectively manage performance, holistic performance management systems are needed. Addressing this gap, however, requires new research. More specifically, research must address the need for a transport logistics performance management framework that integrates performance measurement with both the system dynamics model and the problem-solving principle. In addition, researchers must explore how transport logistics system components (in multiple-role companies) interconnect and interact to affect logistics performance. Figure 1 illustrates this need for further research in logistics performance measurement and especially, in the context of companies with multiple roles.

2. Literature review

2.1 The use of system dynamics in performance management

On several occasions, system dynamics has been used to manage the performance of logistics and supply chain systems. Wu et al. (2006) examine the dynamic behaviour of a supply chain by constructing a system dynamics model. Their research reveals that the proposed model can evaluate supply chain performance and be used as a consultation tool for forecasting policy in the real business world. Additionally, Rafele and Cagliano (2006) use system dynamics to connect the performance of a company and its suppliers through homogeneous indicators.

On a broader scale, Bianchi and Rivenbark (2012) show that system dynamics can be used to help decision-makers move from the measurement of performance to its management. According to them, system dynamics modelling can play an important role in improving the quality of organisational performance management systems and supporting decision making. Bianchi and Rivenbark reveal that system dynamics can be used to frame the trade-offs in time and space associated with several alternative scenarios, understand how the accumulation and depletion of strategic assets are impacted by different policy levers, and determine how performance drivers affect end results.

Several efforts have also been made to synthesise a system dynamics approach to performance evaluation. Santos et al. (2001) argues that integrating system dynamics with multi-criteria decision analysis (MCDA) can address issues of interrelationships. It can also address trade-offs between performance measures and consider the dynamism of organisations and measurement systems. According to Santos, the complementary strengths of the system dynamics model and MCDA can be combined to form a valuable tool for understanding and making informed decisions about organisational performance. In his next publication, Santos et al. (2002) reveal that employing system dynamics and MCDA provides a means of studying explicit trade-offs between performance measures and assessing the impacts of initiatives to improve performance. By using a case study related to a service-based business, Bernabe (2011) demonstrates that matching the traditional BSC architecture with system dynamics principles offers better support for strategic management decisions. Their finding suggests that a BSC based on system dynamics allows for exploration and understanding of the complexity and dynamics of a system. Additionally, combining these two tools facilitates organisational learning, policy design, and managerial strategic analysis. Bernabe also shows that using system dynamics’ modelling principles resolves some of the limitations of the original BSC framework. The aforementioned studies suggest that, to effectively manage performance, system dynamics can be integrated with other tools. However, these studies have not explicitly shown how these integrated approaches can address the risk of local optima problems and short-term solutions as well as enhance alignment between strategic performance, tactics and operations.

2.2 Logistics performance

Logistics performance is defined as the ability of a system to deliver the right product to the correct location at the appropriate time and at the lowest cost (Zhang and Okoroafo, 2015). Similarly, Leończuk (2016) defines logistics performance as the ability of the entire system to meet end-customer needs through ensuring the appropriate inventory levels and through ensuring the delivery of the product on time and in the right way. Huber et al. (2015) define logistics as the process of planning, implementing and controlling procedures for the efficient and effective transportation and storage of goods, services and related information in a way that satisfies customer requirements. Logistics, then, covers the whole transport chain from the points of production to consumption and includes inbound, outbound, internal and external flows of goods and services. In sum, according to all these definitions, logistics performance refers to an ability to satisfy customer needs, ensure product availability and on-time delivery and maintain cost-efficiency.

Wisner (2003) hypothesised that logistics performance is a positive predictor of organisational performance. Kumar and Nambirajan (2013) developed an integrated model to assess the causal linkage among critical components of logistics systems and impacts on organisational performance. Their results show that critical logistics components affect supply chain performance, which, in turn, influences organisational performance. In all, these studies show that logistics performance plays an important role in both supply chain and overall organisational performance.

Rezaei et al. (2018) explained that logistics performance can be measured on multiple scales, including on a micro-scale at enterprise level and on a macro-scale at the national level. Since supply chain and logistics are closely related, some researchers have experimented with the use of supply chain performance measurement frameworks to manage logistics performance. In 1996, the Supply Chain Council proposed the SCOR, which comprises five core supply chain performance attributes: reliability, responsiveness, agility, costs and asset management.

Several authors have proposed other frameworks for managing logistics performance. Mentzer and Konrad (1991) defined logistics performance as effectiveness and efficiency in delivering logistics services. In logistics, effectiveness has been described as the ability to achieve the pre-defined objectives, for example, to meet customer requirements in certain critical result areas. On the other hand, efficiency is categorised as the ratio of resources utilised against the results derived (Mentzer and Konrad, 1991). Gunasekaran et al. (2001) also suggested using effectiveness and efficiency dimensions for measuring logistics performance.

Besides effectiveness and efficiency, other authors have proposed various dimensions for measuring logistics system. Langley and Holcomb (1992) added logistics differentiation as the key element in logistics performance because the value customers receive from logistics activities also serves as an indicator of logistics performance. Bowersox et al. (2000) suggested speed, delivery dependability, delivery flexibility, responsiveness and order fill capacity to measure logistics performance. Green et al. (2008) explained that a logistics performance construct reflects the organisation’s performance as it relates to its ability to deliver goods and services in precise quantities and time required by customers.

2.3 Transport logistics performance

In a supply chain, transport logistics facilitates the physical flows of goods from origin to destination (Lai, 2002). Lai asserts that the goal of a transport logistics service provider is to satisfy both upstream and downstream customers more effectively and efficiently than competitors. In this case, Lai proposes three measures for transport logistics performance: the service effectiveness of shipper operations; the efficiency of transport logistics service providers; and the service effectiveness of consignees. Kleinsorge et al. (1991) agree that supply chain performance and transport logistics should emphasise not only operations-efficiency parameters but also service effectiveness ones. Panayides (2006) and Huber et al. (2015) concur with the idea that transport logistics systems aim to satisfy both upstream and downstream customers and to do so with acceptable costs.

As the objective of transport logistics is customer satisfaction, performance can be evaluated by using the concept of service quality. Parasuraman et al. (1988) argue that the gap between performance and expectation (disconfirmation) is the basis for measuring service quality. Similar to Parasuraman et al. (1988), Gronroos (1984) proposes that service quality is assessed based on a comparison of expectation and perception of performance. In contrast to Parasuraman et al. (1988), Teas (1993) argues that service quality is derived from a comparison of performance with ideal standards. Cronin and Taylor (1992) argue that importance-weighted assessment may not be necessary, as performance alone appears to be effective in defining service quality. All these perspectives indicate that performance of transport logistics can be evaluated in multiple ways, either by comparing the perception of performance with expectations and standards or by using the perception of performance alone.

Based on the concept of service quality, several authors have elaborated the dimensions for measuring the performance of transport logistics. Lai (2002) proposes three dimensions to measure the performance of transport logistics, namely service effectiveness for shippers, operational efficiency for transport logistics service providers, and service effectiveness for consignees. Schönsleben (2007) suggests capacity and on-time delivery for measuring transport logistics performance. Krauth et al. (2005) propose several indicators, including distance travelled per day, delivery frequency, and vehicle loading capacity utilised. Garcia et al. (2012) advocate the use of transport correctness and completeness and loading/unloading duration for measuring transport logistics performance.

Several studies have focused on identifying factors that affect transport logistics systems performance. Micco and Perez (2002) explored these factors by sifting through more than 300,000 pieces of annual shipping data in several international ports. According to Micco and Perez, distance and process efficiency in ports affect the performance of a transport logistics system. In addition, Gkonis and Psaraftis (2004) conducted literature studies and identified the following as important variables: ship capacity, vessel speed, port time, cargo volume, cargo space utilisation, port infrastructure capacity, and congestion. Caldeirinha et al. (2013) used data envelopment analysis (DEA), factor analysis and linear regression to determine factors that influence system performance. They identified the number of berths and the quality and quantity of port infrastructure as factors influencing port and, ultimately, transport logistics performance.

2.4 Theory of constraint

Groop (2012) showed that TOC can serve as a systematic framework for identifying and resolving factors that hinder productivity in the service sector. Besides Groop, Ellis (2011) also applied TOC successfully in the service sector to increase operational and financial performance. Mulyono et al. (2016) applied a TOC-based framework to identify and manage the constraints in a sea transportation system. In general, TOC seems to offer a systematic mechanism for performance improvement, but it does not provide practical steps or tools for measuring and modelling system components.

3. Methodology

3.1 Framework building

To build a new framework integrating performance measurement, system dynamics and the problem-solving principle, this study employed literature review and case study methods. The literature review covered articles on the attributes of performance measurement frameworks, system dynamics, TOC, logistics systems, supply chain management and multiple-role companies. These articles were collected from the Scopus, ProQuest and Web of Science databases, and the initial search identified 569 unique articles. The abstracts were reviewed against the following initial inclusion criteria:

1. the articles described mechanisms to manage supply chain or logistics performance;

2. they described methods or case studies related to performance measurement frameworks, system dynamics or TOC;

3. they described characteristics of organisational, logistics or supply chain performance relevant to the context of multiple-role companies; and

4. they cited the performance measurement framework presented in this paper.

Based on these criteria, 394 papers qualified for the next stage of the process, which was a full paper review. Performance attributes were analysed using memo-writing techniques to identify each article’s main values and ideas. Of those papers, 65 were considered appropriate for this study. Figure 2 shows the publication date distribution of reviewed articles.

Performance attributes of system dynamics, TOC and PMS frameworks relevant to the context of logistics systems were then reconstructed into an integrated framework. Theoretical memos were used to identify relationships between performance attributes. Additionally, operational memos were used to determine the direction of process linearity within the new PMS framework.

Once the framework was developed, it was applied to a sea transportation system in a case company to test for applicability. The framework was considered applicable if it could do the following:

1. identify and select the performance indicators of the transport logistics system on the strategic, tactical and operational layers;

2. represent the dynamic relationships among components in the sea transportation system and their linkages to end results;

3. identify the main problems in the system and factors causing the problems; and

4. simulate in several scenarios of policy changes and analyse impact on system performance.

In addition to document studies, data were collected through interviews with case company management teams and analysts. The target interviewees were from the following categories:

1. employees who were responsible for the operation of logistics systems, including those who worked in network configuration management or in transport, planning and sourcing departments;

2. employees who were responsible for the performance measurement system; and

3. management teams.

3.2 Integrated logistics performance measurement framework

This paper proposes a new performance management framework: IntegRated Framework for ANalysing and Improving (IRFANI). IRFANI Performance integrates the concepts of performance measurement, system dynamics and TOC-based problem solving as shown in Figure 3.

The explanation of each step in the IRFANI Performance is as follows.

Stage 1: performance measurement design

This stage focused on selecting performance measures relevant to the context of logistics systems. Performance indicators should be aligned on strategic, tactical and operational levels. In addition, selected performance measures must encompass both financial and service quality dimensions. Lastly, selected indicators are expected to measure not only short-term but also long-term performance. Some proposed indicators for measuring the performance of transport logistics system are shown in Table I.

In addition to the indicators collected from the literature as listed in Table I, contextual performance indicators can be gathered from the field if necessary. After the identification of potential indicators, the final key indicators need to be selected. Key indicators, in this case, are selected based on relevance and importance. Data can be collected through interviewing stakeholders from the system or by questionnaire. Once the key indicators are selected, they can be grouped according to category of strategic resources, performance drivers, and performance output.

Stage 2: system dynamics modelling

After selecting key indicators for strategic resources, performance drivers and performance outputs, the next step is building a dynamic non-linear model to represent how the components in the logistics system interconnect and interact to affect performance. The proposed step-by-step approach for modelling is as follows:

1. Examine the interrelationships among system components or system variables: this can be done through studying previous literature, analysing historical data using statistical methods, interviewing key stakeholders, and employing MCDA tools such as analytical hierarchy process (AHP), analytical network process (ANP) or decision making trial and evaluation laboratory (DEMATEL).

2. Building system dynamics models: causal loop or stock and flow diagrams can be used to model component interrelationships in the logistics system. To make it easier for decision-makers to trace factors that cause poor performance, indicators and system components can be positioned by category (strategic resources, performance drivers, and performance output) as shown in Figure 4.

3. Develop simulation tools to show decision-makers the impacts of policy changes on future system performance.

Stage 3: problem solving

After developing the dynamic model to represent the inner workings of the logistics system, the next step is to formulate the problem-solving scenario. The proposed step-by-step approach for this stage is as follows:

1. Analyse system performance.

2. System performance can be analysed by running the base model. The parameters used to run the model are actual parameters that represent the existing condition in the system.

3. Identify system constraints.

4. This step adapts the first step in the Process of Ongoing Improvement (POOGI) framework for the TOC. A constraint is a component or factor that prevents the system from achieving its main goal. Constraints can be identified by conducting simulations using current or future parameters.

5. Develop scenario to improve performance.

6. This step can be performed in several ways, including through exploiting constraint utilisation, conducting resource subordination or increasing constraint capacity.

7. Simulate the effectiveness of each scenario and select the best scenario.

4. Results and discussion

The integrated framework developed in this study was applied to the logistics system in Company A, a state-owned enterprise (SOE) in the oil and gas industry. Company A is an integrated business, including upstream activities such as exploration and production and downstream activities including processing, distribution and sales. Company A produces and sells a wide range of oil and gas products, including crude oil, refined fuels, liquefied petroleum gas, aviation gas, etc. To distribute its products, Company A uses sea, land and air transportation. As a National Oil and Gas SOE, Company A receives mandates from the government to distribute subsidised products to all areas of the country in which it operates. At the same time, the company also has a duty to generate profits for its shareholders. Its dual roles as both profit generator and public service provider complicate its logistics system. The integrated framework in this case is used to design, model, and simulate improvement efforts for Company A’s sea transportation system.

4.1 Stage 1: performance measurement design

To support distribution, Company A operates about 300 tankers and 100 specialised ports. The company also has departments with internal shippers, vessel operators, network planners, port operators, consignees and asset managers. Since the company is both a public service provider and profit generator, the evaluation of its performance is based on two main perspectives: financial and service quality dimensions. To measure these dimensions, key performance indicators were selected through interviews with the company’s management teams. The selection criteria for indicators were their relevance and importance as perceived by the management teams. Interviews were carried out with 24 mid-level managers in departments related to logistics system and performance evaluation. The results of key indicator selection are shown in Table II.

4.2 Stage 2: system dynamics modelling

In general, Company A’s sea transportation system is driven by shipment demand. Rate of shipment demand is from the accumulation of the volume of cargo to be shipped, which is a stock variable whose value decreases every time the system successfully transports cargo. If the system fails to transport, the volume of this unshipped cargo is stored in the rate variable, unlifted cargo. If in a certain period of time, unlifted cargo days exceed inventory coverage days, product stockout at the end channel occurs. This ultimately affects the service level of the system. The basic concept of the company’s sea transportation system is modelled in the stock and flow diagram is shown in Figure 5.

To fulfil shipment demand, the sea transportation system requires vessel resources, which in turn affect the amount of operating budget. Thus, decision-makers need a model that explains the relationships among factors related to the number of required vessels and the operating budget. These relationships are shown in Figure 6.

The stock and flow diagram in Figure 6 shows that shipment demand, effective load factor (ratio between utilised capacity and carrying capacity), and carrying capacity are variables that affect voyage needs. Next, voyage needs, commission days, and round-trip days (duration required for a vessel to go on a journey and return to where it started from) determine the number of vessels needed to meet shipment demand (vessels required). By using vessel operating cost and bunker cost variables, required operating budget can be obtained. The stock and flow diagram in Figure 6 also explains in more detail that round-trip days of transportation are influenced by factors such as vessel loading duration, discharging duration, sailing duration (laden and ballast), and idle duration, which, in turn, are affected by operational parameters such as shore pumping rate, vessel pumping rate, vessel speed (laden and ballast) and jetty capacity. The factors port draft and vessel draft affect vessel capacity utilisation (effective load factor). Lastly, the diagram indicates that the duration of vessel breakdown and vessel maintenance determines vessel operating days.

After obtaining the operating budget, decision-makers need a system dynamics model that explains the relationships between the operating budget and the performance output variables. The linkages between the operating budget and performance output are mediated by several performance drivers, which are shown in Figure 7.

Figure 7 illustrates that the requirements of the operating budget affect the injected operating budget. Then, the injected operating budget is the input variable for the stock variable called available operating budget, the value of which decreases as the budget is used for operating vessels. The amount of budget absorbed depends on the number of operated vessels and several other parameters, such as vessel charter rate, commission days, bunker consumption rate and bunker price. The actual number of operated vessels and the vessel carrying capacity will affect the volume of cargo that can be shipped by the system. The volume of cargo transported determines the company’s revenue, and ultimately, the company’s financial profit is derived by deducting operating budget used from revenue. By outlining these relationships, the system dynamics model can link the system’s resource category components with performance drivers and outputs.

Outlining the key factors that impinge on a system’s financial performance is helpful for decision-makers, but it is not enough. As explained in the previous section, companies with multiple roles also need to evaluate system reliability, which in this case refers to the system’s ability to avoid product stockout at the end channel as well as its ability to deliver product on time. Stock and flow diagrams that describe relationships between system components underlying the timeliness of transport delivery are shown in Figure 8.

From Figure 8, it can be seen that the system’s reliability is affected by service level and on-time delivery. Service level has been explained in Figure 5. On-time delivery is influenced by actual round-trip days and standard round-trip days. Factors that affect actual round-trip days were explained in Figure 6. Standard round-trip days can be calculated by using standard parameters, such as standard speed, standard vessel pumping rate, and standard shore pumping rate. In this case, the higher the ratio between actual round-trip days and standard round-trip days, the better the on-time delivery performance.

A performance management system in multiple-role companies is expected not only to evaluate performance but also to support decision making, especially when it is related to performance improvement efforts. Therefore, the model musts show the dynamic causal relationships underlying the use of the capital budget to improve system performance. Decision-makers in this case can decide what percentage of the capital budget to inject to increase infrastructure capacity, which, in turn, can improve performance. Figure 9 is the model that shows the relationship between capital budget accounts and performance output.

Figure 9 indicates that management policy for allocating capital budget affects the available capital budget. Decision-makers then have the option whether to spend the available capital budget. Should the decision-makers decide to use the capital budget, they have these alternatives:

1. use the budget for dredging to increase effective load factor;

2. use the budget to increase jetty capacity to decrease vessel idle duration at port; and

3. use the budget to increase pump capacity to reduce vessel loading duration.

An increase in effective load factor reduces the number of vessels required, which, in turn, lowers operating cost. Similarly, a decrease in vessel idle duration and loading duration decreases the round-trip day, which in the end increases on-time delivery performance.

4.3 Stage 3: problem solving

After developing models to represent the dynamic relationships among components and their linkages with performance output, the next step is to analyse the system performance, identify problems and the causes of problems in the system, and formulate a performance improvement plan. The applications of these steps in Company A are as follows.

Analyse system performance and identify system constraints

The developed framework is applied to the crude oil transportation system in one of the company’s operating regions, where shipment demand is around 100m barrels of crude oil per year. Since the company receives mandates from the government to fulfil shipment demand, it needs to provide a sufficient number of vessels. Currently, the company operates five large range tankers and manages several berths that can accept those vessels. According to historical data, the average annual growth rate of crude oil shipment demand is around 7.27 per cent. Using all of the existing parameters, the reliability of the company’s distribution system for the next 10 years can be simulated, as shown in Figures 10 and 11.

Figure 10 shows that the company has to maintain its service level at 100 per cent, since it has to meet the entire shipment demand. This means that the company needs to provide as many vessels as needed. However, if current conditions prevail, on-time delivery will decrease from 82 per cent in the first year to 45 per cent in the tenth year. Figure 11 shows that what predominantly causes poor performance is an increase in vessel idle duration. In this case, the vessel becomes idle at port when the jetty capacity is less than the number of vessels operated by the company. If the company’s jetties are only able to accept five vessels but starting in the third year, the company will need to operate more than five vessels, vessel idle duration will increase in the future. Under these conditions, berth capacity becomes one of the constraints that limit the future performance of the system.

All of these steps demonstrate the ability of the integrated framework to comprehensively analyse system performance as well as to identify problems and their underlying causes.

Improve system performance

After determining that current jetty capacity will not be able to accommodate future shipment demands, which will also reduce system reliability, the next step is to formulate an improvement plan. A jetty capacity upgrade requires financial resources and a substantial amount of time to evaluate its effectiveness. Therefore, decision-makers could benefit from a simulation tool to more thoroughly assess the upgrade’s impact on system performance. Since Company A has multiple societal roles, the benefit of the investment must consider both financial performance and reliability. The simulation results of the jetty capacity upgrade are shown in Figures 12–14.

Figures 12 and 13 show that the jetty capacity upgrade improves the system’s reliability by reducing vessel idle duration. In addition, the upgrade also reduces the unit transportation cost. Figure 14 shows that the financial benefit of the jetty capacity upgrade starts to appear after the third year. Upgrading jetty capacity by more than one vessel will become effective after the fifth year. Lastly, upgrading jetty capacity by three more vessels becomes superior in the eighth year.

This paper proposes the use of a ratio of the benefit of the capacity upgrade to the investment costs to evaluate the effectiveness of each scenario. The three-scenario comparison is illustrated in Table III; upgrading jetty capacity by three more vessels will bring the best return for the company. Results may differ when more scenarios are added, and different time frames are used. Overall, though, this exercise demonstrates that the combination of performance measurement design, system dynamics modelling, and problem-solving principle can enable decision-makers to formulate system improvement plans and then select the optimal scenario.

The integrated framework proposed in this study offers a mechanism for decision-makers to comprehensively plan and evaluate the transport logistics performance in multiple-role companies. The performance measurement design stage, in this case, helps decision-makers to select balanced performance indicators. The selected performance indicators consist of measures for evaluating both roles of profit generators and public service providers. Meanwhile, the dynamic linkages between strategic resources, performance drivers and performance output in the system dynamic models help decision-makers in multiple-role companies to understand how the logistics system works and assess the overall performance of the system. Decision-makers can utilise the models for supporting resource allocation activities, especially those related to capital and operational budgeting. Additionally, the problem-solving stage helps decision-makers to identify the system problems, analyse factors that lead to poor performance, and formulate scenarios for improving logistics performance. Most importantly, the dynamic linkages presented in this study help decision-makers to balance the cost and reliability aspects of logistics performance as the linkages help them to show the relationships between conflicting performance indicators.

The integrated framework, in this case, differs from some existing models. In terms of trade-off management, the proposed model has several advantages over the SCOR framework. In this case, the proposed framework can show the dynamic linkages between conflicting performance indicators and can frame the trade-offs in time and space associated with several alternative scenarios. On the other hand, as SCOR focuses on the standard descriptions of management processes and standard metrics to measure process performance (Chen and Huang, 2007), its capability in establishing the dynamic linkages between conflicting performance indicators is limited.

5. Conclusion

Transport logistics systems in multiple-role companies are complex and unique systems that require tailored approaches to performance management. These companies’ decision-makers confront conflicting goals in financial and social dimensions as well as constantly changing regulations. Not managing these issues properly may detract from overall system performance. To avoid this, integrated as opposed to simple performance measurement systems are needed.

Responding to this need, this study proposes a new model for performance management that integrates performance measurement, system dynamics and the problem-solving principle. The proposed model offers a step-by-step approach for identifying and selecting balanced performance indicators. In addition, the study suggests several potential indicators that can be used to measure the performance of transport logistics system in multiple-role companies. To address conflicting goals that may lead to local optima solution and silo thinking, this study proposes the use of system dynamics modelling. This approach can illuminate the trade-off between conflicting performance indicators as well as the interactions between strategic resources, performance drivers and performance output. To ensure continuous improvement, the model incorporates the problem-solving principle by employing TOC. In sum, the first task of this study was to integrate performance measurement design, system dynamics and problem solving into a unique chronological performance management framework.

To assess the suitability of the framework, it was applied to the performance of a sea transportation system in Company A. Using the new framework, the company was able to identify several key indicators that were perceived as important and relevant for measuring the transport logistics system performance. The framework was also used to build dynamic models to help the company’s decision-makers analyse how interconnections between system components influence system performance. In cases like these, the system dynamics model can be used to elucidate the basic dynamics of supply and demand, analyse tonnage requirements, and understand relationships between the company’s strategic resources and its performance output. Furthermore, the study showed that the combination of the system dynamics model with the simulation tool and the problem-solving method can be used to analyse system performance and constraints as well as to formulate and select an optimal performance improvement plan.

While previous studies tended to focus on only one of the three methods (performance measurement, system dynamics modelling or problem solving), this study has integrated them all into a new holistic performance management framework, thereby making a significant contribution to the existing body of research. The integrated framework proposed in this study contributes towards transport planning and evaluation, more specifically trade-off management between the cost and reliability aspects of logistics performance. Decision-makers in transport logistics in multiple-role companies, in this case, can use the proposed framework to support both operational and capital budgeting processes as well as evaluate the effectiveness and efficiency of several actions that are considered to improve the performance of transport logistics.

Since the case study section was based on findings from a single organisation, future research is needed to assess the generalisability of the developed framework in other contexts. In addition, future work is required to understand more about the potential of the new framework to minimise silo thinking and local optima solutions within organisations. Additionally, this study can be extended by quantitatively testing the structural relationships between strategic resources, performance drivers, and performance outputs that have been conceptually designed.

Figures

Figure 1

The conceptual basis of current research

Figure 2

Distribution of reviewed papers

Figure 3

Integrated framework for analysing and improving logistics performance

Figure 4

Integrating the system dynamics model with performance measurement

Figure 5

Basic stock and flow diagram of Company A’s marine distribution system

Figure 6

Logistics operating budget requirement

Figure 7

Linkages between operating budget and performance output

Figure 8

Relationship of system components to reliability

Figure 9

Model of relationship between capital budget and system infrastructure

Figure 10

Reliability performance

Figure 11

Factors affecting on-time delivery

Figure 12

Impacts of Jetty upgrade on system’s reliability

Figure 13

Impacts of Jetty capacity upgrade on vessel idle duration

Figure 14

Impacts of Jetty capacity upgrade on unit transportation cost

Table I

Potential indicators to measure transport logistics performance

			Roles		Layer
Dimensions	Indicators	Description	Profit generator	Public service provider	Strategic resource	Driver	Output
Reliability	Reliability	Overall transport logistics reliability score		✓			✓
	On-time delivery	Percentage of orders received on time as compared with standard		✓		✓
	Service level	The ratio of time the product is available at end channels to total duration		✓		✓
	Stockout	Number of day when product is not available		✓		✓
	Vessel breakdown	The duration when vessel cannot be operated due to technical problem		✓		✓
	Correctness	Percentage of orders delivered with no error		✓		✓
	Completeness	Percentage of cargo shipped		✓		✓
	Vessel pumping capacity	The maximum rate at which vessel can discharge cargo during a period of time		✓	✓
	Shore pumping capacity	The maximum rate at which terminal can load cargo to vessel during a period of time		✓	✓
	Vessel speed	The rate at which vessel is able to move		✓	✓
Responsiveness	Order-to-delivery cycle	The average elapsed time from the moment the order is ready to the reception by the customer		✓		✓
	Vessel round- trip duration	The average elapsed time from the moment vessel arrives at port to load until it comes back to the original port after discharging		✓		✓
	Sailing duration	The average elapsed time for vessel departing from one port until arriving at another port		✓		✓
	Loading duration	The average elapsed time from vessel starts loading cargo until it finishes loading cargo		✓		✓
	Discharging duration	The average elapsed time from vessel starts discharging cargo until it finishes		✓		✓
	Vessel idle duration	The average elapsed time vessel is in idle state in port		✓		✓
	Vessel repair duration	The average elapsed time vessel out of operation for drydocking		✓		✓
	Sourcing duration	The average elapsed time from the moment the sourcing order is accepted until it is completed		✓		✓
Asset	Operated vessel	Total number of vessels operated for transporting cargo	✓		✓
	Draft capacity	Depth of water in port where ship can navigate	✓		✓
	Jetty capacity	The maximum number of vessels or amount of cargo that jetty can serve	✓		✓
	Port equipment capacity	The maximum number of vessels or amount of cargo that port equipment can serve	✓		✓
	Delivery frequency	Total number of deliveries that took place in a certain period	✓			✓
	Effective load factor	The ratio between volume of cargo loaded and vessel carrying capacity	✓			✓
	Vessel utilisation ratio	The number of days when vessel is operated compared with number of days in a period	✓			✓
	Distance travelled per day	Total number of nautical miles travelled during a period of time over the period of days	✓			✓
	Berth utilisation	Utilised berth capacity in a period of time	✓			✓
	Port utilisation	Utilised port capacity in a period of time	✓			✓
	Labour utilisation	Utilised labour in a certain period of time	✓			✓
	Inventory level	Frequency of excessive stock	✓			✓
Financial	Total freight cost	The sum of vessel operating cost, bunker cost, and port cost	✓			✓
	Vessel operating cost	Amount of money spent to operate vessel in a certain period	✓			✓
	Bunker Cost	Amount of money spent for bunker in a period	✓			✓
	Port cost	Amount of money spent for port charge in a certain period	✓			✓
	Management cost	Amount of money spent to manage transport logistics system	✓			✓
	Revenue	Amount of money received from selling products supported by logistics system	✓			✓
	Profit	Difference between revenue and operational expenditure spent for transporting cargo	✓				✓
	Use of capital expenditure	Amount of money spent for upgrading logistics infrastructure	✓		✓
	Return on Investment	Ratio of need benefit of investment to capital budget used for investment	✓				✓
	Use of operational expenditure	Amount of operational expenditure for transporting cargo	✓		✓
	Unit Cost	Ratio of operational expenditure to transport cargo to volume of cargo shipped	✓			✓
	Vessel bunker consumption	Rate of bunker consumption per day	✓			✓

Table II

Selection of key performance indicators

Indicators	Group	Importance level (1–7)	Relevance (1–7)	Weighted score
Reliability	Performance output	5.8	6.1	5.95*
On-time delivery	Performance driver	6.2	6.3	6.25*
Service level	Performance driver	5.6	6.2	5.90*
Stockout	Performance driver	6.1	6.3	6.20*
Vessel breakdown	Performance driver	4.9	5.1	5.00*
Correctness	Performance driver	3.4	3.3	3.35
Completeness	Performance driver	3.6	3.2	3.40
Vessel pumping capacity	Strategic resource	5.2	5.3	5.25*
Shore pumping capacity	Strategic resource	5.4	5.1	5.25*
Vessel speed	Strategic resource	5.4	5.7	5.55*
Order-to-delivery cycle	Performance driver	3.6	3.1	3.35
Vessel round- trip duration	Performance driver	5.4	5.6	5.50*
Sailing duration	Performance driver	4.8	4.9	4.85*
Loading duration	Performance driver	5.8	5.1	5.45*
Discharging duration	Performance driver	5.6	5.2	5.40*
Vessel idle duration	Performance driver	5.8	5.1	5.45*
Vessel repair duration	Performance driver	4.9	5.1	5.00*
Sourcing duration	Performance driver	3.7	3.2	3.45
Operated vessel	Strategic resource	5.7	5.2	5.45*
Draft capacity	Strategic resource	5.9	5.6	5.75*
Jetty capacity	Strategic resource	6.1	5.9	6.00*
Port equipment capacity	Strategic resource	3.2	3.4	3.30
Delivery frequency	Performance driver	3.2	3.7	3.45
Effective load factor	Performance driver	5.6	4.8	5.20*
Vessel utilisation ratio	Performance driver	5.1	4.7	4.90*
Distance travelled per day	Performance driver	3.2	3.1	3.15
Berth utilisation	Performance driver	3.6	3.2	3.40
Port utilisation	Performance driver	3.8	3.1	3.45
Labour utilisation	Performance driver	3.1	3.6	3.35
Inventory level	Performance driver	4.8	5.5	5.15*
Total freight cost	Performance output	6.1	5.9	6.00*
Vessel operating cost	Performance driver	5.1	4.8	4.95*
Bunker cost	Performance driver	5.0	4.8	4.90*
Port cost	Performance driver	3.6	3.2	3.40
Management cost	Performance driver	3.1	2.7	2.90
Revenue	Performance driver	6.2	5.9	6.05*
Profit	Performance output	6.3	6.2	6.25*
Use of capital expenditure	Strategic resource	5.6	5.9	5.75*
Return on Investment	Performance output	5.4	5.7	5.55*
Use of operational expenditure	Strategic resource	5.9	5.8	5.85*
Unit cost	Performance driver	6.1	6.2	6.15*
Vessel bunker consumption	Performance driver	5.9	5.7	5.80*

Note: *p<0.05

Table III

Evaluating improvement plan scenarios

Scenario	Investment cost (USD/year)	Reliability increase (%)	Unit cost decrease (USD/Barrel)	(ΔReliability/ΔInvestment cost)	(ΔUnit cost/ΔInvestment cost)
1. Unit Jetty Capacity Upgrade	800,000	5	0.04	0.000006	0.00000005
2. Units Jetty Capacity Upgrade	1,600,000	14	0.10	0.000009	0.00000006
3. Units Jetty Capacity Upgrade	2,400,000	26	0.20	0.000011	0.00000008