Search results

Finding the optimal solution to address problems in sewer management systems has always challenged asset managers. An understanding of deterioration mechanisms in sewers can help asset managers in developing prediction models for estimating whether or not sewer collapse is likely. The effective use of deterioration prediction models along with the development and use of life cycle cost analysis (LCCA) can contribute to the goals of reducing construction, operation and maintenance costs in sewer systems. When sewer system maintenance/rehabilitation options are viewed as investment alternatives, it is important, and in some cases, imperative, to make decisions based on life cycle costs instead of relying totally on initial construction costs. The objective of this paper is to discuss the application of deterioration modelling and life cycle cost principles in sewer system management, and to explore the role of the Markov chain model in decision making regarding sewer rehabilitation. A test case is used to demonstrate the application of the Markov chain decision model for sewer system management. The analysis includes evaluation of this concept using dynamic programming and the policy improvement algorithm.

The purpose of this paper is to establish a version of a theorem that originated from population genetics and has been later adopted in evolutionary computation theory that will lead to novel Monte‐Carlo sampling algorithms that provably increase the AI potential.

In the current paper the authors set up a mathematical framework, state and prove a version of a Geiringer‐like theorem that is very well‐suited for the development of Mote‐Carlo sampling algorithms to cope with randomness and incomplete information to make decisions.

This work establishes an important theoretical link between classical population genetics, evolutionary computation theory and model free reinforcement learning methodology. Not only may the theory explain the success of the currently existing Monte‐Carlo tree sampling methodology, but it also leads to the development of novel Monte‐Carlo sampling techniques guided by rigorous mathematical foundation.

The theoretical foundations established in the current work provide guidance for the design of powerful Monte‐Carlo sampling algorithms in model free reinforcement learning, to tackle numerous problems in computational intelligence.

Establishing a Geiringer‐like theorem with non‐homologous recombination was a long‐standing open problem in evolutionary computation theory. Apart from overcoming this challenge, in a mathematically elegant fashion and establishing a rather general and powerful version of the theorem, this work leads directly to the development of novel provably powerful algorithms for decision making in the environment involving randomness, hidden or incomplete information.

Purpose – The purpose of this chapter is to introduce new methods to behavioral research on group negotiation.

Design/methodology/approach – We describe three techniques from the field of Machine Learning and discuss their possible application to modeling dynamic processes in group negotiation: Markov Models, Hidden Markov Models, and Inverse Reinforcement Learning. Although negotiation research has employed Markov modeling in the past, the latter two methods are even more novel and cutting-edge. They provide the opportunity for researchers to build more comprehensive models and to use data more efficiently. To demonstrate their potential, we use scenarios from group negotiation research and discuss their hypothetical application to these methods. We conclude by suggestions for researchers interested in pursuing this line of work.

Originality/value – This chapter introduces methods that have been successfully used in other fields and discusses how these methods can be used in behavioral negotiation research. This chapter can be a valuable guide to researchers that would like to pursue computational modeling of group negotiation.

The main purpose of the expert system presented in the paper is to support proper decision-making to perform the operation of the complex and crucial technical system in a rational way.

The proposed system was developed using the universal concepts of a semi-Markov process, quality space and a multi-objective analysis. The maintenance and operation processes of a machine were modelled in the form of a semi-Markov process, the quality space was used to exclude the operation and maintenance process of critical quality and finally, thanks to implementation of a multi-objective analysis, the assessment system was build.

By generating each flow of the process, the expert system supports optimization of a technical system operation to choose the best maintenance strategy. Application of the expert system created based on a real industrial system is presented at the end of the paper.

The limitations of the proposed approach can be found in the parts of simulation and assessment. As the number of states to be taken into consideration increases, the time of calculation gets longer as well. As regards the assessment, ranges of the criteria argument have to be determined. Unfortunately, in some industrial systems, they are difficult to define or they are infinite and should be artificially limited.

The system provides three most important benefits as compared to other solutions. The first benefit is the system ability to make a choice of the best strategy from the perspective of the accepted criteria. The second advantage is the ability to choose the best operation and maintenance strategy from the point of view of a decision-maker. And the third is that the decision-maker can be completely sure that the chosen way of operation is not of critical quality.

The novelty of the proposed solution involves the system approach to the expert system design, thanks to the described procedure which is flexible and can be easily implemented in different technical systems which have a crucial impact on reliability and safety of their operation. It is the unique combination of probability-based simulation, multi-dimensional quality considerations and multi-objective analysis.

Asset intensive process industries are under immense pressure to achieve promised return on investments and production targets. This can be accomplished by ensuring the highest level of availability, reliability and utilization of the critical equipment in processing facilities. In order to achieve designed availability, asset characterization and maintainability play a vital role. The most appropriate and effective way to characterize the assets in a processing facility is based on risk and consequence of failure. The paper aims to discuss these issues.

In this research, a risk-based stochastic modeling approach using a Markov decision process is investigated to assess a processing unit's availability, which is referred as the risk-based availability Markov model (RBAMM). RBAMM will not only provide a realistic and effective way to identify critical assets in a plant but also a method to estimate availability for efficient planning purposes and resource optimization.

A unique risk matrix and methodology is proposed to determine the critical equipment with direct impact on the availability, reliability and safety of the process. A functional block diagram is then developed using critical equipment to perform efficient modeling. A Markov process is utilized to establish state diagrams and create steady-state equations to calculate the availability of the process. RBAMM is applied to natural gas absorption process to validate the proposed methodology. In the conclusion, other benefits and limitations of the proposed methodology are discussed.

A new risk-based methodology integrated with Markov model application of the methodology is demonstrated using a real-life application.

Several rehabilitation planning methods are reported in the literature for public infrastructures, such as bridges, pavements, sewers, etc. These methods, however, are limited to specific types of infrastructures. The purpose of the present research is to develop a novel and generic method for Maintenance and Rehabilitation Planning for Public Infrastructure (M&RPPI), which aims at determining the optimal rehabilitation profile over a desired analysis period.

The M&RPPI method is based on life‐cycle costing (LCC) with probabilistic and continuous rating approach for condition states. The M&RPPI uses a new approach of “dynamic” Markov chain to represent the deterioration mechanism of an infrastructure and the impact of rehabilitation interventions on such infrastructure. It also uses genetic algorithm (GA) in conjunction with Markov chains in order to find the optimal rehabilitation profile. A case study is presented with a comparison between the traditional Markov decision process (MDP) and the newly developed method.

The new method, which generates lower LCC, is found practical in providing a complete M&R plan over a required study period, compared to a stationary decision policy with the traditional MDP. In addition, GA is found useful in the optimization process and overcomes the computational difficulties for large combinatorial problems.

The implementation of the developed models is limited to only four alternatives/actions. However, the developed models and framework are superior for MDP.

The developed methodology and model play essential roles in the decision‐making process.

The new method is beneficial to researchers and practitioners. It is developed for a single facility; however, it provides a major step towards a broader infrastructure management system and capital budgeting problems.

The purpose of this paper is to analyze the usability of a remote learning system in its initial development phase, using a quantitative usability evaluation method through Markov models.

The paper opted for an exploratory study. The data of interest of the research correspond to the possible accesses of users in a learning system's pre‐project. The present research is intended to answer questions of the type “how”, evaluating the usability through Markov models and the interaction of the learning system's users.

The paper provides a study which allowed the generation of a probability matrix among states and actions, which was utilized in the evaluation of usability based on Markov models, where the usability criteria specified were analyzed. Markov models allow a series of measures of interest to be calculated and they have been successfully utilized in the evaluation of computing and communication systems.

This research aimed to investigate the utilization of Markov models in the evaluation of usability in a remote learning system in its pre‐project phase and to help in proposing improvement suggestions.

The purpose of this paper is to present an approach which can evaluate the ability that successfully achieves command and control, both in qualitative and quantitative modes, to improve decision accuracy and speed, as well as construct an executable architecture for analyzing and verifying different decision projects.

By defining command and control (C2) decision architecture and decomposing C2 decision processes into measurable subfunctions, measures and metrics will be associated with each of the lowest level decomposed functions, and will be used to provide support for performance evaluation. Both Markov decision process analysis and conditional probability (CP) logic are used for modeling the decision‐making process of course of action (COA). Meanwhile, an executable architecture constructed by Petri net is applied to logic structural verification and performance evaluation.

The paper presents an idea and methodology for net‐centric command and control decision‐making process analysis.

The paper describes and decomposes C2 decision processes for complex missions in uncertain environments.

The paper could be an important reference of analysis and application in net‐centric command and control of decision making.

The paper combines methodology with qualitative methods (decision process decomposition), quantitative method (Markov decision process analysis and CP logic), as well as structural verification and performance evaluation.

The purpose of this study is to consider the acquisition of end‐of‐life products under variable quality consideration for remanufacturing so as to determine optimal control policy that minimizes per‐period expected costs that may guide future consideration by practitioners.

The authors review recent literature on reverse supply chains and remanufacturing. They utilize an undiscounted Markov decision process methodology to ascertain the order amount of remanufacturable products using optimal control under minimum cost criterion.

The authors conclude that it makes sense for firms to focus on the cost management with production control based on quality levels with different acquisition costs of remanufacturable products.

Although the Markov decision process methodology – which is well supported in literature – was diligently followed, the nature of analysis and discussion may be subject to authors’ bias. Future investigation and adoption of the methodological approach used will verify the paper findings.

This study determines optimal control policy for ordering specific amount of product that minimizes per‐period expected costs for remanufacturing. Reverse supply‐chain professionals now have an easy‐to‐follow guide when acquiring end‐of‐life remanufacturable products alternatives with variable quality.

This study determines the optimal policy for ordering remanufacturable products. This information enables practitioners to reduce their carbon footprint in reverse supply chain through inspection/sorting before remanufacturing by processing only the type, quality, and quantity of needed product.

For reverse supply chain to be taken seriously by senior management in firms, it is imperative that practitioners in this field synchronize their operational‐level ordering decisions with holistic cost minimization objective (to maximize value recovery) to stay viable.

Although very significant and applicable, there have been no formal justifications for the use of Monte‐Carlo models and Markov chains in evaluating hospital admission decisions or concrete data supporting their use. For these reasons, this research was designed to provide a deeper understanding of these models. The purpose of this paper is to examine the usefulness of a computerized Monte‐Carlo simulation of admission decisions under the constraints of emergency departments.

The authors construct a simple decision tree using the expected utility method to represent the complex admission decision process terms of quality adjusted life years (QALY) then show the advantages of using a Monte‐Carlo simulation in evaluating admission decisions in a cohort simulation, using a decision tree and a Markov chain.

After showing that the Monte‐Carlo simulation outperforms an expected utility method without a simulation, the authors develop a decision tree with such a model. real cohort simulation data are used to demonstrate that the integration of a Monte‐Carlo simulation shows which patients should be admitted.

This paper may encourage researchers to use Monte‐Carlo simulation in evaluating admission decision implications. The authors also propose applying the model when using a computer simulation that deals with various CVD symptoms in clinical cohorts.

Aside from demonstrating the value of a Monte‐Carlo simulation as a powerful analysis tool, the paper's findings may prompt researchers to conduct a decision analysis with a Monte‐Carlo simulation in the healthcare environment.