Search results

Many dynamic problems in economics are characterized by large state spaces which make both computing and estimating the model infeasible. We introduce a method for approximating the value function of high-dimensional dynamic models based on sieves and establish results for the (a) consistency, (b) rates of convergence, and (c) bounds on the error of approximation. We embed this method for approximating the solution to the dynamic problem within an estimation routine and prove that it provides consistent estimates of the modelik’s parameters. We provide Monte Carlo evidence that our method can successfully be used to approximate models that would otherwise be infeasible to compute, suggesting that these techniques may substantially broaden the class of models that can be solved and estimated.

Rapid sensitivity analysis and near-optimal decision-making in contested environments are valuable requirements when providing military logistics support. Port of debarkation denial motivates maneuver from strategic operational locations, further complicating logistics support. Simulations enable rapid concept design, experiment and testing that meet these complicated logistic support demands. However, simulation model analyses are time consuming as output data complexity grows with simulation input. This paper proposes a methodology that leverages the benefits of simulation-based insight and the computational speed of approximate dynamic programming (ADP).

This paper describes a simulated contested logistics environment and demonstrates how output data informs the parameters required for the ADP dialect of reinforcement learning (aka Q-learning). Q-learning output includes a near-optimal policy that prescribes decisions for each state modeled in the simulation. This paper's methods conform to DoD simulation modeling practices complemented with AI-enabled decision-making.

This study demonstrates simulation output data as a means of state–space reduction to mitigate the curse of dimensionality. Furthermore, massive amounts of simulation output data become unwieldy. This work demonstrates how Q-learning parameters reflect simulation inputs so that simulation model behavior can compare to near-optimal policies.

Fast computation is attractive for sensitivity analysis while divorcing evaluation from scenario-based limitations. The United States military is eager to embrace emerging AI analytic techniques to inform decision-making but is hesitant to abandon simulation modeling. This paper proposes Q-learning as an aid to overcome cognitive limitations in a way that satisfies the desire to wield AI-enabled decision-making combined with modeling and simulation.

An aircraft manufacturer faces the problem of allocating inventory to a set of distributed warehouses in response to random, nonstationary demands. There is particular interest in managing high value, low volume spare parts which must be available to respond to low‐frequency demands in the form of random failures of major components. The aircraft fleet is young and in expansion. In addition, high‐value parts can be repaired, implying that they reenter the system after they are removed from an aircraft and refurbished. This paper aims to present a model and a solution approach to the problem of determining the inventory levels at each warehouse.

The problem is solved using approximate dynamic programming (ADP), but this requires developing new methods for approximating value functions in the presence of low‐frequency observations.

The model and solution approach have been implemented, tested and validated internally at the manufacturer through the analysis of the inventory policy recommendations in different network scenarios and for different pools of parts. The results seem promising and compelling.

The uniqueness of this research is in the use of ADP for the modeling and solution of a distributed inventory problem. Its main value resides on the incorporation of the issue of spatial substitution in demand satisfaction within the problem of determining inventory levels in a distributed warehouse network.

The purpose of this paper is to maximize the expected revenue of the outpatient department considering patient preferences and choices.

Patient preference refers to the preferred physician and time slot that patients hold before asking for appointments. Patient choice is the appointment decision the patient made after receiving a set of options from the scheduler. The relationship between patient choices and preferences is explored. A dynamic programming (DP) model is formulated to optimize appointment scheduling with patient preferences and choices. The DP model is transformed to an equivalent linear programming (LP) model. A decomposition method is proposed to eliminate the number of variables. A column generation algorithm is used to resolve computation problem of the resulting LP model.

Numerical studies show the benefit of multiple options provided, and that the proposed algorithm is efficient and accurate. The effects of the booking horizon and arrival rates are studies. A policy about how to make use of the information of patient preferences is compared to other naive polices. Experiments show that more revenue can be expected if patient preferences and choices are considered.

This paper proposes a framework for appointment scheduling problem in outpatient departments. It is concluded that more revenue can be achieved if more choices are provided for patients to choose from and patient preferences are considered. Additionally, an appointment decision can be made timely after receiving patient preference information. Therefore, the proposed model and policies are convenient tools applicable to an outpatient department.

This paper aims to propose an innovative building information modelling (BIM)-based framework for multi-objective and dynamic temporary construction site layout design (SLD), which uses a hybrid approach of systematic layout planning (SLP) and mathematical modelling.

The hybrid approach, which follows a step-by-step process for site layout planning, is designed to facilitate both qualitative and quantitative data collection and processing. BIM platform is usedto facilitate the determination of the required quantitative data, while the qualitative data are generated through knowledge-based rules.

The multi-objective layout model represents two important aspects: layout cost and adjacency score. The result shows that the model meets construction managers’ requirements in not only saving cost but also assuring the preferences of temporary facility relationships. This implies that the integration of SLP and mathematical layout modelling is an appropriate approach to deliver practical multi-objective SLD solutions.

The proposed framework is expected to serve as a solution, for practical application, which takes the advantage of technologies in data collection and processing. Besides, this paper demonstrates, by using numerical experimentation and applying Microsoft Excel Solver for site layout optimisation, how to reduce the complexity in mathematical programming for construction managers.

The original contribution of this paper is the attempt of developing a framework in which all data used for the site layout modelling are collected and processed using a systematic approach, instead of being predetermined, as in many previous studies.

We derive marginal conditions of optimality (i.e., Euler equations) for a general class of Dynamic Discrete Choice (DDC) structural models. These conditions can be used to estimate structural parameters in these models without having to solve for approximate value functions. This result extends to discrete choice models the GMM-Euler equation approach proposed by Hansen and Singleton (1982) for the estimation of dynamic continuous decision models. We first show that DDC models can be represented as models of continuous choice where the decision variable is a vector of choice probabilities. We then prove that the marginal conditions of optimality and the envelope conditions required to construct Euler equations are also satisfied in DDC models. The GMM estimation of these Euler equations avoids the curse of dimensionality associated to the computation of value functions and the explicit integration over the space of state variables. We present an empirical application and compare estimates using the GMM-Euler equations method with those from maximum likelihood and two-step methods.

The purpose of this paper is to develop a mathematical framework to address some algorithmic features of approximate dynamic programming (ADP) by using an average cost formulation based on the concepts of differential costs and performance gradients. Under such a framework, a modified value iteration algorithm is developed that is easy to implement, in the mean time it can address a class of partially observable Markov decision processes (POMDP).

Gradient‐based policy iteration (GBPI) is a top‐down, system‐theoretic approach to dynamic optimization with performance guarantees. In this paper, a bottom‐up, algorithmic view is provided to complement the original high‐level development of GBPI. A modified value iteration is introduced, which can provide solutions to the same type of POMDP problems dealt with by GBPI. Numerical simulations are conducted to include a queuing problem and a maze problem to illustrate and verify features of the proposed algorithms as compared to GBPI.

The direct connection between GBPI and policy iteration is shown under a Markov decision process formulation. As such, additional analytical insights were gained on GBPI. Furthermore, motivated by this analytical framework, the authors propose a modified value iteration as an alternative to addressing the same POMDP problem handled by GBPI.

Several important insights are gained from the analytical framework, which motivate the development of both algorithms. Built on this paradigm, new ADP learning algorithms can be developed, in this case, the modified value iteration, to address a broader class of problems, the POMDP. In addition, it is now possible to provide ADP algorithms with a gradient perspective. Inspired by the fundamental understanding of learning and optimization problems under the gradient‐based framework, additional new insight may be developed for bottom‐up type of algorithms with performance guarantees.

In a kitting supply system, the occurrence of material-handling errors is unavoidable and will cause serious production losses to an assembly line. To minimize production losses, this paper aims to present a dynamic scheduling problem of automotive assembly line considering material-handling mistakes by integrating abnormal disturbance into the material distribution problem of mixed-model assembly lines (MMALs).

A multi-phase dynamic scheduling (MPDS) algorithm is proposed based on the characteristics and properties of the dynamic scheduling problem. In the first phase, the static material distribution scheduling problem is decomposed into three optimization sub-problems, and the dynamic programming algorithm is used to jointly optimize the sub-problems to obtain the optimal initial scheduling plan. In the second phase, a two-stage rescheduling algorithm incorporating removing rules and adding rules was designed according to the status update mechanism of material demand and multi-load AGVs.

Through comparative experiments with the periodic distribution strategy (PD) and the direct insertion method (DI), the superiority of the proposed dynamic scheduling strategy and algorithm is verified.

To the best of the authors’ knowledge, this study is the first to consider the impact of material-handling errors on the material distribution scheduling problem when using a kitting strategy. By designing an MPDS algorithm, this paper aims to maximize the absorption of the disturbance caused by material-handling errors and reduce the production losses of the assembly line as well as the total cost of the material transportation.