Search results

Current reinforcement learning (RL) algorithms are facing issues such as low learning efficiency and poor generalization performance, which significantly limit their practical application in real robots. This paper aims to adopt a hybrid model-based and model-free policy search method with multi-timescale value function tuning, aiming to allow robots to learn complex motion planning skills in multi-goal and multi-constraint environments with a few interactions.

A goal-conditioned model-based and model-free search method with multi-timescale value function tuning is proposed in this paper. First, the authors construct a multi-goal, multi-constrained policy optimization approach that fuses model-based policy optimization with goal-conditioned, model-free learning. Soft constraints on states and controls are applied to ensure fast and stable policy iteration. Second, an uncertainty-aware multi-timescale value function learning method is proposed, which constructs a multi-timescale value function network and adaptively chooses the value function planning timescales according to the value prediction uncertainty. It implicitly reduces the value representation complexity and improves the generalization performance of the policy.

The algorithm enables physical robots to learn generalized skills in real-world environments through a handful of trials. The simulation and experimental results show that the algorithm outperforms other relevant model-based and model-free RL algorithms.

This paper combines goal-conditioned RL and the model predictive path integral method into a unified model-based policy search framework, which improves the learning efficiency and policy optimality of motor skill learning in multi-goal and multi-constrained environments. An uncertainty-aware multi-timescale value function learning and selection method is proposed to overcome long horizon problems, improve optimal policy resolution and therefore enhance the generalization ability of goal-conditioned RL.

In the construction of charging piles, traditional gas stations possess significant advantages in terms of regional and financial resources. The transformation of gas stations into “refueling+charging” integrated gas stations relies on charging pile manufacturers and government, involving coordination issues with them. This paper aims to propose a joint coordination contract based on the principles of cost-sharing and revenue-sharing. The objective is to achieve systemic coordination among integrated gas stations, charging pile manufacturers, and the government, optimizing the planning of the quantity of charging piles and charging prices.

We have constructed an operational system model based on the Stackelberg game between charging pile manufacturers, integrated gas stations, and government. We have analyzed the optimal quantity of charging piles and charging prices under the impact of government subsidy policies in both decentralized and centralized operation scenarios. Additionally, we have proposed a joint coordination contract based on cost-sharing and revenue-sharing to coordinate this tripartite operational system.

The study reveals that, under simple cooperative contracts, the optimal decision does not yield maximum profits for the operational system due to the “double-marginal effect”. However, under the impact of the joint coordination contract, which combines cost-sharing and revenue-sharing as proposed in this paper, gas stations will consider the charging pile manufacturer’s costs and government subsidies when determining the optimal quantity and price. This not only achieves system coordination but also results in Pareto improvement in the benefits of all system members by adjusting contract parameters.

The value of this research lies in its insights into operational strategies for the construction of charging piles for electric vehicles. By analyzing optimal decisions under different contract arrangements, the study provides guidance to relevant stakeholders, enabling the operational system to achieve greater efficiency and coordination and realize more extensive Pareto improvements. Furthermore, it extends the application of coordination contract theory in the context of charging pile construction and operations.

While the main purpose of reporting – e.g. reporting for taxes – is to gauge the economic state of a company, the fact that reporting is done at pre-determined dates distorts the reporting results. For example, to create a larger impression of their productivity, companies fire temporary workers before the reporting date and re-hire then right away. The purpose of this study is to decide how to avoid such distortion.

This study aims to come up with a solution which is applicable for all possible reasonable optimality criteria. Thus, a general formalism for describing and analyzing all such criteria is used.

This study shows that most distortion problems will disappear if the fixed pre-determined reporting dates are replaced with individualized random reporting dates. This study also shows that for all reasonable optimality criteria, the optimal way to assign reporting dates is to do it uniformly.

This study shows that for all reasonable optimality criteria, the optimal way to assign reporting dates is to do it uniformly.

It is found that the individualized random tax reporting dates would be beneficial for economy.

It is found that the individualized random tax reporting dates would be beneficial for society as a whole.

This study proposes a new idea of replacing the fixed pre-determining reporting dates with randomized ones. On the informal level, this idea may have been proposed earlier, but what is completely new is our analysis of which randomization of reporting dates is the best for economy: it turns out that under all reasonable optimality criteria, uniform randomization works the best.

By introducing the mass customization service mode into the cloud logistics environment, this paper studies the joint optimization of service provider selection and customer order decoupling point (CODP) positioning based on the mass customization service mode to provide customers with more diversified and personalized service content with lower total logistics service cost.

This paper addresses the general process of service composition optimization based on the mass customization mode in a cloud logistics service environment and constructs a joint decision model for service provider selection and CODP positioning. In the model, the two objective functions of minimum service cost and most satisfactory delivery time are considered, and the Pareto optimal solution of the model is obtained via the NSGA-II algorithm. Then, a numerical case is used to verify the superiority of the service composition scheme based on the mass customization mode over the general scheme and to verify the significant impact of the scale effect coefficient on the optimal CODP location.

(1) Under the cloud logistics mode, the implementation of the logistics service mode based on mass customization can not only reduce the total cost of logistics services by means of the scale effect of massive orders on the cloud platform but also make more efficient use of a large number of logistics service providers gathered on the cloud platform to provide customers with more customized and diversified service content. (2) The scale effect coefficient directly affects the total cost of logistics services and significantly affects the location of the CODP. Therefore, before implementing the mass customization logistics service mode, the most reasonable clustering of orders on the cloud logistics platform is very important for the follow-up service combination.

The originality of this paper includes two aspects. One is to introduce the mass customization mode in the cloud logistics service environment for the first time and summarize the operation process of implementing the mass customization mode in the cloud logistics environment. Second, in order to solve the joint decision optimization model of provider selection and CODP positioning, this paper designs a method for solving a mixed-integer nonlinear programming model using a multi-layer coding genetic algorithm.

Throughout human history, the occurrence of disasters has been inevitable, leading to significant human, financial and emotional consequences. Therefore, it is crucial to establish a well-designed plan to efficiently manage such situations when disaster strikes. The purpose of this study is to develop a comprehensive program that encompasses multiple aspects of postdisaster relief.

A multiobjective model has been developed for postdisaster relief, with the aim of minimizing social dissatisfaction, economic costs and environmental damage. The model has been solved using exact methods for different scenarios. The objective is to achieve the most optimal outcomes in the context of postdisaster relief operations.

A real case study of an earthquake in Haiti has been conducted. The acquired results and subsequent management analysis have effectively assessed the logic of the model. As a result, the model’s performance has been validated and deemed reliable based on the findings and insights obtained.

Ultimately, the model provides the optimal quantities of each product to be shipped and determines the appropriate mode of transportation. Additionally, the application of the epsilon constraint method results in a set of Pareto optimal solutions. Through a comprehensive examination of the presented solutions, valuable insights and analyses can be obtained, contributing to a better understanding of the model’s effectiveness.

Design a novel optimal integrated control algorithm for the automotive electric steering system to improve the stability and adaptation of the system.

Simulation and calculation.

The output signals follow the reference signal with high accuracy.

The optimal integrated algorithm is established based on the combination of PID and SMC. The parameters of the PID controller are adjusted using a fuzzy algorithm. The optimal range of adjustment values is determined using a genetic algorithm.

In this study, a dynamic contracting model is developed between a venture capitalist (VC) and an entrepreneur (EN) to explore the influence of asymmetric beliefs regarding output-relevant parameters, agency conflicts and complementarity on the VC's posterior beliefs through the EN's unobservable effort choices to influence the optimal dynamic contract.

The authors construct the contracting model by incorporating the VC's effort, which is ignored in most studies. Using backward induction and a discrete-time approximation approach, the authors solve the continuous-time contract design problem, which evolves into a nonlinear ordinary differential equation (ODE).

The optimal equity share that the VC provides to the EN decreases over time. In accordance with the empirical evidence, the EN's optimistic beliefs regarding the project's profitability positively affect its equity share. However, the interactions between the optimal equity share, project risk and both partners' degrees of risk aversion are not monotonic. Moreover, the authors find that the optimal equity share increases with the degree of complementarity, which indicates that the EN is willing to cooperate with the VC. This study’s results also show that the optimal equity shares at each time are interdependent if the VC is risk-averse and independent if the VC is risk-neutral.

In conclusion, the authors highlight two potential directions for future research. First, the authors only considered a single VC, whereas in practice, a risk project may be carried out by multiple VCs, and it is interesting to discuss how the degree of complementarity affects the number of VCs that ENs contract. Second, the authors may introduce jumps and consider more general multivariate stochastic volatility models for output dynamics and analyze the characteristics of the optimal contracts. Third, further research can deal with other forms of discretionary output functions concerning complementarity, such as Cobb–Douglas and constant elasticity of substitution (See Varian, 1992).

The results of this study have several implications. First, it offers a novel approach to designing dynamic contracts that are specific and easy to operate. To improve the complicated venture investment situation and abate conflict between contractual parties, this study plays a good reference role. Second, the synergy effect proposed in this study provides a theoretical explanation for the executive compensation puzzle in economics, in which managers are often “rewarded for luck” (Bertrand and Mullainathan, 2001; Wu et al., 2018). This result indicates a realistic perspective on financing and establishing cooperative relationships, which enhances the efficiency of venture investment. Third, from an empirical standpoint, one can apply this framework to study research and development (R&D) problems.

First, the authors introduce asymmetric beliefs and Bayesian learning to study the dynamic contract design problem and discuss their effects on equity share. Second, the authors incorporate the VC's effort into the contracting problem, and analyze the synergistic effect of effort complementarity on the optimal dynamic contract.

The paper attempts to discuss the impact of reference price effect on pricing decisions.

With the growth of the Internet and e-commerce, more and more customers purchase products in through online channels and choose products by comparing different prices and services, and the reference price effect has an impact on pricing decisions. To investigate the impact of consumers' reference price effect on the dual-channel supply chain, the authors establish a basic model consisting of a single dominant manufacturer and a single downstream retailer, and analyze the optional decisions under different situations and discuss the influence of reference price effect. Finally, a number case verifies the validity and rationality of the proposed model.

The results show that (1) the reference price effect has varying effects on the price, channel demand and income of manufacturers and retailers in the channel depending on the role of customers' channel preferences. (2) The manufacturer's online channel demand and profits always increase with the reference pricing effect, whereas the retailer's offline demand and profits always decline. (3) When the proportion of consumers preferring offline is higher, the manufacturer's network price and wholesale price increase with the reference price effect, while the retailer's retail price decreases with the reference price effect; when the proportion of consumers preferring offline is lower, the opposite is true, and the centralized decision results are consistent with the decentralized decision results.

This paper can clarify the impact of consumer reference price effects on the operation of dual-channel supply chains, and help inform pricing decisions of manufacturers and retailers in dual-channel supply chains.

The proposed approach can well analyze the impact of consumer reference price effect and give channel their optional decisions.

This study aims to study the thermal identification issue by harvesting both solar energy and atmospheric thermal updraft for a solar-powered unmanned aerial vehicle (SUAV) and its electric energy performance under continuous soaring conditions.

The authors develop a specific dynamic model for SUAVs in both soaring and cruise modes. The support vector machine regression (SVMR) is adopted to estimate the thermal position, and it is combined with feedback control to implement the SUAV soaring in the updraft. Then, the optimal path model is built based on the graph theory considering the existence of several thermals distributed in the environment. The procedure is proposed to estimate the electricity cost of SUAV during flight as well as soaring, and making use of dynamic programming to maximize electric energy.

The simulation results present the integrated control method could allow SUAV to soar with the updraft. In addition, the proposed approach allows the SUAV to fly to the destination using distributed thermals while reducing the electric energy use.

Two simplified dynamic models are constructed for simulation considering there are different flight mode. Besides, the data-driven-based SVMR method is proposed to support SUAV soaring. Furthermore, instead of using length, the energy cost coefficient in optimization problem is set as electric power, which is more suitable for SUAV because its advantage is to transfer the three-dimensional path planning problem into the two-dimensional.

Every business company deals with the dilemma of how much to invest in long-term (LT) versus short-term (ST) problem (LTvST problem). LT operations increase the reputation of the company, and revenue is rewarded in the future. In contrast, ST operations result in immediate rewards. Thus, every organization faces the dilemma of how much to invest in LT versus ST activities. The former deals with the “what” or effectiveness, and the latter deals with the “how” or efficiency. The role of managers is to solve this dilemma; however, they often fail to do so, mainly because of a lack of knowledge. This study aims to propose a dynamic optimal control model that formulates and solves the LTvST problem.

This study proposes a dynamic optimal control model that formulates and solves the dilemma whether to invest in short- or LT operations.

This model is illustrated as an example of an academic institute that wants to maximize its reputation. Investing in effectiveness in the academy translates into investing in research, whereas investing in efficiency translates into investing in teaching. Universities and colleges with a good reputation attract stronger candidates and benefit from higher tuition fees. Steady-state conditions and insightful observations were obtained by studying the optimal solution and performing a sensitivity analysis.

To the best of the authors’ knowledge, this paper is the first one to explore the optimal strategy when trying to maximize the short and LT activities of a company and solve the LTvST problem. Furthermore, it is applied on universities where teaching is the ST activity and research the LT activity. The insights gleaned from the application are relevant to many different fields. The authors believe that the paper makes a significant contribution to academic literature and to business managers.