Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems
Buch in deiner Nähe kaufen
Beschreibung
This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.
The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.
Among the more important novel considerations presented are:
- the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;
- proof of semi-smoothness of the value function at degenerate points;
- attention to the under-selectivity issue for the long-run average and bias optimality;
- discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and
- development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.
The book will be of interest to researchers and students in the field of stochastic control and performance optimization alike.
This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.
The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.
Among the more important novel considerations presented are:
- the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;
- proof of semi-smoothness of the value function at degenerate points;
- attention to the under-selectivity issue for the long-run average and bias optimality;
- discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and
- development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.
The book will be of interest to researchers and students in the field of stochastic control andperformance optimization alike.
Solves existing problems without requiring deep knowledge of partial differential equations Presents a new framework for optimization of stochastic systems, promoting new research pathways Shows the reader how to link optimisation of continuous systems with reinforcement-learning algorithms and artificial intelligence
Autor*in
Xi-Ren Cao
Themen in »Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems«
Performance Optimization Stochastic Optimization Stochastic Control Perturbation Analysis Sensitivity Analysis Dynamic Programming Continuous-Time Dynamical Systems Operations Research Ito–Tanaka formula
Stimmen zu »Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems«
“This book develops an alternative viewpoint for stochastic control inspired by ideas rooted in sensitivity formulae and perturbation analysis. … The theory is accompanied by several examples and exercises from time to time and includes the background material as needed, either in the main text or in an appendix.” (Vivek S. Borkar, Mathematical Reviews, April, 2022)
()