[NOTE in progress] Simulation Optimization

最新推荐文章于 2023-02-17 21:34:06 发布

原创

最新推荐文章于 2023-02-17 21:34:06 发布 · 543 阅读

CC 4.0 BY-SA版权

简单记录一下关于仿真优化的一些知识点和思考。主要基于：Handbook of Simulation Optimization, Michael Fu

Table of Contents

Overview

Overview

这是本书的overview 实际上也可以看做是这一field的overview.

SimuOpt : optimize, when the obj function $f$ cannot be computed directly, but can be simulated, with noise (focus on stochastic simulation environment).

一种分类方式：Discrete vs Continuous

Discrete Optimization
- Solution space is small -> Ranking & Selection (based on statistics or simulation budget allocation)
- Solution space is large be finite -> Ordinal Optimization (no need to estimate accurately every candidate, only need to know their order. Much faster convergence (exponential))
- Solution space is countably infinite -> Random Search (globally or locally convergent)
Continuous Opt
- RSM (Response Surface Methodology). Also has constraint considerations and robust variants
- Stochastic Approximation (RM, KW, simutaneous perturbation stochastic approximation for high-dim pbs)
- SAA (Sample Average Approximation) with consideration on stochastic constraints.
- Random Search, focus on estimation and on the search procedure. Model-based RS is newer class, assuming probability matrix is known.

Since stochasticity is the keyword, some base knowledge is important for DO as well as for CO.

are needed to 1) ensure the correctness of the algo; 2) define the stopping criteria
for constrain free non-linear optimization, we stop at a settle point
for integer optimization, we check the gap between LB and UB
here for SBO, it's difficult because:
- the cost of solution g(x) can only be estimated
- no structural info can be used to prune solution zone
- complete enumeration of the solution space is often computationally intractable

Small. Less than hundreds of candidate. The key is then how to well estimate all solutions and return the best. Practically we analyze the Probability of Sel