[NOTE in progress] ECE236C - Optimization Methods for Large-Scale Systems [on going]

Source:http://www.seas.ucla.edu/~vandenbe/ee236c.html

 

Introduction

Outline

1. First-order algorithms
2. Decomposition and splitting
3. Second-order algorithms for unconstrained optimization
4. Interior point for conic optimization

Gradient

Convexity

- ∇^2(f (x) )> 0 is not necessary for strict convexity (cf., f (x) = x^4) but sufficient

- a differentiable function f is convex if and only if dom f is convex and (∇ f (x) − ∇ f (y)) ^T (x − y) ≥ 0 for all x, y ∈ dom f i.e., the gradient ∇ f : Rn → Rn is a monotone mapping

- Lipschitz Continuity 即利普希茨连续条件，是一个比通常连续更强的光滑性条件。 直觉上，利普希茨连续函数使用dual norm限制了函数改变的速度（从效果上感觉与限制二阶导数ub类似），符合利普希茨条件的函数的斜率，必小于一个称为利普希茨常数的实数（该常数依函数而定）。

- ||∇ f (x) − ∇ f (y)||∗ ≤ L||x −