[LA] Lipschitz continuous gradient

1. Definitions

$\nabla f(x)$ is L- Lipschitz continuous, then we have

• (1) $\|\nabla f(x) - \nabla f(y)\|_2\leq L\|x-y\|_2$(note that this does not assume convexity of $f(x)$)
• (2) $\frac{L}{2}x^T x - f(x)$ is convex ( if $\text{dom}(f)$ is convex )
• (3) $\nabla ^2 f(x)\leq \text{L} \cdot I$ ( if $f(x)$ is twice differentiable )
• (4) $f(y) \leq f(x)+\nabla f(x)\cdot (y-x)+\frac{L}{2}\|y-x\|_2^2$ ( if $f(x)$ is convex)

(一) (1) to (2): From the equivalent definition of convexity, that
f(x) is convex iff $(\nabla f(x) - \nabla f(y))^T (x-y)\geq 0$ and $\text{dom}(f)$ is convex.
We just need to prove that

[(L⋅x−∇f(x))−(L⋅y−∇f