Levenberg-Marquardt(LM算法)的理解

1. convex optimization

先简单介绍一下凸优化的内容
First quickly introduction the convex optimization concepts.

1.1 convex set

Definition:
for $x_1,x_2\in A$, we say $A$ is a convex set if and only if the following holds for any $\theta \in [0,1]$:
$$\theta x_1+(1-\theta )x_2\in A$$

凸函数的定义如上，不过可以通过下面的图简单理解。可以直觉得说，如果两点集合内的点的“连线“上的点都在集合内，那么它就是一个凸集合。
We can quickly understand it by the following graph. If all the elements between $x_1$ and $x_2$ are in the set $A$, we could say it is a convex set.

1.2 convex function

Definition:
we say $f(x)$ with domain $dom(f)$ is a convex function, if and only if , the following conditions holds for any $\theta \in [0,1]$:
$$f($$