逻辑斯蒂回归的代价函数

Logistic回归的代价函数

$$J(\theta )=-\frac{1}{m}\left[\sum _{i=1}^m{y}^{(i)}log{h}_{\theta }(x^{(i)})+(1-y^{(i)})log(1-h_{\theta }(x^{(i)}))\right]$$对代价函数求偏导：
补充：
$$\begin{array}{rl}& h_{\theta }(x^{(i)})=g(x^{(i)}\theta )\\ & g(x)=\frac{1}{1+e^{-x}}\\ & g^{{}^{\prime }}(x)=g(x)(1-g(x))\\ & \frac{\partial h_{\theta }(x^{(i)})}{\partial \theta }=h_{\theta }^{{}^{\prime }}(x^{(i)})(x^{(i)}{)}^T\end{array}$$
$$\begin{array}{rl}\frac{\partial J(\theta )}{\partial \theta }& =-\frac{1}{m}\left[\sum _{i=1}^m\frac{y^{(i)}}{h_{\theta }(x^{(i)})}-\frac{(1-y^{(i)})}{log(1-h_{\theta }(x^{(i)}))}\right]\frac{\partial h_{\theta }(x^{(i)})}{\partial \theta }\\ & =-\frac{1}{m}\left[\sum _{i=1}^m\frac{y^{(i)}}{h_{\theta }(x^{(i)})}-\frac{(1-y^{(i)})}{log(1-h_{\theta }(x^{(i)}))}\right]h_{\theta }^{{}^{\prime }}(x^{(i)})(x^{(i)}{)}^T\\ & =\frac{1}{m}\sum _{i=1}^m\frac{h_{\theta }^{{}^{\prime }}(x^{(i)})(x^{(i)}{)}^T}{h_{\theta }(x^{(i)})(1-h_{\theta }(x^{(i)}))}(h_{\theta }(x^{(i)})-y^{(i)})\\ & =\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})(x^{(i)}{)}^T\\ & =\frac{1}{m}{X}^T(h_{\theta }(X)-y)\end{array}$$请注意，$x^{(i)}$是数据条，是行向量；$h_{\theta }(X)-y$是列向量；$\theta$是关于各属性权值的列向量。

利用Gradient Decent求解最优值$\theta$

Repeat{θj:=θj−α∑i=1m(hθ(x(i))−y(i))xj(i)}simultaneously update all θj\begin{aligned} Repeat\{&\\ &\theta_j:=\theta_j-\alpha\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_j\\ &\}simultaneously\ update\ all\ \theta_j \end{aligned}Repeat{​θj​:=θj​−αi=1∑m​(hθ​(x(i))−y(i))xj(i)​}simultaneously update all θj​​
你妈，被这些鬼东西搞了一下午，fuck！