机器学习实战第五章——Logistic回归

回归系数更新公式 $weigths=weights+alpha\times dataMatrix.transpose()\times error$ 公式原理

损失函数定义

$$J(\theta )=\frac{1}{2m}\sum _{i=1}^m(\widehat{y_i}-y_i{)}^2=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x_i)-y_i{)}^2$$
即为预测函数值（$\hat{y}$）与真实值（$y$）差的平方和，其中$\theta$为此时预测函数所使用的回归系数（向量）

梯度上升（下降）法回归系数更新公式

书中所提到梯度上升算法迭代公式为：
$$w=w\pm \alpha {▽}_{w}f(w)$$
即：
$${\theta }_j={\theta }_j\pm \alpha \frac{\partial }{\partial {\theta }_j}J(\theta )$$
对$\frac{\partial }{\partial {\theta }_j}J(\theta )$推导：
$$\begin{array}{rl}\frac{\partial }{\partial {\theta }_j}J(\theta )& =\frac{\partial }{\partial {\theta }_j}\frac{1}{2}(h_{\theta }(x)-y{)}^2\\ & =\frac{1}{2}\times 2\times (h_{\theta }(x)-y)\times \frac{\partial }{\partial {\theta }_j}(h_{\theta }(x)-y)\\ & =(h_{\theta }(x)-y)\times \frac{\partial }{\partial {\theta }_j}(({\theta }_0{x}_0+{\theta }_1{x}_1+\cdots +{\theta }_n{x}_n)-y)\\ & =(h_{\theta }(x)-y)\times x_j\end{array}$$
则回归系数${\theta }_j$：
$${\theta }_j={\theta }_j\pm \alpha \times (h_{\theta }(x)-y)x_j$$
回归向量$\theta$：
$$\theta =\theta \pm \alpha \times (h_{\theta }(x)-y)x$$
$\theta$（回归系数(向量)）即为书中所提到的$weights$，$\alpha$为步长，$x$为输入数据$dataMatrix$
在此以$\theta =\left[\begin{array}{c}{\theta }_0\\ {\theta }_1\end{array}\right]$为例，则可得更新量为：
$$\left[\begin{array}{c}{\theta }_0\\ {\theta }_1\end{array}\right]\to \left[\begin{array}{c}{\theta }_0\pm \alpha \frac{1}{m}{\sum }_{i=1}^m(h_{\theta }(x_i)-y_i)\\ {\theta }_1\pm \alpha \frac{1}{m}{\sum }_{i=1}^m(h_{\theta }(x_i)-y_i)\end{array}\right]$$