softmax 回归原理及python实现

最新推荐文章于 2024-12-04 08:56:45 发布

howardact

最新推荐文章于 2024-12-04 08:56:45 发布

阅读量4.5k

点赞数 8

分类专栏： machineLearning

本文链接：https://blog.youkuaiyun.com/howardact/article/details/54581901

版权

softmax回归python实现程序

1、Logistic回归：

1.1、逻辑回归数据集

[(x (1), y (1)), (x (2), y (2)), . . ., (x (m), y (m))]

$\left [(x^{(1)},y^{(1)}),(x^{(2)},y^{(2)}),...,(x^{(m)},y^{(m)})\right]$

y \in {0, 1}

$y \in \{0,1\}$

1.2、样本发生的概率，即y取1的概率：

h θ (x) = 1 1 + e x p ( - θ \cdot x )

$h_{\theta}(x)=\frac{1}{1+ exp(-\theta\cdot x)}$

1.3、整个样本的似然函数为：

似然函数
$L = \prod h θ (x (i)) y (i) (1 - h θ (x (i))) 1 - y (i)$ $L=\prod h_{\theta}(x^{(i)})^{y^{(i)}}\left(1-h_{\theta}(x^{(i)})\right)^{1-y^{(i)}}$
对数似然函数为：
$l o g L = \sum i = 1 m (y (i) l o g (h θ (x (i)) + (1 - y (i)) l o g (1 - h θ (x (i))))$ $logL=\sum_{i=1}^{m} \left(y^{(i)}log(h_{\theta}(x^{(i)}) +(1-y^{(i)})log(1-h_{\theta}(x^{(i)}))\right)$

1.4、代价函数，及代价函数偏导：

代价函数
$J (θ) = - 1 m \sum i = 1 m (y (i) l o g (h θ (x (i)) + (1 - y (i)) l o g (1 - h θ (x (i))))$ $J(\theta) = -\frac{1}{m}\sum_{i=1}^{m} \left(y^{(i)}log(h_{\theta}(x^{(i)}) +(1-y^{(i)})log(1-h_{\theta}(x^{(i)}))\right)$
代价函数的偏导数：
$\partial J ( θ ) \partial θ j = - 1 m (\sum i = 1 m (y (i) - h θ (x (i))) x (i))$ $\frac{\partial J(\theta)}{\partial \theta_j}=-\frac{1}{m}\left(\sum_{i=1}^{m}\left(y^{(i)}-h_{\theta}(x^{(i)})\right)x^{(i)}\right)$

1.5、梯度下降更新参数：

θ j : = : = θ j - α \partial J ( θ ) \partial θ j θ j + α m (\sum i = 1 m (y (i) - h θ (x (i))) x (i) j) (1) (2)

$\begin{eqnarray} \theta_j &:=& \theta_j - \alpha \frac{\partial J(\theta)}{\partial \theta_j} \\ &:=& \theta_j + \frac{\alpha}{m}\left(\sum_{i=1}^{m}\left(y^{(i)}-h_{\theta}(x^{(i)})\right)x_j^{(i)}\right) \end{eqnarray}$