逻辑回归（Logistics Regression）

最新推荐文章于 2025-04-21 11:01:14 发布

原创最新推荐文章于 2025-04-21 11:01:14 发布 · 139 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#逻辑回归 #logistics

NLP 同时被 2 个专栏收录

14 篇文章

订阅专栏

基础算法

7 篇文章

订阅专栏

逻辑回归，虽然名字中带有“回归”，但却是解决分类问题的一种基础算法。本文主要介绍其原理。

逻辑回归采用最大似然概率作为其损失函数，最大似然概率为：

$max\prod_{i=1}^{N}p(y_{i}|x_{i}, w, b) \ \ \ \ \ \ \ \ \ \ \(1)$

对于二分类问题

$\begin{aligned} p_{1} = p(y=1|x,w,b) = \frac{1}{1+e^{-(w\cdot x+b)}}, \ y=1\ \ \ \ \ \ \ (2) \\ p_{0} = p(y=0|x,w,b) = \frac{1}{1+e^{-(w\cdot x+b)}}, \ y=0\ \ \ \ \ \ \ (3) \end{aligned}$

将以上(2)和(3)进行合并

$p(y|x,w,b)=p_{1}^{y_{i}}\cdot p_{0}^{1-y_{i}} \ \ \ \ \ \ \ \ \ (4)$

将(4)代入(1)，再取对数

$max \sum_{i=1}^{N}[y_{i}\cdot log\phi (x_{i})+(1-y_{i})\cdot log(1-\phi (x_{i}))] \ \ \ \ \ \ \ \ (5)$

其中， $\phi (x_{i}) = p_{0}=p_{1}= \frac{1}{1+e^{-(w\cdot x_{i}+b)}}$

则，逻辑回归的损失函数可写为：

$L = -\frac{1}{N} \sum_{i=1}^{N}[y_{i}\cdot log\phi (x_{i})+(1-y_{i})\cdot log(1-\phi (x_{i}))] \ \ \ \ \ \ \ \ (6)$

对 $w$ 和 $b$ 求导

$\begin{align*} \ \frac{\partial L}{\partial w}&=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}\cdot \frac{1}{\phi (x_{i})}\cdot \frac{\partial \phi (x_{i})}{\partial w}-(1-y_{i})\cdot \frac{1}{1-\phi (x_{i})}\cdot \frac{\partial \phi x_{i}}{\partial w}] \\ &=-\frac{1}{N}\sum_{i=1}^{N}\frac{y_{i}-\phi (x_{i})}{\phi (x_{i})\cdot(1-\phi (x_{i}))} \cdot \frac{\partial \phi (x_{i})}{\partial w} \\ &=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}-\phi (x_{i})]\cdot x_{i} \ \ \ \ \ \ \ \ \ \ \ (7) \end{align*}$

$\begin{align*} \ \frac{\partial L}{\partial b}&=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}\cdot \frac{1}{\phi (x_{i})}\cdot \frac{\partial \phi (x_{i})}{\partial b}-(1-y_{i})\cdot \frac{1}{1-\phi (x_{i})}\cdot \frac{\partial \phi x_{i}}{\partial b}] \\ &=-\frac{1}{N}\sum_{i=1}^{N}\frac{y_{i}-\phi (x_{i})}{\phi (x_{i})\cdot(1-\phi (x_{i}))} \cdot \frac{\partial \phi (x_{i})}{\partial b} \\ &=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}-\phi (x_{i})] \ \ \ \ \ \ \ \ \ \ \ (8) \end{align*}$ .

训练过程中，参数更新

$\begin{align*} w:&=w-\lambda\cdot\frac{\partial L}{\partial w} = w+\lambda\cdot \frac{1}{N}\sum_{i=1}^{N}[y_{i}-\phi (x_{i})]\cdot x_{i} \ \ \ \ \ \ \ \ (9) \\ b:&= b-\lambda\cdot\frac{\partial L}{\partial b} =b+\lambda\cdot\frac{1}{N}\sum_{i=1}^{N}[y_{i}-\phi (x_{i})] \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (10) \end{align*}$