Adaboost模型公式的参数推导_bohart-adams模型-优快云博客

博客围绕Adaboost模型展开，先给出由基模型加权得到的模型fm(x)，接着阐述基学习器和整个Adaboost模型的损失函数。通过一系列推导，求解损失函数中的αm和Gm(x)，还得出αm的表达式以及ωim+1的推导结果。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

假设最后根据各个基模型 $Gi(x)i∈[1,m]G_i(x) {i\in[1,m]}$ ,各个基模型重要程度为 $αi\alpha_i$ 加权得到的模型为 $f_m(x)$ ,其中 $y∈{−1,1}y\in{\{-1,1\}}$

$f_m(x) = \sum_{i = 1}^{m}\alpha_{i} G_i(x)$
$f_m(x) = \sum_{i=1}^{m-1}\alpha_{i} G_i(x)+\alpha_m G_m(x)$
$f_m(x) = f_{m-1}(x) +\alpha_m G_m(x)$

基学习器的损失函数为 $L(y,f(x)) = e^{-yf(x)}$

所以整个Adaboost模型的损失函数为：
$\sum_{i =1}^{n}{exp{(-y_if(x_i))}}$

该损失函数的 $αm\alpha_m$ 和 $G_m(x)$ 是需要求得的

$(αm,Gm(x))=argminαm,Gm∑i=1nexp(−yi(fm−1(xi)+αmGm(xi)))(\alpha_m,G_m(x))= \mathop{argmin}\limits_{\alpha_m,G_m} \sum_{i=1}^{n}exp{(-y_i(f_{m-_1} (x_i)+\alpha_m G_m(x_i)))}$

其中 $∑i=1nexp(−yi(fm−1(xi)+αmGm(xi))=∑i=1nexp(−yifm−1(xi))exp(−yiαmGm(xi))\sum_{i=1}^{n}exp(-y_i(f_{m-1}(x_i)+\alpha_m G_m(x_i))=\sum_{i=1}^{n}exp(-y_if_{m-1}(x_i))exp(-y_i\alpha_m G_m(x_i))$ –(1)

设 $ωim=exp(−yifm−1(xi))\omega_i^{m} = exp(-y_if_{m-1}(x_i))$ ,将其带入式(1)得到：

$∑i=1nωimexp(−yiαmGm(xi))\sum\limits_{i=1}^{n}\omega_ i^ {m} exp(-y_i\alpha_mG_m(x_i))$ –(2)

当 $y_i=G_m(x_i)$ 时， $y_iG_m(x_i) = 1$ ,当 $yi≠Gm(xi)y_i\neq G_m(x_i)$ 时， $y_iG_m(x_i) = -1$ ,所以式(2)可以写成

$∑yi=Gm(xi)nωimexp(−αm)+∑yi≠Gm(xi)nωimexp(αm)\sum\limits_{y_i=G_m(x_i)}^{n}\omega_{i}^{m}exp(-\alpha_m)+\sum\limits_{y_i \neq G_m(x_i)}^{n}\omega_{i}^{m} exp(\alpha_m)$

$=∑yi=Gm(xi)nωimexp(−αm)+∑yi≠Gm(xi)nωimexp(αm)+∑yi≠Gm(xi)nωimexp(−αm)−∑yi≠Gm(xi)nωimexp(−αm)=\sum\limits_{y_i=G_m(x_i)}^{n}\omega_{i}^{m}exp(-\alpha_m)+\sum\limits_{y_i \neq G_m(x_i)}^{n} \omega_{i}^{m}exp(\alpha_m) +\sum\limits_{y_i \neq G_m(x_i)}^{n} \omega_{i}^{m}exp(-\alpha_m) -\sum\limits_{y_i \neq G_m(x_i)}^{n} \omega_{i}^{m}exp(-\alpha_m)$

$=e−αm∑i=1nωim−(e−αm−eαm)∑yi≠Gm(xi)nωim=e^{-\alpha_m}\sum_{i=1}^{n}\omega_{i}^{m}-(e^{-\alpha_m}-e^{\alpha_m})\sum\limits_{y_i\neq G_m(x_i)}^{n}\omega_{i}^{m}$

$=e−αm∑i=1nωim−(e−αm−eαm)∑i=1nωimI(yi≠Gm(xi))=e^{-\alpha_m}\sum\limits_{i=1}^{n}{\omega_{i}^{m}}-(e^{-\alpha_m}-e^{\alpha_m})\sum\limits_{i=1}^{n} {\omega_{i}^{m}I(y_i \neq G_m(x_i))}$ –(3)

要使式(3)最小，则 $G_m(x)$ 应该取 $argminG∑i=1nωimI(yi≠Gm(xi))\mathop{argmin}\limits_{G}\sum\limits_{i=1}^{n}\omega_{i}^{m}I(y_i \neq G_m(x_i))$

因为每次求每个点的重要程度时都会除以总和，所以 $∑i=1nωim=1\sum\limits_{i=1}^{n}{\omega_{i}^{m}}=1$ ，对式(3)中的 $αm\alpha_m$ 求导得到：

$−e−αm−(−e−αm−eαm)∑i=1nωinI(yi≠Gm(xi))=0-e^{-\alpha_m}-(-e^{-\alpha_m}-e^{\alpha_m})\sum\limits_{i=1}^{n}\omega_{i}^{n}I(y_i \neq G_m(x_i)) = 0$

$e−αm(1−∑i=1nωinI(yi≠Gm(xi)))=eαm∑i=1nI(yi≠Gm(xi))e^{-\alpha_m}(1-\sum\limits_{i=1}^{n}{\omega_{i}^{n}I(y_i\neq G_m(x_i))})= e^{\alpha_m}\sum\limits_{i=1}^{n}I(y_i \neq G_m(x_i))$ –(4)

令 $∑i=1nωi=1nI(yi≠Gm(xi))=em\sum\limits_{i=1}^{n}\omega_{i=1}^{n}I(y_i \neq G_m(x_i))= e_m$

化简式(4)得到 $αm=12ln(1−emem)\alpha_m = \frac{1}{2}ln(\frac{1-e_m}{e_m})$

因为 $ωim=exp(−yifm−1(xi))\omega_i^{m} = exp(-y_if_{m-1}(x_i))$ 可以推导得

$ωim+1=exp(−yifm(xi))=exp(−yi(fm−1(xi)+αmGm(xi)))=exp(−yifm−1)exp(−yiαmGm(xi))\omega_i^{m+1} = exp(-y_if_{m}(x_i))=exp(-y_i(f_{m-1}(x_i)+\alpha_m G_m(x_i)))=exp(-y_if_{m-1})exp(-y_i \alpha_m G_m(x_i))$