求导方法

最新推荐文章于 2023-08-13 20:03:05 发布

转载最新推荐文章于 2023-08-13 20:03:05 发布 · 292 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：https://my.oschina.net/cttmayi/blog/812518

文章标签：

#python

2019独角兽企业重金招聘Python工程师标准>>>

为了学习"梯度上升/下降算法", 不得不大补高中数学, 重新学习求导.

基本公式

\begin{align*}
(c)' &= 0 \\
(cx)' &= c \\
(x^a)' &= ax^{a-1} \\
(\frac{c}{v})' &= - \frac{cv'}{v^2} \Longrightarrow (\frac{1}{x})' = - \frac{1}{x^2} \\
(\log_a x)' &= \frac{1}{x\ln a} \Longrightarrow (\ln x)' = \frac{1}{x} \\
(a^x)' &= a^x\ln a \Longrightarrow (e^x)' = e^x \\
\end{align*}

导数的四则运算法则

$$ (u \pm v)' = u' \pm v' $$
$$ (uv)' = u'v + uv' $$
$$ (\frac{u}{v})' = \frac{u'v - uv'}{v^2} $$

复合函数运算法则

$$ y = f(u), u = g(x) $$
$$ (f[g(x)])' = f'(u)g'(x) $$

设, 那么

推导"梯度上升算法"

\begin{align*}
& \Big[\ln \Big(\frac{1}{1+e^{-x}})^y (1-\frac{1}{1+e^{-x}})^{(1-y}\Big)\Big]' \\
= &\Big[y\ln (\frac{1}{1+e^{-x}}) + (1-y)\ln (\frac{e^{-x}}{1+e^{-x}}) \Big]' \\
= &y(1+e^{-x})(\frac{1}{1+e^{-x}})' + (1-y)\frac{(1+e^{-x})}{e^{-x}} (\frac{e^{-x}}{1+e^{-x}})' \\
= &y(1+e^{-x})\frac{e^{-x}}{(1+e^{-x})^2}x' + (1-y)\frac{(1+e^{-x})}{e^{-x}}\frac{-e^{-x}}{(1+e^{-x})^2}x' \\
= &y\frac{e^{-x}}{1+e^{-x}}x' + (y-1)\frac{1}{1+e^{-x}}x' \\
= &(y-\frac{1}{1+e^{-x}})x'
\end{align*}

其中, 对进行求导

\begin{align*}
& (y-\frac{1}{1+e^{-x}})x' \\
= & (y-\frac{1}{1+e^{-x}})(w_0 + w_1x_1 + \dots + w_jx_j + \dots + w_nx_n)' \\
= & (y-\frac{1}{1+e^{-x}})x_j
\end{align*}

最终结果

\begin{align*}
&\Big[\ln \Big( \prod_{i} (\frac{1}{1+e^x_i})^yi (1-\frac{1}{1+e^{x_i}})^{(1-{y_i})}\Big)\Big]' \\
= &\sum_i(y-\frac{1}{1+e^{-x_i}})x_j
\end{align*}