机器学习——线性回归(Linear Regression)

最新推荐文章于 2025-01-20 21:37:21 发布

daocaoren_

最新推荐文章于 2025-01-20 21:37:21 发布

阅读量284

点赞数

CC 4.0 BY-SA版权

分类专栏：机器学习

本文链接：https://blog.youkuaiyun.com/daocaoren_/article/details/96989715

机器学习专栏收录该内容

13 篇文章

订阅专栏

本文详细介绍了线性回归算法的工作原理及实现步骤。包括输入特征集和标签集的数学表示、伪逆的计算方法以及参数ω的求解过程。通过最小化平方误差来找到最佳拟合直线。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

算法描述：

Linear Regression Algorithm
1.输入特征集X和标签集y: ${\left[ {\begin{array}{} { - - x_1^T - - }\\ { - - x_2^T - - }\\ \cdots \\ { - - x_N^T - - } \end{array}} \right]_{N \times (d + 1)}}$ ${\left[ {\begin{array}{} {{y_1}}\\ {{y_2}}\\ \cdots \\ {{y_N}} \end{array}} \right]_{N \times 1}}$ 2.计算伪逆(pseudo-inverse)： $x†=(xTx)−1xT((d+1)×N){x^\dag } = {({x^T}x)^{ - 1}}{x^T}((d + 1) \times N)$ 3.返回 $ωLIN=x†y((d+1)×1){\omega _{LIN}} = {x^\dag }y((d + 1) \times 1)$

Linear Regression Algorithm

1.输入特征集X和标签集y:

{\left[ {\begin{array}{} { - - x_1^T - - }\\ { - - x_2^T - - }\\ \cdots \\ { - - x_N^T - - } \end{array}} \right]_{N \times (d + 1)}}

{\left[ {\begin{array}{} {{y_1}}\\ {{y_2}}\\ \cdots \\ {{y_N}} \end{array}} \right]_{N \times 1}}

2.计算伪逆(pseudo-inverse)：

x†=(xTx)−1xT((d+1)×N){x^\dag } = {({x^T}x)^{ - 1}}{x^T}((d + 1) \times N)

3.返回

ωLIN=x†y((d+1)×1){\omega _{LIN}} = {x^\dag }y((d + 1) \times 1)

计算过程：

$Ein(ω)=1N∑n=1N(ωTxn−yn)2=1N∑n=1N(xnTω−yn)2(内积可交换)=1N∥x1Tω−y1x2Tω−y2⋯xNTω−yN∥2=∥[−−x1T−−−−x2T−−⋯−−xNT−−]ω−[y1y2⋯yN]∥2=1N∥X⎵N×(d+1)ω⎵(d+1)×1−y⎵N×1∥2  \begin{array}{l} {E_{in}}(\omega ) = \frac{1}{N}{\sum\limits_{n = 1}^N {({\omega ^T}{x_n} - {y_n})} ^2} = \frac{1}{N}{\sum\limits_{n = 1}^N {(x_n^T\omega - {y_n})} ^2}(内积可交换) \\ \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} = \frac{1}{N}{\left\| {\begin{array}{} {x_1^T\omega - {y_1}}\\ {x_2^T\omega - {y_2}}\\ \cdots \\ {x_N^T\omega - {y_N}} \end{array}} \right\|^2} = {\left\| {\left[ {\begin{array}{} { - - x_1^T - - }\\ { - - x_2^T - - }\\ \cdots \\ { - - x_N^T - - } \end{array}} \right]\omega - \left[ {\begin{array}{} {{y_1}}\\ {{y_2}}\\ \cdots \\ {{y_N}} \end{array}} \right]} \right\|^2}\\ \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} = \frac{1}{N}{\left\| {\underbrace X_{N \times (d + 1)}\underbrace \omega _{(d + 1) \times 1} - \underbrace y_{N \times 1}} \right\|^2} \\ \;\; \end{array}{}$

这里的误差用 平方误差(Squared Error) 来衡量，即，
$err(y^,y)=(y^−y)2err(\hat y,y) = {(\hat y - y)^2}$
在线性回归里就是：
$min⁡ωEin(ω)=1N∥xω−y∥2\mathop {\min }\limits_\omega {E_{in}}(\omega ) = \frac{1}{N}{\left\| {x\omega - y} \right\|^2}$