最小二乘法
J(w)=12∑i=1N(wTxi−yi)2J(w)=\Large\frac{1}{2}\normalsize\sum\limits_{i=1}^N(w^Tx_i-y_i)^2J(w)=21i=1∑N(wTxi−yi)2,其中w=(w1,w2,...,wn,b)T,xi=(xi1,xi2,...,xin,1)Tw=(w_1,w_2,...,w_n,b)^T,x_i=(x_i^1,x_i^2,...,x_i^n,1)^Tw=(w1,w2,...,wn,b)T,xi=(xi1,xi2,...,xin,1)T
设:
X=[x11x12...x1nx21x22...x2n............xN1xN2...xNn]
X= \left[
\begin{matrix}
x_1^1 & x_1^2 & ... &x_1^n \\
x_2^1 & x_2^2 & ... &x_2^n \\
...& ...& ...& ...\\
x_N^1 & x_N^2 & ... &x_N^n \\
\end{matrix}
\right]
X=⎣⎢⎢⎡x11x21...xN1x12x22...xN2............x1nx2n...xNn⎦⎥⎥⎤
Y=(y1,y2,...,yN)TY=(y_1,y_2,...,y_N)^TY=(y1,y2,...,yN)T
则
J(w)=12∣∣Xw−Y∣∣2J(w)=\Large\frac{1}{2}\normalsize||Xw-Y||^2J(w)=21∣∣Xw−Y∣∣2,其中||为第二范式
J(w)=12(Xw−Y)T(Xw−Y)J(w)=\Large\frac{1}{2}\normalsize(Xw-Y)^T(Xw-Y)J(w)=21(Xw−Y)T(Xw−Y)
=12(wTXT−YT)(Xw−Y)=\Large\frac{1}{2}\normalsize(w^TX^T-Y^T)(Xw-Y)=21(wTXT−YT)(Xw−Y)
=12(wTXTXw+YTY−wTXTY−YTXw)=\Large\frac{1}{2}\normalsize(w^TX^TXw+Y^TY-w^TX^TY-Y^TXw)=21(wTXTXw+YTY−wTXTY−YTXw)
则
∂J(w)∂ w=XTXw−XTY=0\frac{\partial J(w)}{\partial \ w}=X^TXw-X^TY=0∂ w∂J(w)=XTXw−XTY=0
w=(XTX)−1XTYw=(X^TX)^{-1}X^TYw=(XTX)−1XTY
OK
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
