θ
^
=
(
X
T
X
+
λ
I
)
−
1
X
y
\hat{\theta} = (X^TX + \lambda I)^{-1}Xy
θ^=(XTX+λI)−1Xy
For any vector
v
∈
R
d
v \in \mathbb{R}^d
v∈Rd, we have
v
T
X
T
X
v
=
(
X
v
)
T
(
X
v
)
>
=
0
v^TX^TXv = (Xv)^T(Xv) >= 0
vTXTXv=(Xv)T(Xv)>=0, so it is a positive semi-definite. Also, we know
λ
>
0
\lambda >0
λ>0 so that
λ
I
\lambda I
λI is already a positive definite matrix.
Therefore,
v
T
(
X
T
X
+
λ
I
)
v
=
(
X
v
)
T
(
X
v
)
+
λ
v
T
I
v
>
=
v
T
(
λ
)
v
>
0
v^T(X^TX + \lambda I)v=(Xv)^T(Xv) + \lambda v^TIv >= v^T(\lambda)v>0
vT(XTX+λI)v=(Xv)T(Xv)+λvTIv>=vT(λ)v>0
is a positve definite and hence invertible.
Ridge Linear Regression Estimation Invertible
最新推荐文章于 2022-04-21 09:38:22 发布
本文探讨了在最小二乘法中加入正则化项如何保证矩阵(XTX+λI)的正定性和可逆性。通过数学推导，证明了对于任意向量v，该矩阵与v的乘积大于等于零，从而确认其为正定矩阵，确保了其可逆性。
扫一扫
2万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
