PCA, PCA whitening and ZCA whitening in 2D

本文介绍了PCA(主成分分析)的基本步骤及其在数据预处理中的应用,包括如何实现数据均值归零、协方差矩阵计算、特征向量求解及数据降维等关键操作,并展示了PCA白化与ZCA白化的实现方法。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

  

Steps:

 Step 0: Load data

  Step 1a: Implement PCA to obtain U

  Step 1b: Compute xRot, the projection on to the eigenbasis

  Step 2: Reduce the number of dimensions from 2 to 1.

  Step 3: PCA Whitening

  Step 3: ZCA Whitening

简单介绍:

First, we need to ensure that the data has (approximately) zero-mean. For natural images, we achieve this (approximately) by subtracting the mean value of each image patch.

We achieve this by computing the mean for each patch and subtracting it for each patch. In Matlab, we can do this by using

avg = mean(x, 1);     % Compute the mean pixel intensity value separately for each patch. 
x = x - repmat(avg, size(x, 1), 1);

Next, we need to compute \textstyle \Sigma = \frac{1}{m} \sum_{i=1}^m (x^{(i)})(x^{(i)})^T. If you're implementing this in Matlab (or even if you're implementing this in C++, Java, etc., but have access to an efficient linear algebra library), doing it as an explicit sum is inefficient. Instead, we can compute this in one fell swoop as

sigma = x * x' / size(x, 2);

(Check the math yourself for correctness.) Here, we assume that x is a data structure that contains one training example per column (so, x is a \textstyle n-by-\textstyle m matrix).

Next, PCA computes the eigenvectors of Σ. One could do this using the Matlab eig function. However, because Σ is a symmetric positive semi-definite matrix, it is more numerically reliable to do this using the svd function. Concretely, if you implement

[U,S,V] = svd(sigma);

then the matrix U will contain the eigenvectors of Sigma (one eigenvector per column, sorted in order from top to bottom eigenvector), and the diagonal entries of the matrix S will contain the corresponding eigenvalues (also sorted in decreasing order). The matrix V will be equal to transpose of U, and can be safely ignored.

(Note: The svd function actually computes the singular vectors and singular values of a matrix, which for the special case of a symmetric positive semi-definite matrix---which is all that we're concerned with here---is equal to its eigenvectors and eigenvalues. A full discussion of singular vectors vs. eigenvectors is beyond the scope of these notes.)

Finally, you can compute \textstyle x_{\rm rot} and \textstyle \tilde{x} as follows:

xRot = U' * x;          % rotated version of the data. 
xTilde = U(:,1:k)' * x; % reduced dimension representation of the data, 
                        % where k is the number of eigenvectors to keep

This gives your PCA representation of the data in terms of \textstyle \tilde{x} \in \Re^k. Incidentally, if x is a \textstyle n-by-\textstyle m matrix containing all your training data, this is a vectorized implementation, and the expressions above work too for computing xrot and \tilde{x} for your entire training set all in one go. The resulting xrot and \tilde{x} will have one column corresponding to each training example.

To compute the PCA whitened data \textstyle x_{\rm PCAwhite}, use

xPCAwhite = diag(1./sqrt(diag(S) + epsilon)) * U' * x;

Since S's diagonal contains the eigenvalues \textstyle \lambda_i, this turns out to be a compact way of computing \textstyle x_{​{\rm PCAwhite},i} = \frac{x_{​{\rm rot},i} }{\sqrt{\lambda_i}} simultaneously for all \textstyle i.

Finally, you can also compute the ZCA whitened data \textstyle x_{\rm ZCAwhite} as:

xZCAwhite = U * diag(1./sqrt(diag(S) + epsilon)) * U' * x;

 

 

 

本文简单的学习下PCA的相关知识,具体内容详见斯坦福公开课,Ng老师的ML课程。。。。

 

 

### ZCA 白化的原理 ZCA 白化是一种特殊的线性变换技术,旨在去除数据中的冗余并保持其结构特性。通过该过程,不仅能够消除不同特征之间的相关性,还能让转换后的数据分布尽可能接近原始输入数据[^1]。 为了实现这一点,在执行 ZCA 白化前需先中心化数据样本,即确保每项特征均值为零: \[ \frac{1}{m}\sum_{i}x^{(i)}=0 \] 其中 \( m \) 表示训练集中实例的数量,\( x^{(i)} \) 是第 i 个观测向量。此步骤有助于后续协方差矩阵计算更加稳定有效[^2]。 接着,基于已知的数据集构建协方差矩阵,并对其进行特征分解得到一组正交基底及其对应的特征值。随后利用这些信息来调整各个方向上的尺度因子,从而达到标准化的目的。特别地,对于 ZCA 白化而言,会采用如下公式进行变换: ```matlab % 假设 X 已经被居中处理过 [U, S, ~] = svd(cov(X)); epsilon = 1e-5; X_zca_white = U * diag(1 ./ sqrt(diag(S) + epsilon)) * U' * X; ``` 这里 `U` 和 `S` 分别代表奇异值分解后获得的左奇异向量矩阵与对角阵;`epsilon` 则是为了防止除数为零引入的小常数值[^3]。 值得注意的是,尽管 PCA 白化同样能起到去相关的作用,但相比之下,ZCA 更倾向于维持原有空间内的几何关系不变——即使经过白化之后的新坐标系仍然保持着相似的空间布局特点[^4]。 另外,由于 ZCA 不涉及降维操作,因此一般情况下不会减少数据维度数量 n ,而是通过对整个特征集合施加均匀缩放和平移的方式完成白化进程[^5]。 ### 应用场景 在机器学习领域特别是计算机视觉任务当中,图像预处理阶段经常运用到 ZCA 白化方法。这是因为自然图片往往存在较强的内部关联性和局部模式重复现象,而经过适当变换可提高模型收敛速度及泛化能力。 此外,在某些特定场合下(比如神经网络训练),如果希望保留更多关于输入信号间相对位置的信息,则更推荐选用 ZCA 而不是简单的 PCA 方法来进行预处理工作。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值