PCA, PCA whitening and ZCA whitening in 2D

最新推荐文章于 2019-06-30 15:00:55 发布

xiaojidan2011

最新推荐文章于 2019-06-30 15:00:55 发布

阅读量1.7k

点赞数

CC 4.0 BY-SA版权

分类专栏：图像处理&机器视觉&matlab

本文链接：https://blog.youkuaiyun.com/xiaojidan2011/article/details/8800923

图像处理&机器视觉&matlab 专栏收录该内容

51 篇文章

订阅专栏

本文介绍了PCA(主成分分析)的基本步骤及其在数据预处理中的应用，包括如何实现数据均值归零、协方差矩阵计算、特征向量求解及数据降维等关键操作，并展示了PCA白化与ZCA白化的实现方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Steps:

Step 0: Load data

Step 1a: Implement PCA to obtain U

Step 1b: Compute xRot, the projection on to the eigenbasis

Step 2: Reduce the number of dimensions from 2 to 1.

Step 3: PCA Whitening

Step 3: ZCA Whitening

简单介绍：

First, we need to ensure that the data has (approximately) zero-mean. For natural images, we achieve this (approximately) by subtracting the mean value of each image patch.

We achieve this by computing the mean for each patch and subtracting it for each patch. In Matlab, we can do this by using

avg = mean(x, 1);     % Compute the mean pixel intensity value separately for each patch. 
x = x - repmat(avg, size(x, 1), 1);

Next, we need to compute $\textstyle \Sigma = \frac{1}{m} \sum_{i=1}^m (x^{(i)})(x^{(i)})^T$ . If you're implementing this in Matlab (or even if you're implementing this in C++, Java, etc., but have access to an efficient linear algebra library), doing it as an explicit sum is inefficient. Instead, we can compute this in one fell swoop as

sigma = x * x' / size(x, 2);

(Check the math yourself for correctness.) Here, we assume that $x$ is a data structure that contains one training example per column (so, $x$ is a $\textstyle n$ -by- $\textstyle m$ matrix).

Next, PCA computes the eigenvectors of $Σ$ . One could do this using the Matlab eig function. However, because $Σ$ is a symmetric positive semi-definite matrix, it is more numerically reliable to do this using the svd function. Concretely, if you implement

[U,S,V] = svd(sigma);

then the matrix $U$ will contain the eigenvectors of $Sigma$ (one eigenvector per column, sorted in order from top to bottom eigenvector), and the diagonal entries of the matrix $S$ will contain the corresponding eigenvalues (also sorted in decreasing order). The matrix $V$ will be equal to transpose of $U$ , and can be safely ignored.

(Note: The svd function actually computes the singular vectors and singular values of a matrix, which for the special case of a symmetric positive semi-definite matrix---which is all that we're concerned with here---is equal to its eigenvectors and eigenvalues. A full discussion of singular vectors vs. eigenvectors is beyond the scope of these notes.)

Finally, you can compute $\textstyle x_{\rm rot}$ and $\textstyle \tilde{x}$ as follows:

xRot = U' * x;          % rotated version of the data. 
xTilde = U(:,1:k)' * x; % reduced dimension representation of the data, 
                        % where k is the number of eigenvectors to keep

This gives your PCA representation of the data in terms of $\textstyle \tilde{x} \in \Re^k$ . Incidentally, if $x$ is a $\textstyle n$ -by- $\textstyle m$ matrix containing all your training data, this is a vectorized implementation, and the expressions above work too for computing $x rot$ and $\tilde{x}$ for your entire training set all in one go. The resulting $x rot$ and $\tilde{x}$ will have one column corresponding to each training example.

To compute the PCA whitened data $\textstyle x_{\rm PCAwhite}$ , use

xPCAwhite = diag(1./sqrt(diag(S) + epsilon)) * U' * x;

Since $S$ 's diagonal contains the eigenvalues $\textstyle \lambda_i$ , this turns out to be a compact way of computing $\textstyle x_{{\rm PCAwhite},i} = \frac{x_{{\rm rot},i} }{\sqrt{\lambda_i}}$ simultaneously for all $\textstyle i$ .

Finally, you can also compute the ZCA whitened data $\textstyle x_{\rm ZCAwhite}$ as:

xZCAwhite = U * diag(1./sqrt(diag(S) + epsilon)) * U' * x;

本文简单的学习下PCA的相关知识，具体内容详见斯坦福公开课，Ng老师的ML课程。。。。