[LA] Centering a data set

最新推荐文章于 2025-08-22 12:17:27 发布

原创最新推荐文章于 2025-08-22 12:17:27 发布 · 588 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#LA

linear algebra 专栏收录该内容

9 篇文章

订阅专栏

本文详细介绍了如何通过中心化矩阵处理数据集，并将其应用于不同组间协方差矩阵的计算中。文中推导了中心化矩阵的性质及其与正交对比的关系，最终展示了如何将这些理论用于实际的数据分析。

Centering data set
Application

1. Centering data set

If we have a data set $X\in \mathbb{R}^{n\times p}$ (each row is a sample), then column mean of this data set can be expressed in

X ¯ = 1 n X T 1 n

$\bar X = \frac{1}{n}X^T 1_n$
So the centered data set is

X c = X - 1 n X ¯ T = X - 1 n 1 n 1 T n X = (I - 1 n 1 n 1 T n) X

$X_c = X -1_n \bar X^T = X - \frac{1}{n} 1_n 1_n^T X= \left(I - \frac{1}{n}1_n 1_n^T\right ) X$

The matrix

C = (I - 1 n 1 p 1 T p)

$C = \left(I - \frac{1}{n}1_p 1_p^T\right )$
is called centering matrix.

2. Application

Note that this is a more complex decomposition by centering matrix

Proof of Proposition 1 in
http://blog.youkuaiyun.com/comeyan/article/details/50514596

proof: Firstly we express mean of full data set by group means,

μ ¯ = \sum g = 1 N n g N μ ¯ g = 1 N (n 1 - - \sqrt μ ¯ 1, n 2 - - \sqrt μ ¯ 2, \dots, n g - - \sqrt μ ¯ G) (n 1 - - \sqrt, n 2 - - \sqrt, \dots, n G - - - \sqrt) T

$\begin{align} \bar \mu &= \sum_{g=1}^N \frac{n_g}{N} \bar\mu_g \\ &= \frac{1}{N}(\sqrt{n_1}\bar\mu_1, \sqrt{n_2}\bar\mu_2,\cdots, \sqrt{n_g} \bar\mu_G)\left(\sqrt{n_1},\sqrt{n_2}, \cdots,\sqrt{n_G}\right)^T \end{align}$

Let $K =\left(\sqrt{n_1},\sqrt{n_2}, \cdots,\sqrt{n_G}\right)^T$ , then using the formula of between covariance matrix, we have

(n 1 - - \sqrt μ ¯ 1 - n 1 - - \sqrt μ ¯, n 2 - - \sqrt μ ¯ 2 - n 2 - - \sqrt μ ¯, \dots, n G - - - \sqrt μ ¯ G - n G - - - \sqrt μ ¯) = (n 1 - - \sqrt μ ¯ 1, n 2 - - \sqrt μ ¯ 2, \dots, n G - - - \sqrt μ ¯ G) - μ ¯ (n 1 - - \sqrt, n 2 - - \sqrt, \dots, n G - - - \sqrt) = (n 1 - - \sqrt μ ¯ 1, n 2 - - \sqrt μ ¯ 2, \dots, n G - - - \sqrt μ ¯ G) (I - 1 N K K T)

$\begin{align} &\left(\sqrt{n_1}\bar\mu_1 - \sqrt{n_1}\bar\mu, \sqrt{n_2}\bar\mu_2 - \sqrt{n_2}\bar\mu ,\cdots, \sqrt{n_G}\bar\mu_G - \sqrt{n_G}\bar\mu\right)\\ &=\left(\sqrt{n_1}\bar\mu_1, \sqrt{n_2}\bar\mu_2 ,\cdots, \sqrt{n_G}\bar\mu_G\right) - \bar\mu(\sqrt{n_1}, \sqrt{n_2},\cdots,\sqrt{n_G})\\ & = \left(\sqrt{n_1}\bar\mu_1, \sqrt{n_2}\bar\mu_2 ,\cdots, \sqrt{n_G}\bar\mu_G\right) \left(I - \frac{1}{N}KK^T\right) \end{align}$

n g - - \sqrt μ ¯ g = n g - - \sqrt 1 n g X T 1 g = 1 n g - - \sqrt X T 1 g

$\begin{align} \sqrt{n_g}\bar \mu_g = \sqrt{n_g} \frac{1}{n_g}X^T 1_{g} = \frac{1}{\sqrt{n_g}}X^T 1_{g} \end{align}$

Σ^b = 1 N \sum g = 1 G n g (μ ¯ g - μ ¯) (μ ¯ g - μ ¯) T = 1 N \sum g = 1 G n g - - \sqrt (μ ¯ g - μ ¯) n g - - \sqrt (μ ¯ g - μ ¯) T = 1 N (n 1 - - \sqrt μ ¯ 1, n 2 - - \sqrt μ ¯ 2, \dots, n G - - - \sqrt μ ¯ G) (I - 1 N K K T) (n 1 - - \sqrt μ ¯ 1, n 2 - - \sqrt μ ¯ 2, \dots, n G - - - \sqrt μ ¯ G) T = 1 N X T (1 n g - - \sqrt 1 g) N \times G (I - 1 N K K T) (1 n g - - \sqrt 1 g) T N \times G X = 1 N X T (1 n g - - \sqrt 1 g) N \times G C (1 n g - - \sqrt 1 g) T N \times G X

$\begin{align} \hat\Sigma_b &= \frac{1}{N}\sum_{g=1}^G n_g(\bar\mu_g - \bar \mu)(\bar\mu_g - \bar \mu)^T\\ &= \frac{1}{N}\sum_{g=1}^G \sqrt{n_g}(\bar\mu_g - \bar \mu)\sqrt{n_g}(\bar\mu_g - \bar \mu)^T\\ &= \frac{1}{N} \left(\sqrt{n_1}\bar\mu_1, \sqrt{n_2}\bar\mu_2 ,\cdots, \sqrt{n_G}\bar\mu_G\right) \left(I - \frac{1}{N}KK^T\right) \left(\sqrt{n_1}\bar\mu_1, \sqrt{n_2}\bar\mu_2 ,\cdots, \sqrt{n_G}\bar\mu_G\right)^T\\ &= \frac{1}{N} X^T\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} \left(I - \frac{1}{N}KK^T\right) \left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} ^T X\\ &=\frac{1}{N} X^T\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} C\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} ^T X \end{align}$

Claim that $C = \tilde H^T\tilde H$ , where $\tilde H\in \mathbb{R}^{G-1\times G}$ . That is to say

C = (I - 1 N K K T) = H ~ H ~ T

$C=\left(I - \frac{1}{N}KK^T\right) = \tilde H\tilde H^T$
so

(K,H~T) $\left(K, \tilde H^T\right)$ is an orthogonal matrix. From the theory of orthogonal contrasts for unbalanced data , we have the

G−1 $G-1$ orthogonal contrasts have the following form:

δ r = n r + 1 - - - - \sqrt (\sum h = 1 r n h (μ ¯ h - μ ¯ r + 1))

$\delta_r = \sqrt{n_{r+1}}\left(\sum_{h=1}^rn_h\left(\bar\mu_h - \bar\mu_{r+1}\right)\right)$

Denoted by $h_r$ the $r$ th row of $\tilde H$ . Then from the definition of orthogonal contrasts, for some constant $C_r$ ,

X T (1 n g - - \sqrt) h T r = C r δ r

$X^T \left(\frac{1}{\sqrt{n_g}}\right) h_r^T= C_r \delta_r$

which can be rewritten as

\sum j = 1 G h r h n j - - \sqrt μ ¯ j = C r n r + 1 - - - - \sqrt ⎛ ⎝ \sum j = 1 r n j (μ ¯ j - μ ¯ r + 1) ⎞ ⎠

$\sum_{j=1}^G h_{rh}\sqrt{n_j}\bar\mu_j = C_r \sqrt{n_{r+1}}\left(\sum_{j=1}^rn_j\left(\bar\mu_j - \bar\mu_{r+1}\right)\right)$

Then

h r j n j - - \sqrt h r r + 1 n r + 1 - - - - \sqrt h r i = C r n r + 1 - - - - \sqrt n j = - C r n r + 1 - - - - \sqrt \sum t = 1 r n t = 0 for j = 1, 2, \dots, r

$\begin{align} h_{rj}\sqrt{n_j}&= C_r \sqrt{n_{r+1}}n_j &\text{for} j=1,2,\cdots,r\\ h_{rr+1}\sqrt{n_{r+1}}&=- C_r\sqrt{n_{r+1}}\sum_{t=1}^rn_t \\ h_{ri}&=0&\\ \end{align}$

which gives

h r j h r r + 1 h r i = C r n r + 1 n j - - - - - - \sqrt = - C r \sum t = 1 r n t = 0 for j = 1, 2, \dots, r

$\begin{align} h_{rj}&= C_r \sqrt{n_{r+1}n_j} &\text{for} j=1,2,\cdots,r\\ h_{rr+1}&=- C_r\sum_{t=1}^rn_t &\\ h_{ri}&=0&\\ \end{align}$

To making $\|h_r\|_2=1$ , set

C 2 r ⎛ ⎝ \sum i = 1 r n r + 1 n i + ⎛ ⎝ \sum j = 1 r n j ⎞ ⎠ 2 ⎞ ⎠ = 1

$C_r^2\left(\sum_{i=1}^r n_{r+1}n_i + \left(\sum_{j=1}^rn_j\right)^2\right)=1$

C r = 1 \sum r i = 1 n i \sum r + 1 j = 1 n j

$C_r = \frac{1}{\sum_{i=1}^rn_i \sum_{j=1}^{r+1}n_j}$

Now

X T (1 n g - - \sqrt 1 g) h r = n r + 1 - - - - \sqrt \sum r i = 1 n i \sum r + 1 j = 1 n j (\sum h = 1 r n h (μ ¯ h - μ ¯ r + 1))

$X^T\left(\frac{1}{\sqrt{n_g}}1_g\right)h_r = \frac{\sqrt{n_{r+1}}}{\sum_{i=1}^rn_i \sum_{j=1}^{r+1}n_j}\left(\sum_{h=1}^rn_h\left(\bar\mu_h - \bar\mu_{r+1}\right)\right)$

Σ^b = 1 N X T (1 n g - - \sqrt 1 g) N \times G C (1 n g - - \sqrt 1 g) T N \times G X = 1 N X T (1 n g - - \sqrt 1 g) N \times G H ~ T H ~ (1 n g - - \sqrt 1 g) T N \times G X = Δ Δ T

$\begin{align} \hat\Sigma_b &=\frac{1}{N} X^T\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} C\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} ^T X\\ &=\frac{1}{N} X^T\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} \tilde H^T \tilde H\left(\frac{1}{\sqrt{n_g}}1_g\right)_{N\times G} ^T X\\ &=\Delta\Delta^T \end{align}$
where