Basics of Matrix Theory -矩阵基础/线性代数基础

Basics of Matrix Theory

Noted by Yuanshuai

• Transpose:

$$\begin{gathered} (AB{)}^T=B^T{A}^T \\ (A^T{)}^T=A \\ (A+B{)}^T=A^T+B^T \end{gathered}$$

• Hermitian transpose: nearly the same as transpose.
• Trace:

$$\begin{gathered} tr(A)=\sum _{i=1}^n{a}_{ii} \\ tr(A^T)=tr(A) \\ tr(A+B)=tr(A)+tr(B) \\ tr(AB)=tr(BA)\text{for A,B of appropriate sizes} \end{gathered}$$

• Matrix power:

$$A^k=AAA\cdots A$$

• Unit vectors: a vector that has only one nonzero element and the nonzero element is 1. Notation: $e_i=[0\cdots 010\cdots 0]$

• Identity matrix

$$I=Diag(1,1,1,\cdots ,1)$$

• Subspace:

$$\text{for }x,y\in S,\alpha ,\beta \in \mathcal{R}\Rightarrow \alpha x+\beta y\in S$$

• Span:

span{a1,⋯ ,an}={y∈Rm∣y=∑i=1nαiai,α∈Rn} span\{a_1,\cdots,a_n\} = \{y \in \mathcal{R}^m | y = \sum_{i=1}^{n} \alpha_i a_i, \alpha \in \mathcal{R}^n\} span{a1​,⋯,an​}={y∈Rm∣y=i=1∑n​αi​ai​,α∈Rn}

• Range space of A:

R(A)={y∈Rm∣y=Ax,x∈Rn} R(A) = \{y \in \mathcal{R}^m | y = Ax, x \in \mathcal{R}^n\} R(A)={y∈Rm∣y=Ax,x∈Rn}

• Null space of A:

N(A)={x∈Rn∣Ax=0} N(A) = \{x \in \mathcal{R}^n | Ax = 0\} N(A)={x∈Rn∣Ax=0}

• Linearly independent if

$$\sum _{i=1}^n{\alpha }_i{a}_i\ne 0,\text{for all }\alpha \in {\mathcal{R}}^n\text{ with }\alpha \ne 0$$

• Dimension of a subspace

  • if $S_1\subseteq S_2$, then $\dim S_1\le \dim S_2$
  • if $S_1\subseteq S_2$ and $\dim S_1\le \dim S_2$, then $S_1=S_2$
  • $\dim (S_1+S_2)\le \dim S_1+\dim S_2$
  • $\dim (S_1+S_2)=\dim S_1+\dim S_2-\dim (S_1\cap S_2)$

• Rank

  • $rank(AB)\le rand(A)+rank(B)$

• Invertible

$$\begin{gathered} (AB{)}^{-1}=B^{-1}{A}^{-1} \\ (A^T{)}^{-1}=(A^{-1}{)}^T \end{gathered}$$

• Determinant

$$\begin{gathered} \det (AB)=\det (A)\det (B) \\ \det (\alpha A)={\alpha }^m\det (A) \\ \det (A^{-1})=1/\det (A) \\ \det (B^{-1}AB)=\det (A)\text{ for any nonsingular }B \end{gathered}$$

• Determinant

  • if $A$ is triangular, either upper or lower

  • $$\det (A)=\prod _{i=1}^m{a}_{ii}$$

  • if $A$ takes a block upper triangular form

  • KaTeX parse error: Undefined control sequence: \matrix at position 13: A = \left[ \̲m̲a̲t̲r̲i̲x̲{B & C\\ 0 & D}…

    where $B$ and $D$ are square, then
    $$\det (A)=\det (B)\det (D)$$

• Cauchy-Schwartz inequality

$$|x^{T}y|\le \Vert x{\Vert }_2\Vert y{\Vert }_2$$

• Holder inequality

$$\begin{gathered} |x^{T}y|\le \Vert x{\Vert }_p\Vert y{\Vert }_p \\ \text{for any p,q such that }1/p+1/q=1,p\ge 1 \end{gathered}$$

• A projection of $y$ onto $S$ is any solution to

$$\underset{z\in S}{\min }\Vert z-y{\Vert }_2^2$$

• Orthogonal complement of S is defined as

S⊥={y∈Rm∣zTy=0} for all z∈S S^{\perp} = \{y\in \mathcal R^m | z^T y = 0 \} \text{ for all } z \in S S⊥={y∈Rm∣zTy=0} for all z∈S

• Properties of $S^{\perp }$

  • $$\begin{gathered} R(A{)}^{\perp }=N(A^T) \\ N(A)=R(A^T{)}^{\perp } \\ Projectio{n}_{S^{\perp }}(y)=y-Projectio{n}_S(y) \\ S+S^{\perp }={\mathcal{R}}^m \\ \dim S+\dim S^{\perp }=m \\ (S^{\perp }{)}^{\perp }=S \end{gathered}$$

• Orthogonal if $a_i^T{a}_j=0$ for all $i,j$ with $i\ne j$

• Orthonormal if $\Vert a_i{\Vert }_2=1$ for all $i$ and $a_i^T{a}_j=0$ for all $i,j$ with $i\ne j$

• A real matrix Q is said to be

  • orthogonal: if it is square and its columns are orthonormal

    • $$\begin{gathered} Q^{T}Q=I \\ Q{Q}^T=I \\ \Vert Qx{\Vert }_2=\Vert x{\Vert }_2 \end{gathered}$$

  • semi-orthogonal if its columns are orthonormal

    • $$Q^{T}Q=I$$

• A complex matrix Q is said to be

  • unitary (similar as orthogonal)
  • semi-unitary (similar as semi-orthogonal)