Tensor Operator
Tensor times matrix: the k-mode product
The
k
k
k-mode product of a tensor
X
∈
R
I
1
×
I
2
×
…
×
I
N
\boldsymbol{X} \in \mathbb{R}^{\boldsymbol{I}_{1} \times \boldsymbol{I}_{2} \times \ldots \times \boldsymbol{I}_{N}}
X∈RI1×I2×…×IN with a matrix
A
∈
R
J
×
I
k
A \in \mathbb{R}^{J \times I_{k}}
A∈RJ×Ik is written as
Y
=
X
×
k
A
Y=X \times_{k} A
Y=X×kA
The resulting tensor
Y
Y
Y is of size
I
1
×
…
×
I
k
−
1
×
J
×
I
k
+
1
×
…
×
I
N
I_{1} \times \ldots \times I_{k-1} \times J \times I_{k+1} \times \ldots \times I_{N}
I1×…×Ik−1×J×Ik+1×…×IN, and contains the elements
y
i
1
⋯
i
k
−
1
j
i
k
+
1
⋯
i
N
=
∑
i
k
=
1
I
k
x
i
1
i
2
⋯
i
N
a
j
i
k
y_{i_{1} \cdots i_{k-1} j i_{k+1} \cdots i_{N}}=\sum_{i_{k}=1}^{I_{k}} x_{i_{1} i_{2} \cdots i_{N}} a_{j i_{k}}
yi1⋯ik−1jik+1⋯iN=ik=1∑Ikxi1i2⋯iNajik
A few important facts about the k-mode product.
–
X
×
m
A
×
n
B
=
X
×
n
B
×
m
A
X \times_m A \times_n B = X \times_n B \times_m A
X×mA×nB=X×nB×mA if
n
≠
m
n \neq m
n=m
– but
X
×
n
A
×
n
B
=
X
×
n
(
B
A
)
X \times_n A \times_n B = X \times_n (BA)
X×nA×nB=X×n(BA) (in general
≠
X
×
n
B
×
n
A
\neq X \times_n B \times_n A
=X×nB×nA)
Tucker Composition
For a 3rd-order tensor
T
∈
F
d
1
×
d
2
×
d
3
T \in F^{d_{1} \times d_{2} \times d_{3}}
T∈Fd1×d2×d3, where
F
F
F is either
R
\mathbb{R}
R or
C
\mathbb{C}
C, ‘’‘Tucker Decomposition’’’ can be denoted as follows,
T
=
T
×
1
U
(
1
)
×
2
U
(
2
)
×
3
U
(
3
)
T = \mathcal{T} \times_{1} U^{(1)} \times_{2} U^{(2)} \times_{3} U^{(3)}
T=T×1U(1)×2U(2)×3U(3)
where T ∈ F d 1 × d 2 × d 3 \mathcal{T} \in F^{d_{1} \times d_{2} \times d_{3}} T∈Fd1×d2×d3 is the ‘‘core tensor’’, a 3rd-order tensor that contains the 1-mode, 2-mode and 3-mode singular values of T T T, which are defined as the ''Frobenius norm" of the 1-mode, 2-mode and 3-mode slices of tensor T \mathcal{T} T respectively. U ( 1 ) , U ( 2 ) , U ( 3 ) U^{(1)}, U^{(2)}, U^{(3)} U(1),U(2),U(3) are unitary matrices in F d 1 × d 1 , F d 2 × d 2 , F d 3 × d 3 F^{d_{1} \times d_{1}}, F^{d_{2} \times d_{2}}, F^{d_{3} \times d_{3}} Fd1×d1,Fd2×d2,Fd3×d3 respectively. Note that T \mathcal{T} T might be much smaller than the original tensor T T T if we accept an approximation instead of an exact equality. The CP decomposition can be seen as a special case of the Tucker decomposition, where the core tensor T \mathcal{T} T is constrained to be superdiagonal.
The ‘‘j’’-mode product (’‘j’’ = 1, 2, 3) of
T
\mathcal{T}
T by
U
(
j
)
U^{(j)}
U(j) is denoted as
T
×
U
(
j
)
\mathcal{T} \times U^{(j)}
T×U(j) with entries as
(
T
×
1
U
(
1
)
)
(
d
1
,
d
2
,
d
3
)
=
∑
i
1
=
1
d
1
T
(
i
1
,
d
2
,
d
3
)
U
(
1
)
(
d
1
,
i
1
)
(
T
×
2
U
(
2
)
)
(
d
1
,
d
2
,
d
3
)
=
∑
i
2
=
1
d
2
T
(
d
1
,
i
2
,
d
3
)
U
(
2
)
(
d
2
,
i
2
)
(
T
×
3
U
(
3
)
)
(
d
1
,
d
2
,
d
3
)
=
∑
i
3
=
1
d
3
T
(
d
1
,
d
2
,
i
3
)
U
(
3
)
(
d
3
,
i
3
)
(\mathcal{T} \times_{1} U^{(1)})(d_{1}, d_{2}, d_{3}) = \sum_{i_{1}=1}^{d_{1}} \mathcal{T}(i_{1}, d_{2}, d_{3})U^{(1)}(d_{1}, i_{1})\\ (\mathcal{T} \times_{2} U^{(2)})(d_{1}, d_{2}, d_{3}) = \sum_{i_{2}=1}^{d_{2}} \mathcal{T}(d_{1}, i_{2}, d_{3})U^{(2)}(d_{2}, i_{2}) \\ (\mathcal{T} \times_{3} U^{(3)})(d_{1}, d_{2}, d_{3}) = \sum_{i_{3}=1}^{d_{3}} \mathcal{T}(d_{1}, d_{2}, i_{3})U^{(3)}(d_{3}, i_{3})
(T×1U(1))(d1,d2,d3)=i1=1∑d1T(i1,d2,d3)U(1)(d1,i1)(T×2U(2))(d1,d2,d3)=i2=1∑d2T(d1,i2,d3)U(2)(d2,i2)(T×3U(3))(d1,d2,d3)=i3=1∑d3T(d1,d2,i3)U(3)(d3,i3)