Feb.27~image super-resolution reconstruction， paper reading

Multi-frame image super-resolution reconstruction via low-rank fusion combined with sparse coding

这篇文章看起来和2月初读的一篇关于图像去噪是自相矛盾的，这篇文章可以方法成立的条件是，HR和LR图片用字典表示时候的协方差系数矩阵是类似的，也就是那个 $\alpha$ 在两种情况下是一样的。在求解中也应用到了从三个方向特征提取的技巧，还有K-SVD方法。

（思考：那么在这里的特征提取是否可以接入神经网络，模型自己训练，自己提取特征呢？）这里面提取特征是在第二部，是为了提取出特征之后这些特征向量组成训练集，再用K-SVD方法提取出字典。

之前的文章中，两种情况下的$\alpha$ 是有偏差的，之后在模型优化中，专门加入了一项，表示两个协方差的距离，接着进行一系列的估计，迭代，然后求解。

According to the number of input LR images, image SR can be divided into single -frame and multi - frame.

Single-frame SR refers to restore a HR image from a LR image, and multi-frame SR means to recover a HR image from LR image sequence.

2. Low rank fusion and sparse coding

2.1 Low rank fusion

A data matrix $M\in {\mathbb{R}}^{n\times m}$ contains structural information as well as noise. Then we can decompose M as
M = L + S
where L is low-rank matrix(contains the internal structure information which are linearly related.)
S is sparse (noise is sparse)
RPCA(robust principle component analysis) method model used to optimize above problem:
$$\underset{L,S}{\min }rank(L)+\lambda ||S|{|}_0,s.t.M=L+S(1)$$
$\lambda$ is a balance parameter.
(1) is a non-convex problem and it can be replaced by (2)
$$\underset{L,S}{\min }||L|{|}_{\ast }+\lambda ||S|{|}_1,s.t.M=L+S(2)$$
Augmented Lagrange Multiplier (ALM) algorithm is usually used to solve (2)

Supposing there are N frame registered images, then they are converted into a column vector and stored in a matrix by column. Now the matrix M={$M_1,M_2,\cdots M_N$} has low-rank characteristics.
Next we decompose the matrix by low-rank and sparse decomposition.
Finally the decomposed low-rank part L = {$L_1,L_2,\cdots L_N$} is average fused to get the final fusion LR image L∗ =($L_1+L_2+\cdots +L_N$)/N,

2.2 Sparse coding

After down sampling S and fuzzy B, HR image X is degenerated into LR image Y:
$$Y=SBX(3)$$
Where SB = L, Y = LX.
If y is an image patch taken from Y ,x is an image patch taken from X which is in the same location with y.
The sparse representation model can be represented as following:
$$min||\alpha |{|}_1,s.t.x=D_h\alpha (4)$$
$D_h\in {\mathbb{R}}^{n\times K}$(K>n) is a HR over-complete dictionary.
According to (3)(4), so y can be expressed as following:
$$y=L{D}_h\alpha =D_l\alpha (5)$$
Where $D_l=L{D}_h,D_l\in {\mathbb{R}}^{m\times K}$(K>m) is a LR over-complete dictionary.
So it can be clarified that HR and LR image patches have the same sparse representation coefficient. (${\alpha }_x={\alpha }_y$)
Based on a pair of HR and LR dictionaries {$D_h,D_l$}, we are able to rebuild the correspond- ing HR image patch as long as we acquire sparse representation coefficient of the LR image patch.

3. Multi-frame image SR via low-rank fusion combines with sparse coding

3.1 method

The Multi-frame SR using low-rank fusion combines with sparse coding includes three steps: image registration and low-rank fusion, {$D_h,D_l$} dictionary training and SR reconstruction, as shown in Fig. 3.
In image registration and low-rank fusion phase, SURF and RANSAC algorithm are used to image registration. Then registered images are decomposed into the low-rank images and the sparse images, and the low-rank images are fused into a LR image.
In {$D_h,D_l$} dictionary training phase, patch haar wavelet transform is used to get image patches characteristic vectors, and characteristic vectors constitute the joint training set. K-SVD algorithm is applied to train joint training set to obtain the {$D_h,D_l$}.
In SR reconstruction phase, after computing sparse representation coefficient $\alpha$ of LR patch, the HR patch can be obtained from the coefficient $\alpha$ multiplied by the HR dictionary $D_h$.

The END of Method of this paper.