论文精读《Multi-scale Residual Network for Image Super-Resolution》

 

目前更多致力于用更深的网络去提升性能，然而盲目追求网络深度，并不能有效改善网络，反而，随着深度加深网络更难训练而且需要很多trick、

多尺度残差网络，从网络广度上进一步探索图像特征，多size卷积核自适应检测不同尺度特征，同时，多尺度直接有一定的交互和融合，得到更有效的多层次信息，结构—MSRB

模块输出作为各尺度的层次信息，首先为一阶注意力模块，将各个尺度的提取特征进行重映射，之后进行多尺度特征通道的concat操作，通过二次注意力网络自适应选择各个尺度通道最有用的信息并进行有效结合，通过全局融合进行高质量图像重建。

Current researches tend to use deeper convolutional neural networks to enhance performance. However, blindly increasing the depth of the network cannot ameliorate the network effectively. Worse still, with the depth of the network increases, more problems occurred in the training process and more training tricks are needed.

Nevertheless, all of these models tend to construct deeper and more complex network structures, which means training these models consumes more resources, time, and tricks.

SR—SISR—SRCNN--learn the mapping between LR and HR images

CNNs-based SISR models SRResNet–EDSR—

经典模型存在的问题：
(a) Hard to Reproduce:难以复现，大多数SR模型对细微的网络体系结构更改都很敏感，同一模型通过使用不同的训练技巧（例如权重初始化，梯度截断，数据归一化等）来实现不同的性能。 这意味着性能的提高可能不是由于模型体系结构的变化，而是由于使用了一些未知的训练技巧

(b) Inadequate of Features Utilization: 特征利用不足，盲目增加网络的深度，忽略充分利用LR图像信息，网络深度增加，特征在传输过程中逐渐消失

but ignore taking full use of the LR image features.

the features gradually disappear in the process of transmission.

(c) Poor Scalability:预处理的LR图像用作输入会增加计算复杂性并产生可见的伪像，更加关注直接放大LR图像。很难找到模型可以适应任何upscale因素

can accommodate to any upscaling factors, or can migrate to any upscaling factors with only minor adjustments to the network architecture.

Firstly，…,which is considered as ….Sencondly,…are combined for…, Finally, the combination of local multi-scale features and global features can maximize the use of the LR image features and completely solve the problem that features disappear in the transmission process. Besides,

can not only adaptively detect the image features, but also achieve feature fusion at different scales.

a simple architecture for hierarchical features fusion (HFFS) and

image reconstruction.

研究现状：

三阶段：
① 插值方法 linear/bicubic—not rebuild the detailed realistic textures

② 邻近embed/稀疏编码：establish complex mapping

③ end-to-end CNNs：LR-HR映射，

SRCNN-bicubic预上采样-计算复杂、产生伪影

FSRCNN/ESPCNN: Sub-pixel模块，后上采样—网络浅层（<5）

DCRN/DRNN/LapSRN/SRResNet/EDSR:残差结构—网络变深/训练难度+++

特征提取：

• the inception block：卷积网络中的稀疏结构—不同尺度简单concat--underutilization
• Residual block ：ease the training of network
• Dense block：computational complexity

Methods：

• detecting the features of images at different scales
• a skip connection different scale feature（可否增加一阶注意力进行重映射）
• a 1×1 convolution layer at the end of be used as a bottleneck layer--feature fusion（可否增加二阶注意力促进多尺度特征通道的重映射）

 

Proposed Method

We convert the image to the YCbCr color space and train only on the Y channel.

structures: multi-scale residualblock (MSRB) and hierarchical feature fusion structure (HFFS).

 

输入输出相关性较强：防止信息丢失—the skip connection

All of these methods try to create different connections between different layers.

Unfortunately, these methods can’t fully utilize the features of the input image, and generate too much redundant information for aimlessness.

全部传输-信息冗余/计算复杂-- a bottleneck layer with 1×1 kernel.（不建议，考虑注意力或者加权concat？参考FC2N）

 

新的重构模块：PixelShuffle

 

 

论文2：Single MR Image Super-Resolution via Channel Splitting and Serial Fusion Network

 

引入：HR 耗费更高扫描时间/硬件设备带来的运动伪影motion artifacts /hardware, physical, and physiological limitations

越deep越难trained—信息随着深度逐渐weaked，医学图像由于训练样本的退化更加严重

重点：Splits the hierarchical features into a series of subfeatures, which are then integrated together in a serial manner.

可更深/有区分地处理不同通道的子特征，DGFF 全局is adopted to integrate the intermediate features, which further promotes the information flow in the network.

 

介绍：

MRI的重要性/普及程度—高质量的MRI的需求

句型：High resolution (HR) magnetic resonance (MR) images are usually preferred in clinical practice due to more clear image structure and texture details, as well as the benefits to subsequent analysis and processing

 

逐次引入一些限制：受硬件、物理、摄像限制，提高R降低SNR+增加成像时间+同时引入伪影motion artifacts

引出SR的重要—一些方法的提出：自然/医学—传统方法：插值/边缘/建模/EL/DL/SPR

不充分信息/表达能力限制

句型：However, the performance of these conventional methods is essentially limited because

they apply inadequate additional information and models with limited representational capacity to solve the notoriously challenging ill-posed inverse problem of image SR tasks

引入DP—SISR 优越性 great superiority over conventional

A pioneering work-SRCNN end-to-end à Subsequently typical examples are DRCN [23], DRRN [24], VDSR [25], MemNet [26], ESPCNN [27], SRResNet [28], EDSR/MDSR [29], RDN

[30], CMSCN [31] and RCAN [32] etc –比传统优越overwhelming advantages over traditional

 

引出医学图像：However, they are mainly aimed at the SISR task of natural images, instead of medical images (or more specifically, MR images).缺点：对退化样本不稳定，尽管对自然有效

 

相关工作：

1. MR Image Super-Resolution

摆脱硬件设置/满足临床需求，专注于皮层表面或精细尺度结构分析时， SR会对结构MRI产生重大影响 focusing on cortical surface or fine-scale structure analysis

开始用MISR,但多LR需要配准和融合—挑战大，[14], [15], [39].而SISR只用LR-HR，避免上述问题  HR counterpart（配对），SISR主要困难在于重建时额外信息的有限性

引出传统机器学习方法，但表达能力受限，高维非线性映射难，引出DL在MR的应用 which have greatly promoted the performance of SR technologies for medical images or, more specifically, MR images.

连接词：Subsequently -However - Recently, more advanced

1. Channel Discrimination

不同通道有不同类型的信息同时对模型表现有不同影响，因此区别性处理不同信息是合理的reasonable。典型方法：attention机制，资源重分配找到most informative componets

然后介绍attention的发展：连接词=In recent years, it has been introduced …to boost the performance of…,such as… 进一步提高了…,举例：For instance, the…  --高光句式

但是However，很少有工作关注医学图像中的低级视觉任务中的通道区分性问题，

在这个方面，代表性工作是CSN…(简单介绍优缺点，解决不同通道层次特征但是限制网络深度)，因此我们结合了子空间特征到单独分支in a serial manner—变深变窄

结尾升华，虽然…但是句型，概括特点

1. Hierarchical Feature Fusion（同理借鉴空间/广度 信息融合）

简述问题—梯度消失信息损失，which hinders the…

将问题升华到医学图像：Unfortunately, the degradation of training samples will further aggravate the difficulty of training deep models in the context of medical images [18].

促进信息融合/提高稳定性---many recent works have been devoted to…

A popular method is to…简述现有方法，列出名字

结尾段升华：简述该种结构的优势…引出自己方法的改进和优势

高光句式：In the proposed , the… are fused together simply by …, which is also known as … or …However, the …can be viewed as a manner of …Besides, we also use … for effective feature exploitation and important information preservation. More importantly, it helps to  relieve …(the instability of model training)