去噪算法: 评价指标、计算参数量以及CBAM回顾

Denoising review:

The trials made can be found on Github.

Assessment indexes:

PSNR: peak signal to noise ratio
$$PSNR=10\ast lo{g}_{10}(\frac{(2^n-1{)}^2}{MSE})=20\ast lo{g}_{10}(\frac{MAX}{\sqrt{MSE}})$$
注意MSE是均方误差(已平均化)，MAX表示图像颜色的最大值，一般8位图表示255

PSNR单位是dB.

SSIM: structural similarity index
$$L(X,Y)=\frac{2u_X{u}_Y+C_1}{u_X^2+u_Y^2+C_1},C(X,Y)=\frac{2{\sigma }_X{\sigma }_Y+C_2}{{\sigma }_X^2+{\sigma }_Y^2+C_2},S(X,Y)=\frac{{\sigma }_{XY}+C_3}{{\sigma }_X{\sigma }_Y+C_3}$$
where $C1=(K1\ast L{)}^2,C2=(K2\ast L{)}^2,C3=C2/2$, and generally $K1=0.01,K2=0.03,l=255$.

$SSIM=L\ast C\ast S$.

Note some basic np operations:

np.contatenate((a,b), dims=1)

np.reshape(a,newshape=(2,2,2,2))

np.split(a,4,0)

np.vstack

np.hstack, etc.

parameters & memory calculation

calculation

Note:

Total GPU memory = memory for model & layer outputs

In training time: forward , backward X2

e.g. VGG16 , memory of params: 528MB, memory of layers: 58.12MB/image

when training:

SGD+momentum, batchsize = 128

memory for model:

1. 528MB*3 = 1.54 GB

   1 for params, 1 for SGD, 1 for momemtum

   If use Adam, need to x4

2. Memory for outputs:

   128*58.12MB *2 = 14.53GB

3. Total

   14.53+1.54 = 16.07GB

FLOPS：
Conv = para*H*W
Linear = para

TFlops/s，可以简单写为T/s， 是数据流量的计数单位，意思是”1万亿次浮点指令每秒”，它是衡量一个电脑计算能力的标准。1TFlops=1024GFlops，即1T=1024G。

a useful tool from torchstat import stat

model

---

1. Residual Dense Network for Image Super-Resolution paper

Here, LR represents Low Resolution Images.

upscale:

sub-pixel convolution

the best position of CBAM is after the up-sampling layer. up 0.0x dB, if patch-size up, perf down.

GRDN:

… yeah, concatenate, concatenate, concatenate.

CBAM: paper

The channel attention is computed as :
$$M_C(F)=\sigma (MLP(AvgPool(F))+MLP(MaxPool(F)))$$
Where $\sigma$ represents activation function, e.g. ReLU.

The spatial attention is computed as:
$$\begin{gathered} M_s(F)=\sigma (f^{7\ast 7}([AvgPool(F);MaxPool(F)])) \\ =\sigma (f^{7\ast 7}([F_{avg}^s;F_{max}^s])) \end{gathered}$$
where, $F_{avg}^s,F_{max}^s\in {\mathbb{R}}^{1\times H\times W}$

class ChannelAttention(nn.Module):
    def __init__(self, in_planes, ratio=16):
        super(ChannelAttention, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.max_pool = nn.AdaptiveMaxPool2d(1)

        self.fc1   = nn.Conv2d(in_planes, in_planes // 16, 1, bias=False)
        self.relu1 = nn.ReLU()
        self.fc2   = nn.Conv2d(in_planes // 16, in_planes, 1, bias=False)

        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avg_out = self.fc2(self.relu1(self.fc1(self.avg_pool(x))))
        max_out = self.fc2(self.relu1(self.fc1(self.max_pool(x))))
        out = avg_out + max_out
        return self.sigmoid(out)

channel attention: $C\times W $ -> pool

class SpatialAttention(nn.Module):
    def __init__(self, kernel_size=7):
        super(SpatialAttention, self).__init__()

        assert kernel_size in (3, 7), 'kernel size must be 3 or 7'
        padding = 3 if kernel_size == 7 else 1

        self.conv1 = nn.Conv2d(2, 1, kernel_size, padding=padding, bias=False)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avg_out = torch.mean(x, dim=1, keepdim=True)
        max_out, _ = torch.max(x, dim=1, keepdim=True)
        x = torch.cat([avg_out, max_out], dim=1)
        x = self.conv1(x)
        return self.sigmoid(x)

For channels, use torch.mean or torch.max to implement the avg or pool for channel info.