Resnet网络残差连接与relu顺序

最新推荐文章于 2024-11-01 13:16:32 发布

xiaohanabc

最新推荐文章于 2024-11-01 13:16:32 发布

阅读量3.5k

点赞数 3

CC 4.0 BY-SA版权

分类专栏：深度学习

本文链接：https://blog.youkuaiyun.com/u012189988/article/details/99711324

在Resnet的Bottleneck模块中，必须先进行残差连接，然后应用ReLU激活函数。若顺序调换，则会导致模型无法收敛，准确率停留在50%-70%区间。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

class Bottleneck(nn.Module):
expansion = 4

def __init__(self, inplanes, planes, stride=1, downsample=None):
    super(Bottleneck, self).__init__()
    self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(planes)
    self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
                           padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(planes)
    self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * 4)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = downsample
    self.stride = stride

def forward(self, x):
residual = x

    out = self.conv1(x)
    out = self.bn1(out)
    out = self.relu(out)

    out = self.conv2(out)
    out = self.bn2(out)
    out = self.relu(out)

    out = self.conv3(out)
    out = self.bn3(out)

    if self.downsample is not None:
        residual = self.downsample(x)