Pytorch与深度学习自查手册3-模型定义

Pytorch与深度学习自查手册3-模型定义

定义神经网络

1. 继承nn.Module类；
2. 初始化函数__init__：网络层设计；
3. forward函数：模型运行逻辑。

class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.flatten = nn.Flatten()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(28*28, 512),
            nn.ReLU(),
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 10),
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_relu_stack(x)
        return logits

权重初始化

pytorch中的权值初始化

方法1：net.apply(weights_init)

def weights_init(m):
    classname = m.__class__.__name__
    if classname.find('Conv') != -1:
        nn.init.normal_(m.weight.data, 0.0, 0.02)
    elif classname.find('BatchNorm') != -1:
        nn.init.normal_(m.weight.data, 1.0, 0.02)
        nn.init.constant_(m.bias.data, 0)

net=Model()
net.apply(weights_init)

方法2：在网络初始化的时候进行参数初始化

1. 使用net.modules()遍历模型中的网络层的类型；
2. 对其中的m层的weigth.data(tensor)部分进行初始化操作。

class Model(nn.Module):
    def __init__(self, block, layers, num_classes=1000):
        self.inplanes = 64
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
                               bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.relu = nn.ReLU(inplace=True)
        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
        self.layer1 = ...
        ...
        #　权值参数初始化
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
                m.weight.data.normal_(0, math.sqrt(2. / n))
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()

常用的操作

利用nn.Parameter()设计新的层

import torch
from torch import nn

class MyLinear(nn.Module):
  def __init__(self, in_features, out_features):
    super().__init__()
    self.weight = nn.Parameter(torch.randn(in_features, out_features))
    self.bias = nn.Parameter(torch.randn(out_features))

  def forward(self, input):
    return (input @ self.weight) + self.bias

nn.Flatten

展平输入的张量： 28x28 -> 784

input = torch.randn(32, 1, 5, 5)
m = nn.Sequential(
    nn.Conv2d(1, 32, 5, 1, 1),
    nn.Flatten()
)
output = m(input)
output.size()

nn.Sequential

一个有序的容器，神经网络模块将按照在传入构造器的顺序依次被添加到计算图中执行，同时以神经网络模块为元素的有序字典也可以作为传入参数。

net = nn.Sequential(
   ('fc1',MyLinear(4, 3)),
   ('act',nn.ReLU()),
   ('fc2',MyLinear(3, 1))
)

常用的层

全连接层nn.Linear()

torch.nn.Linear(in_features, out_features, bias=True,device=None, dtype=None)

in_features: 输入维度
out_features:输出维度
bias: 是否有偏置

m = nn.Linear(20, 30)
input = torch.randn(128, 20)
output = m(input)
print(output.size())

卷积torch.nn.ConvNd()

class torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

in_channels(int) – 输入信号的通道   
out_channels(int) – 卷积产生的通道
kerner_size(int or tuple) - 卷积核的尺寸
stride(int or tuple, optional) - 卷积步长
padding (int or tuple, optional)- 输入的每一条边补充0的层数   
dilation(int or tuple, optional) – 卷积核元素之间的间距
groups(int, optional) – 从输入通道到输出通道的阻塞连接数
bias(bool, optional) - 如果bias=True，添加偏置

input: (N,C_in,H_in,W_in) N为批次，C_in即为in_channels，即一批内输入二维数据个数，H_in是二维数据行数，W_in是二维数据的列数
output: (N,C_out,H_out,W_out) N为批次，C_out即为out_channels，即一批内输出二维数据个数，H_out是二维数据行数，W_out是二维数据的列数

conv2 = nn.Conv2d( 
            in_channels=5,
            out_channels=32,
            kernel_size=5,
            stride=1,
            padding=2  #padding是需要计算的，padding=（stride-1）/2
)
#只能接受tensor/variable
conv2(torch.Tensor(16, 5, 3, 10))
conv2(Variable(torch.Tensor(16, 5, 3, 10)))

池化

最大池化torch.nn.MaxPoolNd()

torch.nn.MaxPool2d(kernel_size, stride=None, padding=0,