1.6学习日记-优快云博客

一、一般将softmax()的输出视作模型预测的概率分布

$y_{i}'=e^{y_{i}}\div \sum_{j}^{} e^{y_{j}}$

二、分类任务的Loss：交叉熵损失

需要的参数：预测的分布、真实值的下标

三、经典神经网络AlexNet

一）创新点：

1.rule：计算方便、可以在一定程度上缓解梯度消失

2.Dropout：缓解过拟合

3.池化

4.归一化：避免受数据量纲影响，让模型关注数据的分布，保持学习有效性，缓解梯度消失和梯度爆炸

二）实战演练

1.Conv2d就是卷积，其参数依次为：输入特征图的数量、输出特征图数量（即卷积核数量）、卷积核大小、步长、padding

2.pool（3，2）表示3格一pool，每次走2格

from torchvision import models
import torch
import torch.nn as nn


# model = models.alexnet()
# print(model)

class myModel(nn.Module):
    def __init__(self, num_cls):
        super(myModel, self).__init__()
        self.conv1 = nn.Conv2d(3, 64, 11, 4, 2)
        self.pool1 = nn.MaxPool2d(3, 2)

        self.conv2 = nn.Conv2d(64, 192, 5, 1, 2)
        self.pool2 = nn.MaxPool2d(3, 2)

        self.conv3 = nn.Conv2d(192, 384, 5, 1, 2)
        self.conv4 = nn.Conv2d(384, 256, 5, 1, 2)
        self.conv5 = nn.Conv2d(256, 256, 5, 1, 2)

        self.pool3 = nn.MaxPool2d(3, 2)
        self.pool4 = nn.AdaptiveAvgPool2d(6)

        self.fc1 = nn.Linear(9216, 4096)  # 全连接
        self.fc2 = nn.Linear(4096, 4096)
        self.fc3 = nn.Linear(4096, num_cls)

    def forward(self, x):
        x = self.conv1(x)
        x = self.pool1(x)
        x = self.conv2(x)
        x = self.pool2(x)
        x = self.conv3(x)
        x = self.conv4(x)
        x = self.conv5(x)
        x = self.pool3(x)
        x = self.pool4(x)  # x : batch*256*6*6
        x = x.view(x.size()[0], -1)  # 用于展平 16*9216
        x = self.fc1(x)
        x = self.fc2(x)
        x = self.fc3(x)

        return x


model = myModel(1000)
data = torch.ones((4, 3, 224, 224))
pred = model(data)


def get_parameter_number(model):  #计算模型参数量
    total_num = sum(p.numel() for p in model.parameters())
    trainable_num = sum(p.numel() for p in model.parameters() if p.requires_grad)
    return {'Total': total_num, 'Trainable': trainable_num}

print(get_parameter_number(model.conv1))

四、VggNet

创新点：

1.更深、更大

2.用小卷积核代替大的卷积核

五、ResNet

创新点：

1.1×1卷积（可降维）

2.残差连接（梯度消失） out = f(x) + x

Q：若输出和输入层数大小维度不同怎么办？

A：利用步长和卷积核数量调整