MLP中实现dropout，批标准化-优快云博客

本文介绍了在多层感知机（MLP）中如何实现dropout和批标准化。通过在基本的三层MLP上应用批标准化，网络的准确率得到显著提升。接着，讨论了dropout作为防止过拟合的手段，指出在训练期间dropout层会随机关闭神经元，但在预测时不发挥作用。文章最后提出了疑问，探讨了批标准化和dropout层在模型结构中顺序安排的可能性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

MLP中实现dropout，批标准化

基本网络代码

三层MLP
使用MNIST数据集

import torch as pt
import torchvision as ptv
import numpy as np

train_set = ptv.datasets.MNIST("../../pytorch_database/mnist/train",train=True,transform=ptv.transforms.ToTensor(),download=True)
test_set = ptv.datasets.MNIST("../../pytorch_database/mnist/test",train=False,transform=ptv.transforms.ToTensor(),download=True)
train_dataset = pt.utils.data.DataLoader(train_set,batch_size=100)
test_dataset = pt.utils.data.DataLoader(test_set,batch_size=100)

class MLP(pt.nn.Module):
    def __init__(self):
        super(MLP,self).__init__()
        self.fc1 = pt.nn.Linear(784,512)
        self.fc2 = pt.nn.Linear(512,128)
        self.fc3 = pt.nn.Linear(128,10)
        
    def forward(self,din):
        din = din.view(-1,28*28)
        dout = pt.nn.functional.relu(self.fc1(din))
        dout = pt.nn.functional.relu(self.fc2(dout))
        return pt.nn.functional.softmax(self.fc3(dout))
model = MLP().cuda()
print(model)

# loss func and optim
optimizer = pt.optim.SGD(model.parameters(),lr=0.01,momentum=0.9)
lossfunc = pt.nn.CrossEntropyLoss().cuda()

# accuarcy
def AccuarcyCompute(pred,label):
    pred = pred.cpu().data.numpy()
    label = label.cpu().data.numpy()
    test_np = (np.argmax(pred,1) == label)
    test_np = np.float32(test_np)
    return