用pytorch搭建一个超级简单的神经网络

• 最近在自学pytorch一些内容，虽然是找着手册来的，但有时候往往一行代码也要我理解很久。

  学习了一些简单的原理和pytorch变成方法后，我试着自己搭建了一个卷积神经网络。

  流程大致是用一个33的卷积核对一个55的张量进行卷积，通过迭代使其结果变为一个3*3的单位张量。

所用环境为Ubuntu18.04 pytorch1.10 cuda10.2 python3.6

import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt

class Net(nn.Module):

    def __init__(self):
        super().__init__()                              #inheritance the classes from nn.Module
        self.conv1 = nn.Conv2d(1, 1, 3)                 #define a kernal（input1，output1，windowsize（3*3））

    def forward(self,x):
        x = self.conv1(x)                               #define forward function,only one layer and no activation function
        return x

net = Net()

net = net.cuda()                                        #use GPU accelerate,need to put the module and tensors on GPU by cuda()

# print(list(net.parameters()))

input = torch.randn(1, 1, 5, 5, requires_grad=True)     #generate input by torch.randn
input = input.cuda()


para = list(net.parameters())                           #net.parameters() have 2 dimensions,one is weights,one is bias
# print(para[0])                                        #can also use net.kernal.bias or net.kernal.weight to show

# print(net.conv1.bias.grad)                            #use net.parameters()[].grad or net.kernal.bias(weight).grad to show the grad



target = torch.ones(1, 1, 3, 3).cuda()                  #generate the target data

learning_rate = 0.00000000001                           #define learning rate

iterations = []                                         #use lists to record iteration times corresponding lossnum
lossnum = []


for i in range(200001):
    loss = nn.MSELoss()                                 #the mean squared error,only single number
    output = net(input).cuda()
    loss = loss(output, target).cuda()
    iterations.append(i)                                #add num in the list
    lossnum.append(loss.cpu().detach().numpy())
    if i%1000 == 0:
        print("iteration times:",i)
        print(loss)
    loss.backward()                                     #do back propagation

    # print(para)
    # print(net.conv1.weight.grad)
    for f in net.parameters():
        f.data.sub_(f.grad.data * learning_rate)        #update the weights and bias,use f in net.parameters() to traverse all parameters
                                                        #use data.sub_() to subtract grad*learning_rate to update

output = net(input).cuda()
print(output)
print(type(lossnum[0]))

plt.xlim()
plt.ylim()
plt.plot(iterations,lossnum,'ro')
plt.show()                                              #plot the curve to show the relationship between iteration times and loss
# print(para)

# output2 = net(input)
# loss2 = nn.MSELoss()
# loss2 = loss2(output2, target)
# print(loss2)
# print(net.conv1.weight.grad)

因为ubuntu这个搜狗输入法在pycharm里一直贴在左下角，很难用，就用英文写了注释，包含了我对一些函数的理解和用法，如有不对也请指正。

一开始学习率设在了0.01左右，发现效果奇差，后来改成了特别小，效果才好起来，可能是本身数据也小吧。

最后自己跑出来的迭代次数和损失值的关系，感觉这样应该算是收敛了吧。