pytorch深度学习实践（刘二大人）课堂代码&作业——反向传播_pytorch 六二大人反向传播作业-优快云博客

本文链接：https://blog.youkuaiyun.com/weixin_51944652/article/details/140386685

一、课堂作业

损失计算：

$\frac{\partial loss}{\partial w}=2r\cdot \frac{\partial r}{\partial w}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial w}=2(wx-y)\cdot x=2(2\cdot 1-4)\cdot 2=-8$

根据公式计算得到loss=1

$\frac{\partial loss}{\partial w}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial w}=2(wx+b-y)\cdot x=2(1\cdot 1+2-2)\cdot 1=2$

$\frac{\partial loss}{\partial b}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial b}=2(wx+b-y)\cdot 1=2(1\cdot 1+2-2)\cdot 1=2$

二、课堂代码

Tensor：pytorch最基本的数据类型（实际是类），可以是标量、向量、矩阵、高维tensor。

tensor包含两个成员：data和 grad

data：保存权重的值

grad：保存损失函数对权重的导数

import numpy as np
import matplotlib.pyplot as plt
import torch

x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]

w = torch.Tensor([1.0])
w.requires_grad = True   #表示需要计算梯度

def forward(x):
    return x * w  #w是tensor，x格式自动转为tensor与w进行点乘，并计算梯度

def loss(x, y):
    y_pred = forward(x)
    return (y_pred - y) ** 2

print('predict (before training)', 4, forward(4).item())

for epoch in range(100):
    for x, y in zip(x_data, y_data):
        l = loss(x, y)
        l.backward()  #计算所有需要计算的梯度（上面创建的w）
        print('\tgrad:', x, y, w.grad.item()) #item用来直接拿出梯度的数值变成标量
        w.data = w.data - 0.01 * w.grad.data  #grad也是一个tensor，直接加进去会建立计算图，需要取其data

        w.grad.data.zero_() #梯度数据清零

    print('progress:', epoch, l.item())

print('predic (after training)', 4, forward(4).item())

结果：

三、课后作业

$\frac{\partial loss}{\partial w_{1}}=\frac{\partial ((\widehat{y}-y)^{2})}{\partial w_{1}}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial w_{1}}= 2(w_{1}x^{2}+w_{2}x+b)\cdot x^{2}$

$\frac{\partial loss}{\partial w_{2}}=\frac{\partial ((\widehat{y}-y)^{2})}{\partial w_{2}}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial w_{2}}= 2(w_{1}x^{2}+w_{2}x+b)\cdot x$

$\frac{\partial loss}{\partial b}=\frac{\partial ((\widehat{y}-y)^{2})}{\partial b}=2(\widehat{y}-y)\cdot \frac{\partial (\widehat{y}-y)}{\partial b}= 2(w_{1}x^{2}+w_{2}x+b)\cdot 1$

import numpy as np
import matplotlib.pyplot as plt
import torch

x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]

w1 = torch.Tensor([1.0])
w1.requires_grad = True   

w2 = torch.Tensor([1.0])
w2.requires_grad = True 

b = torch.Tensor([1.0])
b.requires_grad = True 

def forward(x):
    return x * x * w1 + x * w2 + b 

def loss(x, y):
    y_pred = forward(x)
    return (y_pred - y) ** 2

print('predict (before training)', 4, forward(4).item())

for epoch in range(100):
    for x, y in zip(x_data, y_data):
        l = loss(x, y)
        l.backward()  
        print('\tgrad:', x, y, w1.grad.item(), w2.grad.item(), b.grad.item()) 
        w1.data = w1.data - 0.01 * w1.grad.data  
        w2.data = w2.data - 0.01 * w2.grad.data
        b.data = b.data - 0.01 * b.grad.data

        w1.grad.data.zero_() 
        w2.grad.data.zero_()
        b.grad.data.zero_()

    print('progress:', epoch, l.item())

print('predic (after training)', 4, forward(4).item())

结果：