pytorch学习——反向传播

由上图，前向传播算出loss，再利用链式法则，倒推w

 y'=2,r=-2,loss=4

偏导：loss对r=-4,loss-y'=-4,loss-w=-8

 

 课上代码：

import numpy.matlib
import torch
import numpy as np

x=[1.0,2.0,3.0]
y=[2.0,4.0,6.0]

#创建张量并允许求导
w=torch.Tensor([1.0])
w.requires_grad=True

def forward(x):
    return x*w

def loss(x,y):
    y_pred=forward(x)
    return(y_pred-y)**2

print("predict(before training)",4,forward(4).item())
for epoch in range(100):
    for i,j in zip (x,y):
        l=loss(i,j)
        l.backward()#进行梯度值运算
        print("\tgrad:",i,j,w.grad.item())  
        w.data=w.data-0.01*w.grad.data
        w.grad.data.zero_()  
    print("progress:",epoch,l.item())

print("predict(after training)",4,forward(4).item())

作业：

import numpy.matlib
import torch
import numpy as np

x=[1.0,2.0,3.0]
y=[11.0,20.0,33.0]

w1=torch.Tensor([1.9])
w1.requires_grad=True
w2=torch.Tensor([2.9])
w2.requires_grad=True
b=torch.Tensor([5.9])
b.requires_grad=True

def forward(x):
    return x*x*w1+x*w2+b

def loss(x,y):
    y_pred=forward(x)
    return(y_pred-y)**2

print("predict(before training)",4,forward(4).item())
for epoch in range(100):
    for i,j in zip (x,y):
        l=loss(i,j)
        l.backward()
        print("\tgrad:",i,j,w1.grad.item(),w2.grad.item(),b.grad.item())  
        w1.data=w1.data-0.01*w1.grad.data
        w2.data=w2.data-0.01*w2.grad.data
        b.data=b.data-0.01*b.grad.data
        w1.grad.data.zero_()
        w2.grad.data.zero_()
        b.grad.data.zero_()  
    print("progress:",epoch,l.item())

print("predict(after training)",4,forward(4).item())

训练后的结果为： 

predict(after training) 4 49.99744415283203

与真实值50相差不大。 

另外画图出现了这个错误Import 'matplotlib.pyplot'

发现是虚拟环境中没有安装matplotlib库，装完尝试画图。

画出来图像很奇怪，怀疑是横纵坐标选取不对。、

重新选取，令x=epoch,y=loss，图像如下

 

另外，将3个权重放到同一张量中可以简化代码，代码如下

import numpy.matlib
import torch
import numpy as np
import matplotlib.pyplot as plt

x=[1.0,2.0,3.0]
y=[11.0,20.0,33.0]

w=torch.tensor([[1.9],[2.9],[5.9]])
w.requires_grad=True

def forward(x):
      return x*x*torch.take(w,torch.tensor([0]))+x*torch.take(w,torch.tensor([1]))+torch.take(w,torch.tensor([2]))

def loss(x,y):
    y_pred=forward(x)
    return(y_pred-y)**2

print("predict(before training)",4,forward(4).item())
l_try=[]
epoch_try=[]
for epoch in range(100):
    for i,j in zip (x,y):
        l=loss(i,j)
        l.backward()
        l_try.append(l.item())
        epoch_try.append(epoch)
        w.data=w.data-0.01*w.grad.data
        w.grad.data.zero_()
    print("progress:",epoch,l.item())

print("predict(after training)",4,forward(4).item())

plt.plot(epoch_try,l_try)
plt.show()

ps:选取tensor的特定值给我看吐了。

参考：

(45条消息) Pytorch基础操作 —— 12. 从张量中选取数据_打码的老程的博客-优快云博客_pytorch 张量取值

torch.index_select(input, dim, index, *, out=None) → Tensor

 

我吐了，为什么不说清楚！！！index这个参数必须是tensor，不然就会报错。 

！！！我吐了！！！居然用索引就可以。。。白查那么久博客

def forward(x):
      return x*x*torch.take(w,torch.tensor([0]))+x*torch.take(w,torch.tensor([1]))+torch.take(w,torch.tensor([2]))

可以改为：

w=torch.tensor([1.9,2.9,5.9])

def forward(x):
    return x*x*w[0]+x*w[1]+w[3]

!!!为什么可以这么简单！！！

还有，他喵的，讲torch.gather函数的博客我一篇都没看懂。

直到遇到了它：
https://zhuanlan.zhihu.com/p/462008911

感谢老师，感谢。