摘要
本文使用纯 Python 和 PyTorch 对比实现affine/linear(仿射/线性)变换函数及其反向传播.
相关
原理和详细解释, 请参考文章 :
affine/linear(仿射/线性)变换函数详解及全连接层反向传播的梯度求导
系列文章索引 :
https://blog.youkuaiyun.com/oBrightLamp/article/details/85067981
正文
import torch
import numpy as np
class Affine:
def __init__(self):
self.x = None
self.weight = None
self.bias = None
self.dx = None
self.dw = None
self.db = None
def __call__(self, x):
self.x = x
out = np.dot(self.x, self.weight.T) + self.bias
return out
def backward(self, d_