深度学习入门笔记 Day8/15 误差反向传播法（二）

一、如何实现加法层和乘法层

用代码实现昨天的苹果、橘子问题。

这里的layer应该理解为节点，MulLayer是乘法节点的实现，AddLayer是加法节点的实现。 

对于每个节点声明一个类变量

mul_apple_layer = MulLayer()    #节点1，乘法节点，苹果单价*苹果个数
mul_orange_layer = MulLayer()    #节点2，乘法节点，橘子单价*橘子个数
add_apple_orange_layer = AddLayer()    #节点3，加法节点，苹果总价+橘子总价
mul_tax_layer = MulLayer()    #节点4，乘法节点，总价*税率

 正向求得税后总价，反向求得各输入的偏导。

class MulLayer:
    def __init__(self):
        self.x = None
        self.y = None

    def forward(self, x, y):    #前向，存储输入变量x，y，便于反向求偏导时使用
        self.x = x
        self.y = y
        out = x * y

        return out

    def backward(self, dout):
        dx = dout*self.y
        dy = dout*self.x

        return dx, dy


class AddLayer:
    def __init__(self):
        pass

    def forward(self, x, y):
        out = x+ y
        return out

    def backward(self, dout):
        dx = dout*1
        dy = dout*1
        return dx, dy


if __name__ == '__main__':
    apple = 100
    apple_num = 2
    orange = 150
    orange_num = 3
    tax = 1.1

    # layer 理解成节点，图上有四个节点，声明四个class
    mul_apple_layer = MulLayer()
    mul_orange_layer = MulLayer()
    add_apple_orange_layer = AddLayer()
    mul_tax_layer = MulLayer()

    # forward
    apple_price = mul_apple_layer.forward(apple, apple_num)
    orange_price = mul_orange_layer.forward(orange, orange_num)
    all_price = add_apple_orange_layer.forward(apple_price,orange_price)
    price = mul_tax_layer.forward(all_price, tax)

    print(price)

    # backward
    dprice = 1
    dall_price, dtax = mul_tax_layer.backward(dprice)
    dapple_price, dorange_price = add_apple_orange_layer.backward(dall_price)
    dapple, dapple_num = mul_apple_layer.backward(dapple_price)
    dorange, dorange_num = mul_orange_layer.backward(dorange_price)

    print(dapple, dapple_num, dorange, dorange_num, dtax)

二、如何实现ReLU层和Sigmoid层

回顾一下ReLU函数：