Pytorch框架学习第七次作业——损失函数

1. Lossfunction依旧属于网络层的概念，即仍旧是Module的子类，为了对lossfunction有一个更清晰的概念，需要大家采用步进(Step into)的调试方法从loss_functoin = nn.CrossEntropyLoss() 语句进入函数，观察从nn.CrossEntropyLoss()到class Module(object)一共经历了哪些类，记录其中所有进入的类及函数。

第一步：CrossEntropyLoss类，super(CrossEntropyLoss, self).init
第二步：_WeightedLoss类，super(_WeightedLoss, self).init(size_average, reduce, reduction)
第三步: _Loss类，super(_Loss, self).init()
第四步：进入Module 类

2. 损失函数的reduction有三种模式，它们的作用分别是什么？
   当inputs和target及weight分别如以下参数时，reduction=’mean’模式时，loss是如何计算得到的？
   inputs = torch.tensor([[1, 2], [1, 3], [1, 3]], dtype=torch.float)
   target = torch.tensor([0, 1, 1], dtype=torch.long)
   weights = torch.tensor([1, 2], dtype=torch.float)

• none：逐个元素计算
• sum：所有元素求和，返回标量
• mean：加权平均，返回标量

    inputs = torch.tensor([[1, 2], [1, 3], [1, 3]], dtype=torch.float)
    target = torch.tensor([0, 1, 1], dtype=torch.long)
    # def loss function
    weights = torch.tensor([1, 2], dtype=torch.float)

    loss_f_none_w = nn.CrossEntropyLoss(weight=weights, reduction='none')
    loss_f_sum = nn.CrossEntropyLoss(weight=weights, reduction='sum')
    loss_f_mean = nn.CrossEntropyLoss(weight=weights, reduction='mean')

    # forward
    loss_none_w = loss_f_none_w(inputs, target)
    loss_sum = loss_f_sum(inputs, target)
    loss_mean = loss_f_mean(inputs, target)

    # view
    print("\nweights: ", weights)
    print(loss_none_w, loss_sum, loss_mean)

将所有元素的loss相加除以所有元素的权重之和（1+2+2=5）