tensorflow交叉熵损失函数-cross_entropy_with_logits 在库函数中比较

最新推荐文章于 2024-09-07 22:06:23 发布

原创

最新推荐文章于 2024-09-07 22:06:23 发布 · 679 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#tensorflow #深度学习 #机器学习

本文详细探讨了二分类与多分类任务中交叉熵损失函数的理论背景，包括sigmoid和softmax的适用场景，以及如何避免溢出和稳定计算。实例演示了SigmoidBinaryCrossEntropyLoss的实现，并展示了在多分类问题中使用softmax和mask处理的技巧。

交叉熵损失函数设计：

softmax_cross_entropy_with_logits_v2
sparse_softmax_cross_entropy_with_logits
softmax_cross_entropy_with_logits_v2
sparse_softmax_cross_entropy_with_logits
sigmoid_cross_entropy_with_logits[sigmoid_cross_entropy_with_logits_v2]

首先，理论上：
二分类：
在这里插入图片描述
直接用 sigmoid_cross_entropy_with_logits[或者sigmoid_cross_entropy_with_logits_v2] 就可以一个label 一个raw logist
多分类：

-Sum(Yt*LogYp)
其实，这个多分类功能能处理的情况包含两种情况：

  Measures the probability error in discrete classification tasks in which each
  class is independent and not mutually exclusive.  For instance, one could
  perform multilabel classification where a picture can contain both an elephant
  and a dog at the same time.

  Measures the probability error in discrete classification tasks in which the
  classes are mutually exclusive (each entry is in exactly one class).  For
  example, each CIFAR-10 image is labeled with one and only one label: an image
  can be a dog or a truck, but not both.

  **NOTE:**  While the classes are mutually exclusive, their probabilities
  need not be.  All that is required is that each row of `labels` is
  a valid probability distribution.  If they are not, the computation of the
  gradient will be incorrect.

  For brevity, let `x = logits`, `z = labels`.  The logistic loss is

        z * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
      = z * -log(1 / (1 +

最低0.47元/天解锁文章