caffe softmax_loss_layer 对于梯度下降的理解

咱先讲下梯度下降,然后对应caffe里的代码。
还是老样子,贴一个网址,人家讲softmax讲的挺好的。
http://www.bubuko.com/infodetail-601263.html
总的来说,梯度下降主要分为一下几部分:
1.先求假设函数;
2.再求代价函数,又叫损失函数;
3.对代价函数求导,最小化;
4.更新,梯度下降公式这就是梯度下降的公式,(斯坦福公开课里形象的解释了:站在小山上,每往下走一步,就观察一下,看是不是最快的下山的路,每次的观察,就是寻找最优的thet的方法。)

我经常会出现一种问题,就是算法看完了,没过多久就忘了,下次还得再看,对这个问题,我的方法就是看代码,把代码看熟悉了,也就记住了。

下面讲代码

**softmax_loss_layer.cpp**
主要看他的forward和backword
template <typename Dtype>
void SoftmaxWithLossLayer<Dtype>::Forward_cpu(
    const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
  // The forward pass computes the softmax prob values.
  //打印了此时的 层参数 见附1
  softmax_layer_->Forward(softmax_bottom_vec_, softmax_top_vec_);
 /* softmax_loss_layer.cpp里LayerSetUp() 函数对上边两个参数进行了定义
 softmax_bottom_vec_.push_back(bottom[0]);
  softmax_top_vec_.clear();
  softmax_top_vec_.push_back(&prob_);
  softmax_layer_->SetUp(softmax_bottom_vec_, softmax_top_vec_);*/
  这里Forward调用了softmax_layer.cpp的Forward_cpu()函数,求出假设函数 h(x)见附2  
/*
(gdb) p *top[0]
$11 = {data_ = {px = 0x7790a0, pn = {pi_ = 0x7790f0}}, diff_ = {px = 0x77b220, 
    pn = {pi_ = 0x7797b0}}, shape_data_ = {px = 0x778ad0, pn = {
      pi_ = 0x779110}}, shape_ = std::vector of length 0, capacity 0, 
  count_ = 1, capacity_ = 1}
(gdb) p *bottom[0]
$12 = {data_ = {px = 0x777e30, pn = {pi_ = 0x772bb0}}, diff_ = {px = 0x7786c0, 
    pn = {pi_ = 0x7786f0}}, shape_data_ = {px = 0x778600, pn = {
      pi_ = 0x778630}}, shape_ = std::vector of length 2, capacity 2 = {100, 
    10}, count_ = 1000, capacity_ = 1000}
(gdb) p *softmax_bottom_vec_[0]
$7 = {data_ = {px = 0x777e30, pn = {pi_ = 0x772bb0}}, diff_ = {px = 0x7786c0, 
    pn = {pi_ = 0x7786f0}}, shape_data_ = {px = 0x778600, pn = {
      pi_ = 0x778630}}, shape_ = std::vector of length 2, capacity 2 = {100, 
    10}, count_ = 1000, capacity_ = 1000}
(gdb) p *softmax_top_vec_[0]
$8 = {data_ = {px = 0x779f00, pn = {pi_ = 0x779f30}}, diff_ = {px = 0x779f50, 
    pn = {pi_ = 0x779f80}}, shape_data_ = {px = 0x779e90, pn = {
      pi_ = 0x779ec0}}, shape_ = std::vector of length 2, capacity 2 = {100, 
    10}, count_ = 1000, capacity_ = 1000}
    你可以发现 bottom和softmax_bottom_vec_所指向的地址是相同的
*/
  const Dtype* prob_data = prob_.cpu_data();
  const Dtype* label = bottom[1]->cpu_data();
  int dim = prob_.count() / outer_num_;
  int count = 0;
  Dtype loss = 0;
  //针对每一张图像
  for (int i = 0; i < outer_num_; ++i) {
    for (int j = 0; j < inner_num_; j++) {
      const int label_value = static_cast<int>(label[i * inner_num_ + j]);
      if (has_ignore_label_ && label_value == ignore_label_) {
        continue;
      }
      DCHECK_GE(label_value, 0);
      DCHECK_LT(label_value, prob_.shape(softmax_axis_));
      //prob_.shape(softmax_axis_)= 10
      /// prob stores the output probability predictions from the SoftmaxLayer.
      //Blob<Dtype> prob_;

      loss -= log(std::max(prob_data[i * dim + label_value * inner_num_ + j],
                           Dtype(FLT_MIN)));

      ++count;
    }
  }
  top[0]->mutable_cpu_data()[0] = loss / get_normalizer(normalization_, count);
   //求代价函数也就是这个公式![这里写图片描述](https://img-blog.youkuaiyun.com/20160511174720969) 
  if (top.size() == 2) {
    top[1]->ShareData(prob_);
  }
}


附1
p *
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值