Caffe Prototxt 特征层系列：InnerProduct Layer

InnerProduct Layer是全连接层，CNN中常常出现在分类网络的末尾，将卷积全连接化或分类输出结果，当然它的用处很多，不只是分类网络中

首先我们先看一下 InnerProductParameter

message InnerProductParameter {
	  optional uint32 num_output = 1; // The number of outputs for the layer   //输出的个数
	  optional bool bias_term = 2 [default = true]; // whether to have bias terms  //是否使用bias
	  optional FillerParameter weight_filler = 3; // The filler for the weight  //全连接权重
	  optional FillerParameter bias_filler = 4; // The filler for the bias   //bias权重
	
	  // The first axis to be lumped into a single inner product computation;
	  // all preceding axes are retained in the output.
	  // May be negative to index from the end (e.g., -1 for the last axis).
	  //将第一个轴集中进行单个内积计算
	  optional int32 axis = 5 [default = 1];
	  // Specify whether to transpose the weight matrix or not.
	  // If transpose == true, any operations will be performed on the transpose
	  // of the weight matrix. The weight matrix itself is not going to be transposed
	  // but rather the transfer flag of operations will be toggled accordingly.
	  // 是否要转置权矩阵。如果true，则对权矩阵执行转置操作，注意权重矩阵本身不会被置换，而是操作传输标志作相应切换
	  optional bool transpose = 6 [default = false];
}

InnerProduct layer 在prototxt里面的书写：

layer {
	  name: "fc"
	  type: "InnerProduct"  
	  bottom: "fc_in"
	  top: "fc_out"
	  
	  param {  # 权重学习参数
		    lr_mult: 1  # 学习率
		    decay_mult: 1
	  }
	  param {  # bias 学习参数
		    lr_mult: 2  # 一般情况，bias 学习率是权重学习率的两倍.
		    decay_mult: 0
	  }
	  
	  inner_product_param {
		    num_output: 1000  # 输出单元个数 
		    weight_filler {  # 权重初始化方法
		      type: "gaussian"
		      std: 0.005
		    }
		    bias_filler {  # bias 初始化方法
		      type: "constant"
		      value: 0.1
		    }
	 }
}

有些网络在实现时，常常用卷积来代替全连接层的功能，如MobileNet里面：

layer {
	  name: "fc7"
	  type: "Convolution"
	  bottom: "pool6"
	  top: "fc7"
	  param {
		    lr_mult: 1
		    decay_mult: 1
	  }
	  param {
		    lr_mult: 2
		    decay_mult: 0
	  }
	  convolution_param {
		    num_output: 1000
		    kernel_size: 1
		    weight_filler {
		      type: "msra"
		    }
		    bias_filler {
		      type: "constant"
		      value: 0
		    }
	}
}