本文介绍如何使用Caffe框架复现ResNet模型在Cifar10数据集上的实验过程及结果，包括ResNet基本模块的实现、网络定义与训练配置。
转载地址：http://blog.youkuaiyun.com/a_1937/article/details/51507545
ResNet在2015年的ImageNet竞赛上的识别率达到了非常高的水平，这里我将使用Caffe在Cifar10上复现论文4.2节的Cifar实验。
- ResNet的基本模块
- Caffe实现
- Cifar10上的实验结果及说明
ResNet的基本模块
本文参照Torch7在Cifar10上复现ResNet的实验，使用Caffe复现之。ResNet的基本模块可以如下Python代码:
<code class="language-python hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-decorator" style="color: rgb(0, 102, 102); box-sizing: border-box;">@requires_authorization</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> __future__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> print_function
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> layers <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> L, params <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> P, to_proto
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe.proto <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> caffe_pb2
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> caffe
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># helper function for building ResNet block structures </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># The function below does computations: bottom--->conv--->BatchNorm</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">conv_factory</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, ks, n_out, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)</span>:</span>
conv = L.Convolution(bottom, kernel_size=ks, stride=stride, num_output=n_out, pad=pad,
param = [dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)],
bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
batch_norm = L.BatchNorm(conv, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>,
param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)])
scale = L.Scale(batch_norm, bias_term=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> scale
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># bottom--->conv--->BatchNorm--->ReLU</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">conv_factory_relu</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, ks, n_out, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)</span>:</span>
conv = L.Convolution(bottom, kernel_size=ks, stride=stride, num_output=n_out, pad=pad,
param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)],
bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
batch_norm = L.BatchNorm(conv, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)])
scale = L.Scale(batch_norm, bias_term=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
relu = L.ReLU(scale, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> relu
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Residual building block! Implements option (A) from Section 3.3. The input</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># is passed through two 3x3 convolution layers. Currently this block only supports </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># stride == 1 or stride == 2. When stride is 2, the block actually does pooling.</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Instead of simply doing pooling which may cause representational bottlneck as</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># described in inception v3, here we use 2 parallel branches P && C and add them</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># together. Note pooling branch may has less channels than convolution branch so we</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># need to do zero-padding along channel dimension. And to the best knowledge of</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ours, we haven't found current caffe implementation that supports this operation. </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># So later I'll give implementation in C++ and CUDA.</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">residual_block</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, num_filters, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)</span>:</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> stride == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:
conv1 = conv_factory_relu(bottom, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
conv2 = conv_factory(conv1, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
add = L.Eltwise(bottom, conv2, operation=P.Eltwise.SUM)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> add
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">elif</span> stride == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>:
conv1 = conv_factory_relu(bottom, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
conv2 = conv_factory(conv1, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
pool = L.Pooling(bottom, pool=P.Pooling.AVE, kernel_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
pad = L.PadChannel(pool, num_channels_to_pad=num_filters / <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
add = L.Eltwise(conv2, pad, operation=P.Eltwise.SUM)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> add
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span>:
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">raise</span> Exception(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'Currently, stride must be either 1 or 2.'</span>)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Generate resnet cifar10 train && test prototxt. n_size control number of layers.</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># The total number of layers is 6 * n_size + 2. Here I don't know any of implementation </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># which can contain simultaneously TRAIN && TEST phase. </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ==========================Note here==============================</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># !!! SO YOU have to include TRAIN && TEST by your own AFTER you use the script to generate the prototxt !!!</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">resnet_cifar</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(train_lmdb, test_lmdb, mean_file, batch_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>, n_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)</span>:</span>
data, label = L.Data(source=test_lmdb, backend=P.Data.LMDB, batch_size=batch_size, ntop=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,
transform_param=dict(mean_file=mean_file, crop_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span>), include=dict(phase=getattr(caffe_pb2, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'TEST'</span>)))
residual = conv_factory_relu(data, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 16, 32, 32 1st group</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size):
residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 32, 16, 16 2nd group</span>
residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>):
residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 64, 8, 8 3rd group</span>
residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>):
residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># -------------> end of residual</span>
global_pool = L.Pooling(residual, pool=P.Pooling.AVE, global_pooling=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
fc = L.InnerProduct(global_pool, param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)],num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>,
bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
loss = L.SoftmaxWithLoss(fc, label)
acc = L.Accuracy(fc, label, include=dict(phase=getattr(caffe_pb2, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'TEST'</span>)))
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> to_proto(loss, acc)
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">make_net</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(tgt_file)</span>:</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">with</span> open(tgt_file, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'w'</span>) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> f:
print(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'name: "resnet_cifar10"'</span>, file=f)
print(resnet_cifar(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/cifar10_train_lmdb'</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/cifar10_test_lmdb'</span>,
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/mean.proto'</span>, n_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>), file=f)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> __name__ == <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'__main__'</span>:
tgt_file=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'D:/VSProjects/caffe/models/ucas_resnet_cifar10/res56_cifar_train_test.prototxt'</span>
make_net(tgt_file)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li></ul>
Caffe实现
首先所有的卷积核采用的都是3x3大小，当stride=1时在ResNet一个block里面通过padding那么feature map的尺寸不变。这是输入可以和经过两个卷积的分支相加。当stride=2时，通过两个卷积的分支feature map的尺寸减半，输入可以通过pooling减半，变为和经过两个卷积分支的尺寸一样，但是可能会导致pooling分支的channel数目少于卷积分支的数目。有两种方式解决这个问题，如原文所描述的那样：1、通过1x1的卷积线性投影，不接任何激活函数，然后直接将卷积核个数设置到跟卷积分支输出的channels个数相等； 2：通过对channel维度进行加零填充，即zero-padding。这里我采用后者方式，下面给出后者的Caffe实现代码（也就是上面python代码中的L.PadChannel对应的C++代码）：
<code class="language-C++ hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#ifndef CAFFE_PAD_CHANNEL_LAYER_HPP_</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#define CAFFE_PAD_CHANNEL_LAYER_HPP_</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/blob.hpp"</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layer.hpp"</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/proto/caffe.pb.h"</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/*
* @brief zero-padding channel to extend number of channels
*
* Note: Back-propagate just drop the pad derivatives
*/</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> PadChannelLayer : <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> Layer<Dtype> {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> PadChannelLayer(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> LayerParameter& param)
: Layer<Dtype>(param) {}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> LayerSetUp(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Reshape(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">char</span>* type() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"PadChannel"</span>; }
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> ExactNumBottomBlobs() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>; }
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> ExactNumTopBlobs() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>; }
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Forward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Backward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Forward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Backward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num_channels_to_pad_;
};
} <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#endif <span class="hljs-comment" style="box-sizing: border-box;">// CAFFE_PAD_CHANNEL_LAYER_HPP_</span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li></ul>
pad_channel_layer.cpp
<code class="hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layers/pad_channel_layer.hpp"</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::LayerSetUp(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
CHECK_NE(top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]) << <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>->type() << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" Layer does not "</span>
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"allow in-place computation."</span>;
num_channels_to_pad_ = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>->layer_param_.pad_channel_param().num_channels_to_pad();
CHECK_GT(num_channels_to_pad_, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"num channels to pad must greater than 0!"</span>;
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Reshape(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
<span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span>></span> top_shape = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->shape();
top_shape[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] += num_channels_to_pad_;
top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->Reshape(top_shape);
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Forward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* bottom_data = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->cpu_data();
Dtype* top_data = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_cpu_data();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->num();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channel_by_dim = channels * dim;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> n = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; n < num; n++){
caffe_copy(channel_by_dim, bottom_data, top_data);
bottom_data += channel_by_dim;
top_data += channel_by_dim;
caffe_set(num_channels_to_pad_ * dim, Dtype(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), top_data);
top_data += num_channels_to_pad_ * dim;
}
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Backward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top_diff = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->cpu_diff();
Dtype* bottom_diff = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_cpu_diff();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->num();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channel_by_dim = channels * dim;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> n = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; n < num; n++){ <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// just drop the padding derivatives part.</span>
caffe_copy(channel_by_dim, top_diff, bottom_diff);
top_diff += (channels + num_channels_to_pad_) * dim;
bottom_diff += channel_by_dim;
}
}
INSTANTIATE_CLASS(PadChannelLayer);
REGISTER_LAYER_CLASS(PadChannel);
} <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li></ul>
pad_channel_layer.cu
<code class="hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layers/pad_channel_layer.hpp"</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Copy (one line per thread) from one array to another, with arbitrary</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// strides in the last two dimensions.</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
__global__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> pad_forward_kernel(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_count, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> src_channels, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_channels,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* src, Dtype* dst)
{
CUDA_KERNEL_LOOP(index, dst_count)
{
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = index / (dim * dst_channels);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_c = index / dim % dst_channels;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> pixel_pos = index % dim;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (dst_c < src_channels)
dst[index] = src[num * src_channels * dim + dst_c * dim + pixel_pos];
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span>
dst[index] = Dtype(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>);
}
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Forward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top){
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* bottom_data = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->gpu_data();
Dtype* top_data = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_gpu_data();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> src_channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_channels = src_channels + num_channels_to_pad_;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_count = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->count();
pad_forward_kernel<Dtype> << <CAFFE_GET_BLOCKS(dst_count), CAFFE_CUDA_NUM_THREADS >> >(
dst_count, src_channels, dst_channels, dim, bottom_data, top_data);
CUDA_POST_KERNEL_CHECK;
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
__global__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> pad_backward_kernel(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_count, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_channels, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> top_channels,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top, Dtype* bottom)
{
CUDA_KERNEL_LOOP(index, bottom_count)
{
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = index / (dim * bottom_channels);
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_c = index / dim % bottom_channels;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> pixel_pos = index % dim;
bottom[index] = top[num * top_channels * dim + bottom_c * dim + pixel_pos];
}
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Backward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top_diff = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->gpu_diff();
Dtype* bottom_diff = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_gpu_diff();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_count = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->count();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> top_channels = bottom_channels + num_channels_to_pad_;
pad_backward_kernel<Dtype> << <CAFFE_GET_BLOCKS(bottom_count), CAFFE_CUDA_NUM_THREADS >> >(
bottom_count, bottom_channels, top_channels, dim, top_diff, bottom_diff);
CUDA_POST_KERNEL_CHECK;
}
INSTANTIATE_LAYER_GPU_FUNCS(PadChannelLayer);
} <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul>
实验结果
1、 使用上述脚本针对Cifar10生成20层的resnet。网络定义太长，这里就不给出来了，但要注意的是因为原图是32x32大小的，这里随机crop 28x28的，并镜像一下。不同于原论文所述的在图片各边补4个像素再随机裁剪32x32的。网络定义的输入部分以及solver.prototxt：
<code class="hljs css has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">name</span>: "<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">resnet_cifar10</span>"
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">layer</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">name</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"input"</span>
type: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data"</span>
top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data1"</span>
top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"label"</span>
include {
phase: TRAIN
</span></span></span>}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">transform_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">mean_file</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/mean.proto"</span>
crop_size: <span class="hljs-number" style="box-sizing: border-box;">28</span>
mirror: true
</span></span></span>}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">data_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">source</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/cifar10_train_lmdb"</span>
batch_size: <span class="hljs-number" style="box-sizing: border-box;">100</span>
backend: LMDB
</span></span></span>}
}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">layer</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">name</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"input"</span>
type: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data"</span>
top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data1"</span>
top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"label"</span>
include {
phase: TEST
</span></span></span>}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">transform_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">mean_file</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/mean.proto"</span>
crop_size: <span class="hljs-number" style="box-sizing: border-box;">28</span>
</span></span></span>}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">data_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
<span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">source</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/cifar10_test_lmdb"</span>
batch_size: <span class="hljs-number" style="box-sizing: border-box;">100</span>
backend: LMDB
</span></span></span>}
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li></ul>
solver.prototxt如下
<code class="hljs http has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">net</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"res20_cifar_train_test.prototxt"</span> <span class="hljs-attribute" style="box-sizing: border-box;">test_iter</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100 # conver the whole test set. 100 * 100 = 10000 images.</span> <span class="hljs-attribute" style="box-sizing: border-box;">test_interval</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">500 # Each 500 is one epoch, test after each epoch</span> <span class="hljs-attribute" style="box-sizing: border-box;">base_lr</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.1</span> <span class="hljs-attribute" style="box-sizing: border-box;">momentum</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.9</span> <span class="hljs-attribute" style="box-sizing: border-box;">weight_decay</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.0001</span> <span class="hljs-attribute" style="box-sizing: border-box;">average_loss</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100</span> <span class="hljs-attribute" style="box-sizing: border-box;">lr_policy</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"multistep"</span> <span class="hljs-attribute" style="box-sizing: border-box;">stepvalue</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">40000</span> <span class="hljs-attribute" style="box-sizing: border-box;">stepvalue</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">80000</span> <span class="hljs-attribute" style="box-sizing: border-box;">gamma</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.1</span> <span class="hljs-attribute" style="box-sizing: border-box;">display</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100</span> <span class="hljs-attribute" style="box-sizing: border-box;">max_iter</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100000 # 100000 iteration is 200 epochs</span> <span class="hljs-attribute" style="box-sizing: border-box;">snapshot</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">500</span> <span class="hljs-attribute" style="box-sizing: border-box;">snapshot_prefix</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"trained_model/cifar"</span> <span class="hljs-attribute" style="box-sizing: border-box;">solver_mode</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">GPU</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>
2、 网络结构可视化
由于网络结构太大，这里使用了Netscope对网络进行可视化，并截取部分网络结构图如下:
3、 Cifar10上20层和56层的实验结果准确率对比
| 网络层数 | 本文复现结果 | 原论文结果 |
|---|---|---|
| 20 | 91.19% | 91.25% |
| 56 | 92.54% | 93.03% |
4、 Cifar10 20层的训练曲线以及测试曲线
训练曲线图：
测试曲线图：
5、实验结论
从第3届的表格可以看出，本文使用Caffe复现的resnet在Cifar10上的识别率非常接近论文结果。但比论文结果稍低，可能的原因有，本实验使用的是28x28的切图，原文使用的是32x32的切图, 另外，初始化也有所不同。但总的来说，可以认为复现了原文的结果。
扫一扫
1127
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
