torch入门笔记20:torch实现freeze layer的操作

这两天一直在尝试着在torch的框架内实现freeze layer，通过search google，从极少的文档中找到比较work的方法，故而总结在这。

You can set the learning rate of certain layers to zero by overriding their updateParameters and accGradParameters to zero. You don't necessarily have to subclass, but it is cleaner.

If you are building the model for the first time, you simply get the initialized layer's handle and overrride the mentioned functions.

If you have a pre-trained network, and you want to freeze certain layers, you can either use the generic :apply function, that applies a closure to each layer, such as:

https://gist.github.com/soumith/5010de75f7a6805d33c9

Or you can traverse them via traversing the model's .modules table:

https://gist.github.com/soumith/b910efc4dd559736a9b0

Finally, here's an example freezing the first 3 layers of a net:

https://gist.github.com/soumith/6cd0f9b8462d0507a91b

• 
  

link:https://www.reddit.com/r/MachineLearning/comments/44ochv/forcing_learning_rate_to_zero_in_torch/

model = nn.Sequential()
	model:add(nn.SpatialConvolution(3, 64, 5, 5, 1, 1))
	model:add(nn.ReLU())
	model:add(nn.SpatialConvolution(64, 64, 5, 5, 1, 1))