卷积神经网络调参经验

最新推荐文章于 2025-06-12 23:39:14 发布

原创

最新推荐文章于 2025-06-12 23:39:14 发布 · 2.8k 阅读

6 ·

CC 4.0 BY-SA版权

文章标签：

#神经网络

本文探讨了二值化神经网络中精度下降的问题，引用论文指出较低的学习率更适合训练BNN以避免频繁的权重符号变化。同时，列举了多个关于神经网络调参的资料和技巧，包括数据预处理、初始化、反向传播优化、激活函数选择、特征工程等方面，旨在提升深度学习模型的性能和效率。

实习遇到的问题吧二值化网络加入XNor操作后，精度下降严重，更改第一层最后一层不使用Xnor，问题仍未解决，一时无从下手。

犹犹豫豫看到一篇paper
How to Train a Compact Binary Neural Network with High Accuracy

To consolidate our analysis, we have the statistics on how
many weights are changing their signs under different learning
rates during BNN and full precision network training.
The results are shown in Figure 2. It can be observed that,
under the same learning rate of 0.01, the sign changes for
BNN is nearly 3 orders of magnitude larger than that of a
full precision network. Only when the learning rate of BNN
is lowered to 0.0001, the two results become close.
Hence we conclude that a lower learning rate is more
preferred for training BNN to avoid frequent sign changes.
Our experiments show that when the learning rate is lowered
to 0.0001, a tremendous accuracy gain is obtained and
BNN can even achieve comparable results with XNOR-net.