本文探讨了在训练过程中遇到的损失发散问题，即损失值变为NaN的情况，并提供了调试建议，比如尝试使用0学习率来检查是否有数值错误出现，以及进行不同的参数初始化策略。
by Yangqing on 14 May 2014
For a sanity check, try running with a learning rate 0 to see if any nan errors pop up (they shouldn’t, since no learning takes place). If data is not initialized well, it might be possible that even 0.0001 is a too high learning rate.
by sguada on 13 May 2014
Try different initializations, for instance bias set to 0.1
References:
On custom data training diverges (loss = NaN) #409
nan issue with CIFAR10 example when running on CPU only #393
扫一扫
7万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
