xungeer29 的博客

Bag of Tricks for Image Classification with Convolutional Neural Networksarxiv论文详解-知乎Large-batch trainingLinear scaling learning rateLearning rate warmupZero γ\gammaγNo bias decayLow-precisi...

Gradually warm-up(increasing) learning rate for pytorch’s optimizer. Proposed in ‘Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour’.# from:https://github.com/ildoonet/pytorch-gradual-warmup...

import torchimport torch.nn as nnclass LabelSmoothSoftmaxCE(nn.Module): def __init__(self, lb_pos=0.9, lb_neg=0.005, reduction='mean', ...