Pytorch：学习率设置笔记

最新推荐文章于 2025-04-29 16:22:08 发布

原创

最新推荐文章于 2025-04-29 16:22:08 发布 · 973 阅读

2 ·

CC 4.0 BY-SA版权

torch.optim.lr_scheduler 包中封装了几种基于epoch的学习率调整方法。

一、StepLR

torch.optim.lr_scheduler.StepLR(optimizer, step_size, gamma=0.1, last_epoch=-1)

Sets the learning rate of each parameter group to the initial lr decayed by gamma every step_size epochs. When last_epoch=-1, sets initial lr as lr.

# Assuming optimizer uses lr = 0.05 for all groups
# lr = 0.05     if epoch < 30
# lr = 0.005    if 30 <= epoch < 60
# lr = 0.0005   if 60 <= epoch < 90
# ...
scheduler = StepLR(optimizer, step_size=30, gamma=0.1)
for epoch in range(100):
    scheduler.step()
    train(...)
    validate(...)