torch.optim.lr_scheduler.LambdaLR与OneCycleLR

最新推荐文章于 2024-07-17 15:19:09 发布
最新推荐文章于 2024-07-17 15:19:09 发布
阅读量3.7k 收藏 13
点赞数 3
分类专栏： pytorch 文章标签： pytorch 人工智能 python
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/qq_37164776/article/details/121078905
版权

目录

LambdaLR 

输出

OneCycleLR

输出

LambdaLR 

函数接口：

LambdaLR(optimizer, lr_lambda, last_epoch=-1, verbose=False)

更新策略：

new\_lr = \lambda \times initial\_lr

 其中 new \_lr是得到的新的学习率，initial\_lr是初始的学习率， λ是通过参数lr_lambda和epoch得到的。

参数：

optimizer （Optimizer）：要更改学习率的优化器；

lr_lambda（function or list）：可以为一个lambda函数，也可以传入列表；

last_epoch （int）：最后一个epoch的index，如果是训练了很多个epoch后中断了，继续训练，这个值就等于加载的模型的epoch。默认为-1表示从头开始训练，即从epoch=1开始。

verbose(bool)：如果未True，则打印输出信息对于每次更新，反之亦然。默认为False。

import torch
import torch.nn as nn
from torch.optim.lr_scheduler import LambdaLR

initial_lr = 0.1

class model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=3, kernel_size=3)

    def forward(self, x):
        pass

net_1 = model()

optimizer_1 = torch.optim.SGD(net_1.parameters(), lr = initial_lr)
print(optimizer_1)
scheduler_1 = LambdaLR(optimizer_1, lr_lambda=lambda epoch: 1/(epoch+1))

print("初始化的学习率：", optimizer_1.defaults['lr'])

for epoch in range(1, 11):
    # train

    optimizer_1.zero_grad()
    optimizer_1.step()
    print("*-------------------------------------------------------*")
    print("更新前，epoch = %d lr = %f" % (epoch, optimizer_1.param_groups[0]['lr']))
    scheduler_1.step()
    print("更新后，epoch = %d lr = %f" % (epoch + 1, optimizer_1.param_groups[0]['lr']))
    print("更新后，epoch = %d lr = %f" % (epoch + 1, (1/(epoch+1))*initial_lr))
    print("*-------------------------------------------------------*")

输出

SGD (
Parameter Group 0
    dampening: 0
    lr: 0.1
    momentum: 0
    nesterov: False
    weight_decay: 0
)
初始化的学习率： 0.1
*-------------------------------------------------------*
更新前，epoch = 1 lr = 0.100000
更新后，epoch = 2 lr = 0.050000
更新后，epoch = 2 lr = 0.050000
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 2 lr = 0.050000
更新后，epoch = 3 lr = 0.033333
更新后，epoch = 3 lr = 0.033333
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 3 lr = 0.033333
更新后，epoch = 4 lr = 0.025000
更新后，epoch = 4 lr = 0.025000
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 4 lr = 0.025000
更新后，epoch = 5 lr = 0.020000
更新后，epoch = 5 lr = 0.020000
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 5 lr = 0.020000
更新后，epoch = 6 lr = 0.016667
更新后，epoch = 6 lr = 0.016667
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 6 lr = 0.016667
更新后，epoch = 7 lr = 0.014286
更新后，epoch = 7 lr = 0.014286
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 7 lr = 0.014286
更新后，epoch = 8 lr = 0.012500
更新后，epoch = 8 lr = 0.012500
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 8 lr = 0.012500
更新后，epoch = 9 lr = 0.011111
更新后，epoch = 9 lr = 0.011111
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 9 lr = 0.011111
更新后，epoch = 10 lr = 0.010000
更新后，epoch = 10 lr = 0.010000
*-------------------------------------------------------*
*-------------------------------------------------------*
更新前，epoch = 10 lr = 0.010000
更新后，epoch = 11 lr = 0.009091
更新后，epoch = 11 lr = 0.009091
*-------------------------------------------------------*

OneCycleLR

import cv2
import torch.nn as nn
import torch
from torchvision.models import AlexNet
import matplotlib.pyplot as plt
from torch.optim.lr_scheduler import LambdaLR
import math

total_steps = 300

# 自定义函数OneCycleLR
def one_cycle(y1=0.0, y2=1.0, steps=100):
    # lambda function for sinusoidal ramp from y1 to y2
    return lambda x: ((1 - math.cos(x * math.pi / steps)) / 2) * (y2 - y1) + y1

lf = one_cycle(1, 0.2, total_steps)  # cosine 1->hyp['lrf']

lr = 0.001
# pytorch函数OneCycleLR
lrs = []
steps = []
model = AlexNet(num_classes=20)
optimizer = torch.optim.SGD(model.parameters(), lr=lr, momentum=0.9)

scheduler =torch.optim.lr_scheduler.OneCycleLR(optimizer,max_lr=0.001,total_steps=total_steps, 
                                               verbose=False)

# 自定义函数OneCycleLR与LambdaLR结合使用
lrs1 = []
steps1 = []
model1 = AlexNet(num_classes=20)
optimizer1 = torch.optim.SGD(model.parameters(), lr=lr, momentum=0.9)
scheduler1 = LambdaLR(optimizer1, lr_lambda=lf)


for epoch in range(total_steps):
    scheduler1.step()
    lrs1.append(scheduler1.get_lr()[0])
    steps1.append(epoch)
    
    scheduler.step()
    lrs.append(scheduler.get_lr()[0])
    steps.append(epoch)
    
    print("custom OneCycleLR: ",scheduler1.get_lr()[0],
          "pytorch OneCycleLR: ",scheduler.get_lr()[0])
    
plt.figure()
plt.legend()
plt.plot(steps, lrs, color='black', marker='',label='pytorch')

plt.plot(steps1, lrs1, color='red', marker='',label='custom')

plt.savefig("OneCycleLR.png")

输出

custom OneCycleLR:  0.000999978067746205 pytorch OneCycleLR:  4.029901011425041e-05
custom OneCycleLR:  0.0009999122733899382 pytorch OneCycleLR:  4.1195667927633396e-05
custom OneCycleLR:  0.0009998026241462925 pytorch OneCycleLR:  4.268885631616862e-05
custom OneCycleLR:  0.0009996491320395434 pytorch OneCycleLR:  4.477671495306198e-05
custom OneCycleLR:  0.0009994518139018296 pytorch OneCycleLR:  4.745664262643999e-05
custom OneCycleLR:  0.0009992106913713087 pytorch OneCycleLR:  5.072530048013703e-05
custom OneCycleLR:  0.0009989257908897833 pytorch OneCycleLR:  5.4578616173493744e-05
custom OneCycleLR:  0.0009985971436998018 pytorch OneCycleLR:  5.901178895498596e-05
custom OneCycleLR:  0.000998224785841232 pytorch OneCycleLR:  6.40192956433644e-05
custom OneCycleLR:  0.0009978087581473092 pytorch OneCycleLR:  6.959489750884987e-05
custom OneCycleLR:  0.0009973491062401586 pytorch OneCycleLR:  7.573164804581384e-05
custom OneCycleLR:  0.0009968458805257913 pytorch OneCycleLR:  8.242190162726e-05
custom OneCycleLR:  0.0009962991361885775 pytorch OneCycleLR:  8.965732303032237e-05
custom OneCycleLR:  0.0009957089331851954 pytorch OneCycleLR:  9.742889782091613e-05
custom OneCycleLR:  0.0009950753362380552 pytorch OneCycleLR:  0.00010572694358459967
custom OneCycleLR:  0.000994398414828202 pytorch OneCycleLR:  0.00011454112198965667
custom OneCycleLR:  0.0009936782431876968 pytorch OneCycleLR:  0.00012386045166737076
custom OneCycleLR:  0.0009929149002914754 pytorch OneCycleLR:  0.0001336733218934436
custom OneCycleLR:  0.0009921084698486888 pytorch OneCycleLR:  0.00014396750705351078
custom OneCycleLR:  0.0009912590402935224 pytorch OneCycleLR:  0.0001547301818747355
custom OneCycleLR:  0.000990366704775499 pytorch OneCycleLR:  0.0001659479374045014
custom OneCycleLR:  0.0009894315611492642 pytorch OneCycleLR:  0.0001776067977162975
custom OneCycleLR:  0.0009884537119638544 pytorch OneCycleLR:  0.00018969223732198301
custom OneCycleLR:  0.0009874332644514525 pytorch OneCycleLR:  0.0002021891992687357
custom OneCycleLR:  0.0009863703305156273 pytorch OneCycleLR:  0.00021508211389814147
custom OneCycleLR:  0.0009852650267190633 pytorch OneCycleLR:  0.00022835491824404845
custom OneCycleLR:  0.0009841174742707772 pytorch OneCycleLR:  0.000241991076045024
custom OneCycleLR:  0.000982927799012827 pytorch OneCycleLR:  0.00025597359834647615
custom OneCycleLR:  0.0009816961314065109 pytorch OneCycleLR:  0.0002702850646667766
custom OneCycleLR:  0.0009804226065180616 pytorch OneCycleLR:  0.0002849076447010104
custom OneCycleLR:  0.0009791073640038343 pytorch OneCycleLR:  0.0002998231205353166
custom OneCycleLR:  0.0009777505480949925 pytorch OneCycleLR:  0.0003150129093441395
custom OneCycleLR:  0.0009763523075816903 pytorch OneCycleLR:  0.0003304580865421162
custom OneCycleLR:  0.0009749127957967566 pytorch OneCycleLR:  0.00034613940936175237
custom OneCycleLR:  0.0009734321705988807 pytorch OneCycleLR:  0.00036203734082751525
custom OneCycleLR:  0.0009719105943553006 pytorch OneCycleLR:  0.00037813207409647157
custom OneCycleLR:  0.000970348233923998 pytorch OneCycleLR:  0.00039440355713514844
custom OneCycleLR:  0.0009687452606354002 pytorch OneCycleLR:  0.00041083151770186885
custom OneCycleLR:  0.0009671018502735925 pytorch OneCycleLR:  0.00042739548860344146
custom OneCycleLR:  0.0009654181830570403 pytorch OneCycleLR:  0.00044407483319473405
custom OneCycleLR:  0.0009636944436188274 pytorch OneCycleLR:  0.00046084877108936394
custom OneCycleLR:  0.0009619308209864078 pytorch OneCycleLR:  0.0004776964040494711
custom OneCycleLR:  0.0009601275085608774 pytorch OneCycleLR:  0.0004945967420223218
custom OneCycleLR:  0.0009582847040957652 pytorch OneCycleLR:  0.0005115287292912982
custom OneCycleLR:  0.0009564026096753472 pytorch OneCycleLR:  0.0005284712707087017
custom OneCycleLR:  0.0009544814316924859 pytorch OneCycleLR:  0.0005454032579776784
custom OneCycleLR:  0.0009525213808259969 pytorch OneCycleLR:  0.0005623035959505287
custom OneCycleLR:  0.0009505226720175455 pytorch OneCycleLR:  0.0005791512289106361
custom OneCycleLR:  0.0009484855244480758 pytorch OneCycleLR:  0.000595925166805266
custom OneCycleLR:  0.0009464101615137755 pytorch OneCycleLR:  0.0006126045113965584
custom OneCycleLR:  0.0009442968108015775 pytorch OneCycleLR:  0.000629168482298131
custom OneCycleLR:  0.0009421457040642026 pytorch OneCycleLR:  0.0006455964428648515
custom OneCycleLR:  0.0009399570771947457 pytorch OneCycleLR:  0.0006618679259035283
custom OneCycleLR:  0.0009377311702008061 pytorch OneCycleLR:  0.0006779626591724849
custom OneCycleLR:  0.0009354682271781696 pytorch OneCycleLR:  0.0006938605906382474
custom OneCycleLR:  0.0009331684962840398 pytorch OneCycleLR:  0.0007095419134578837
custom OneCycleLR:  0.0009308322297098248 pytorch OneCycleLR:  0.0007249870906558605
custom OneCycleLR:  0.0009284596836534817 pytorch OneCycleLR:  0.0007401768794646835
custom OneCycleLR:  0.0009260511182914218 pytorch OneCycleLR:  0.0007550923552989896
custom OneCycleLR:  0.000923606797749979 pytorch OneCycleLR:  0.0007697149353332234
custom OneCycleLR:  0.0009211269900764458 pytorch OneCycleLR:  0.0007840264016535239
custom OneCycleLR:  0.0009186119672096785 pytorch OneCycleLR:  0.0007980089239549761
custom OneCycleLR:  0.0009160620049502761 pytorch OneCycleLR:  0.0008116450817559515
custom OneCycleLR:  0.000913477382930336 pytorch OneCycleLR:  0.0008249178861018585
custom OneCycleLR:  0.0009108583845827883 pytorch OneCycleLR:  0.0008378108007312642
custom OneCycleLR:  0.0009082052971103158 pytorch OneCycleLR:  0.000850307762678017
custom OneCycleLR:  0.0009055184114538569 pytorch OneCycleLR:  0.0008623932022837024
custom OneCycleLR:  0.0009027980222607026 pytorch OneCycleLR:  0.0008740520625954986
custom OneCycleLR:  0.0009000444278521839 pytorch OneCycleLR:  0.0008852698181252645
custom OneCycleLR:  0.0008972579301909578 pytorch OneCycleLR:  0.0008960324929464892
custom OneCycleLR:  0.0008944388348478938 pytorch OneCycleLR:  0.0009063266781065563
custom OneCycleLR:  0.0008915874509685647 pytorch OneCycleLR:  0.0009161395483326291
custom OneCycleLR:  0.0008887040912393449 pytorch OneCycleLR:  0.0009254588780103433
custom OneCycleLR:  0.0008857890718531214 pytorch OneCycleLR:  0.0009342730564154004
custom OneCycleLR:  0.000882842712474619 pytorch OneCycleLR:  0.0009425711021790838
custom OneCycleLR:  0.0008798653362053463 pytorch OneCycleLR:  0.0009503426769696777
custom OneCycleLR:  0.0008768572695481627 pytorch OneCycleLR:  0.00095757809837274
custom OneCycleLR:  0.0008738188423714755 pytorch OneCycleLR:  0.0009642683519541861
custom OneCycleLR:  0.0008707503878730643 pytorch OneCycleLR:  0.0009704051024911501
custom OneCycleLR:  0.0008676522425435433 pytorch OneCycleLR:  0.0009759807043566356
custom OneCycleLR:  0.0008645247461294607 pytorch OneCycleLR:  0.000980988211045014
custom OneCycleLR:  0.0008613682415960422 pytorch OneCycleLR:  0.0009854213838265064
custom OneCycleLR:  0.0008581830750895803 pytorch OneCycleLR:  0.0009892746995198629
custom OneCycleLR:  0.0008549695958994758 pytorch OneCycleLR:  0.00099254335737356
custom OneCycleLR:  0.0008517281564199351 pytorch OneCycleLR:  0.000995223285046938
custom OneCycleLR:  0.0008484591121113242 pytorch OneCycleLR:  0.0009973111436838314
custom OneCycleLR:  0.0008451628214611906 pytorch OneCycleLR:  0.0009988043320723666
custom OneCycleLR:  0.00084183964594495 pytorch OneCycleLR:  0.0009997009898857496
custom OneCycleLR:  0.0008384899499862462 pytorch OneCycleLR:  0.001
custom OneCycleLR:  0.0008351141009169893 pytorch OneCycleLR:  0.0009999440511289331
custom OneCycleLR:  0.0008317124689370715 pytorch OneCycleLR:  0.0009997762170368871
custom OneCycleLR:  0.0008282854270737727 pytorch OneCycleLR:  0.0009994965352845243
custom OneCycleLR:  0.0008248333511408524 pytorch OneCycleLR:  0.000999105068463608
custom OneCycleLR:  0.0008213566196973377 pytorch OneCycleLR:  0.0009986019041829956
custom OneCycleLR:  0.0008178556140060108 pytorch OneCycleLR:  0.0009979871550490316
custom OneCycleLR:  0.0008143307179915986 pytorch OneCycleLR:  0.000997260958640346
custom OneCycleLR:  0.0008107823181986711 pytorch OneCycleLR:  0.000996423477477066
custom OneCycleLR:  0.0008072108037492522 pytorch OneCycleLR:  0.0009954748989844438
custom OneCycleLR:  0.0008036165663001484 pytorch OneCycleLR:  0.0009944154354509117
custom OneCycleLR:  0.0007999999999999999 pytorch OneCycleLR:  0.0009932453239805729
custom OneCycleLR:  0.0007963615014460563 pytorch OneCycleLR:  0.0009919648264401376
custom OneCycleLR:  0.0007927014696406861 pytorch OneCycleLR:  0.0009905742294003194
custom OneCycleLR:  0.0007890203059476216 pytorch OneCycleLR:  0.0009890738440717015
custom OneCycleLR:  0.0007853184140479448 pytorch OneCycleLR:  0.0009874640062350875
custom OneCycleLR:  0.0007815961998958187 pytorch OneCycleLR:  0.0009857450761663572
custom OneCycleLR:  0.000777854071673971 pytorch OneCycleLR:  0.0009839174385558363
custom OneCycleLR:  0.000774092439748931 pytorch OneCycleLR:  0.0009819815024222052
custom OneCycleLR:  0.000770311716626029 pytorch OneCycleLR:  0.0009799377010209615
custom OneCycleLR:  0.0007665123169041606 pytorch OneCycleLR:  0.0009777864917474587
custom OneCycleLR:  0.00076269465723032 pytorch OneCycleLR:  0.0009755283560345441
custom OneCycleLR:  0.0007588591562539122 pytorch OneCycleLR:  0.0009731637992448144
custom OneCycleLR:  0.0007550062345808412 pytorch OneCycleLR:  0.0009706933505575182
custom OneCycleLR:  0.0007511363147273869 pytorch OneCycleLR:  0.0009681175628501272
custom OneCycleLR:  0.0007472498210738712 pytorch OneCycleLR:  0.0009654370125746047
custom OneCycleLR:  0.00074334717981812 pytorch OneCycleLR:  0.0009626522996283973
custom OneCycleLR:  0.0007394288189287261 pytorch OneCycleLR:  0.0009597640472201802
custom OneCycleLR:  0.0007354951680981166 pytorch OneCycleLR:  0.000956772901730385
custom OneCycleLR:  0.0007315466586954334 pytorch OneCycleLR:  0.0009536795325665432
custom OneCycleLR:  0.000727583723719228 pytorch OneCycleLR:  0.0009504846320134736
custom OneCycleLR:  0.0007236067977499789 pytorch OneCycleLR:  0.000947188915078353
custom OneCycleLR:  0.0007196163169024347 pytorch OneCycleLR:  0.000943793119330699
custom OneCycleLR:  0.0007156127187777885 pytorch OneCycleLR:  0.0009402980047373052
custom OneCycleLR:  0.0007115964424156918 pytorch OneCycleLR:  0.0009367043534921636
custom OneCycleLR:  0.0007075679282461063 pytorch OneCycleLR:  0.0009330129698414117
custom OneCycleLR:  0.0007035276180410084 pytorch OneCycleLR:  0.0009292246799033457
custom OneCycleLR:  0.0006994759548659419 pytorch OneCycleLR:  0.0009253403314835384
custom OneCycleLR:  0.0006954133830314323 pytorch OneCycleLR:  0.0009213607938851022
custom OneCycleLR:  0.0006913403480442624 pytorch OneCycleLR:  0.0009172869577141438
custom OneCycleLR:  0.000687257296558617 pytorch OneCycleLR:  0.0009131197346804487
custom OneCycleLR:  0.0006831646763271038 pytorch OneCycleLR:  0.0009088600573934443
custom OneCycleLR:  0.0006790629361516505 pytorch OneCycleLR:  0.0009045088791534849
custom OneCycleLR:  0.0006749525258342899 pytorch OneCycleLR:  0.0009000671737385071
custom OneCycleLR:  0.0006708338961278333 pytorch OneCycleLR:  0.0008955359351861013
custom OneCycleLR:  0.000666707498686441 pytorch OneCycleLR:  0.0008909161775710499
custom OneCycleLR:  0.0006625737860160923 pytorch OneCycleLR:  0.0008862089347783812
custom OneCycleLR:  0.0006584332114249647 pytorch OneCycleLR:  0.0008814152602719894
custom OneCycleLR:  0.0006542862289737217 pytorch OneCycleLR:  0.0008765362268588734
custom OneCycleLR:  0.0006501332934257217 pytorch OneCycleLR:  0.0008715729264490461
custom OneCycleLR:  0.0006459748601971467 pytorch OneCycleLR:  0.0008665264698111694
custom OneCycleLR:  0.0006418113853070615 pytorch OneCycleLR:  0.000861397986323968
custom OneCycleLR:  0.0006376433253274059 pytorch OneCycleLR:  0.0008561886237234785
custom OneCycleLR:  0.0006334711373329262 pytorch OneCycleLR:  0.00085089954784619
custom OneCycleLR:  0.0006292952788510527 pytorch OneCycleLR:  0.0008455319423681343
custom OneCycleLR:  0.0006251162078117254 pytorch OneCycleLR:  0.000840087008539984
custom OneCycleLR:  0.0006209343824971776 pytorch OneCycleLR:  0.0008345659649182163
custom OneCycleLR:  0.0006167502614916799 pytorch OneCycleLR:  0.0008289700470924044
custom OneCycleLR:  0.0006125643036312512 pytorch OneCycleLR:  0.0008233005074086972
custom OneCycleLR:  0.0006083769679533429 pytorch OneCycleLR:  0.0008175586146895496
custom OneCycleLR:  0.0006041887136464983 pytorch OneCycleLR:  0.000811745653949763
custom OneCycleLR:  0.0006000000000000001 pytorch OneCycleLR:  0.0008058629261089048
custom OneCycleLR:  0.0005958112863535017 pytorch OneCycleLR:  0.0007999117477001675
custom OneCycleLR:  0.0005916230320466573 pytorch OneCycleLR:  0.0007938934505757319
custom OneCycleLR:  0.0005874356963687486 pytorch OneCycleLR:  0.0007878093816087053
custom OneCycleLR:  0.0005832497385083202 pytorch OneCycleLR:  0.0007816609023916948
custom OneCycleLR:  0.0005790656175028224 pytorch OneCycleLR:  0.0007754493889320882
custom OneCycleLR:  0.0005748837921882747 pytorch OneCycleLR:  0.0007691762313441089
custom OneCycleLR:  0.0005707047211489473 pytorch OneCycleLR:  0.0007628428335377126
custom OneCycleLR:  0.0005665288626670737 pytorch OneCycleLR:  0.0007564506129043982
custom OneCycleLR:  0.0005623566746725943 pytorch OneCycleLR:  0.0007500009999999999
custom OneCycleLR:  0.0005581886146929387 pytorch OneCycleLR:  0.0007434954382245347
custom OneCycleLR:  0.0005540251398028534 pytorch OneCycleLR:  0.0007369353834991743
custom OneCycleLR:  0.0005498667065742782 pytorch OneCycleLR:  0.0007303223039404152
custom OneCycleLR:  0.0005457137710262783 pytorch OneCycleLR:  0.0007236576795315195
custom OneCycleLR:  0.0005415667885750355 pytorch OneCycleLR:  0.0007169430017913008
custom OneCycleLR:  0.0005374262139839078 pytorch OneCycleLR:  0.0007101797734403261
custom OneCycleLR:  0.000533292501313559 pytorch OneCycleLR:  0.000703369508064614
custom OneCycleLR:  0.0005291661038721667 pytorch OneCycleLR:  0.0006965137297768985
custom OneCycleLR:  0.0005250474741657101 pytorch OneCycleLR:  0.0006896139728755389
custom OneCycleLR:  0.0005209370638483495 pytorch OneCycleLR:  0.0006826717815011488
custom OneCycleLR:  0.0005168353236728962 pytorch OneCycleLR:  0.0006756887092910232
custom OneCycleLR:  0.000512742703441383 pytorch OneCycleLR:  0.0006686663190314402
custom OneCycleLR:  0.0005086596519557378 pytorch OneCycleLR:  0.0006616061823079151
custom OneCycleLR:  0.0005045866169685678 pytorch OneCycleLR:  0.0006545098791534849
custom OneCycleLR:  0.0005005240451340582 pytorch OneCycleLR:  0.0006473789976951033
custom OneCycleLR:  0.0004964723819589916 pytorch OneCycleLR:  0.0006402151337982227
custom OneCycleLR:  0.0004924320717538938 pytorch OneCycleLR:  0.0006330198907096463
custom OneCycleLR:  0.0004884035575843083 pytorch OneCycleLR:  0.0006257948786987268
custom OneCycleLR:  0.00048438728122221134 pytorch OneCycleLR:  0.0006185417146969932
custom OneCycleLR:  0.0004803836830975653 pytorch OneCycleLR:  0.0006112620219362892
custom OneCycleLR:  0.000476393202250021 pytorch OneCycleLR:  0.0006039574295854978
custom OneCycleLR:  0.0004724162762807721 pytorch OneCycleLR:  0.0005966295723859409
custom OneCycleLR:  0.0004684533413045667 pytorch OneCycleLR:  0.0005892800902855287
custom OneCycleLR:  0.0004645048319018834 pytorch OneCycleLR:  0.0005819106280717459
custom OneCycleLR:  0.0004605711810712739 pytorch OneCycleLR:  0.0005745228350035547
custom OneCycleLR:  0.0004566528201818799 pytorch OneCycleLR:  0.000567118364442296
custom OneCycleLR:  0.00045275017892612893 pytorch OneCycleLR:  0.0005596988734816729
custom OneCycleLR:  0.0004488636852726131 pytorch OneCycleLR:  0.0005522660225769001
custom OneCycleLR:  0.0004449937654191588 pytorch OneCycleLR:  0.0005448214751730989
custom OneCycleLR:  0.00044114084374608767 pytorch OneCycleLR:  0.0005373668973330249
custom OneCycleLR:  0.0004373053427696799 pytorch OneCycleLR:  0.0005299039573642091
custom OneCycleLR:  0.00043348768309583953 pytorch OneCycleLR:  0.0005224343254455967
custom OneCycleLR:  0.00042968828337397095 pytorch OneCycleLR:  0.0005149596732537669
custom OneCycleLR:  0.000425907560251069 pytorch OneCycleLR:  0.0005074816735888177
custom OneCycleLR:  0.0004221459283260291 pytorch OneCycleLR:  0.0005000019999999999
custom OneCycleLR:  0.00041840380010418134 pytorch OneCycleLR:  0.0004925223264111823
custom OneCycleLR:  0.0004146815859520554 pytorch OneCycleLR:  0.0004850443267462332
custom OneCycleLR:  0.00041097969405237846 pytorch OneCycleLR:  0.0004775696745544034
custom OneCycleLR:  0.00040729853035931387 pytorch OneCycleLR:  0.0004701000426357908
custom OneCycleLR:  0.0004036384985539436 pytorch OneCycleLR:  0.00046263710266697503
custom OneCycleLR:  0.0003999999999999999 pytorch OneCycleLR:  0.0004551825248269011
custom OneCycleLR:  0.0003963834336998514 pytorch OneCycleLR:  0.0004477379774230999
custom OneCycleLR:  0.00039278919625074796 pytorch OneCycleLR:  0.000440305126518327
custom OneCycleLR:  0.000389217681801329 pytorch OneCycleLR:  0.00043288563555770405
custom OneCycleLR:  0.00038566928200840133 pytorch OneCycleLR:  0.00042548116499644507
custom OneCycleLR:  0.00038214438599398927 pytorch OneCycleLR:  0.00041809337192825395
custom OneCycleLR:  0.0003786433803026623 pytorch OneCycleLR:  0.0004107239097144713
custom OneCycleLR:  0.00037516664885914785 pytorch OneCycleLR:  0.00040337442761405903
custom OneCycleLR:  0.00037171457292622733 pytorch OneCycleLR:  0.00039604657041450203
custom OneCycleLR:  0.00036828753106292835 pytorch OneCycleLR:  0.00038874197806371076
custom OneCycleLR:  0.00036488589908301085 pytorch OneCycleLR:  0.00038146228530300675
custom OneCycleLR:  0.0003615100500137536 pytorch OneCycleLR:  0.00037420912130127325
custom OneCycleLR:  0.00035816035405505 pytorch OneCycleLR:  0.0003669841092903536
custom OneCycleLR:  0.0003548371785388095 pytorch OneCycleLR:  0.0003597888662017773
custom OneCycleLR:  0.0003515408878886759 pytorch OneCycleLR:  0.0003526250023048967
custom OneCycleLR:  0.00034827184358006505 pytorch OneCycleLR:  0.0003454941208465151
custom OneCycleLR:  0.0003450304041005241 pytorch OneCycleLR:  0.000338397817692085
custom OneCycleLR:  0.00034181692491041984 pytorch OneCycleLR:  0.00033133768096855977
custom OneCycleLR:  0.0003386317584039579 pytorch OneCycleLR:  0.00032431529070897674
custom OneCycleLR:  0.00033547525387053913 pytorch OneCycleLR:  0.0003173322184988512
custom OneCycleLR:  0.00033234775745645664 pytorch OneCycleLR:  0.0003103900271244611
custom OneCycleLR:  0.0003292496121269357 pytorch OneCycleLR:  0.00030349027022310155
custom OneCycleLR:  0.00032618115762852453 pytorch OneCycleLR:  0.00029663449193538614
custom OneCycleLR:  0.0003231427304518373 pytorch OneCycleLR:  0.00028982422655967396
custom OneCycleLR:  0.0003201346637946537 pytorch OneCycleLR:  0.0002830609982086992
custom OneCycleLR:  0.000317157287525381 pytorch OneCycleLR:  0.00027634632046848027
custom OneCycleLR:  0.0003142109281468787 pytorch OneCycleLR:  0.00026968169605958483
custom OneCycleLR:  0.00031129590876065506 pytorch OneCycleLR:  0.00026306861650082563
custom OneCycleLR:  0.00030841254903143543 pytorch OneCycleLR:  0.00025650856177546514
custom OneCycleLR:  0.0003055611651521063 pytorch OneCycleLR:  0.0002500030000000001
custom OneCycleLR:  0.00030274206980904217 pytorch OneCycleLR:  0.00024355338709560175
custom OneCycleLR:  0.00029995557214781613 pytorch OneCycleLR:  0.00023716116646228723
custom OneCycleLR:  0.0002972019777392974 pytorch OneCycleLR:  0.00023082776865589112
custom OneCycleLR:  0.00029448158854614306 pytorch OneCycleLR:  0.0002245546110679117
custom OneCycleLR:  0.00029179470288968434 pytorch OneCycleLR:  0.00021834309760830508
custom OneCycleLR:  0.00028914161541721163 pytorch OneCycleLR:  0.00021219461839129468
custom OneCycleLR:  0.00028652261706966397 pytorch OneCycleLR:  0.00020611054942426806
custom OneCycleLR:  0.0002839379950497238 pytorch OneCycleLR:  0.00020009225229983252
custom OneCycleLR:  0.00028138803279032144 pytorch OneCycleLR:  0.00019414107389109498
custom OneCycleLR:  0.00027887300992355423 pytorch OneCycleLR:  0.00018825834605023698
custom OneCycleLR:  0.000276393202250021 pytorch OneCycleLR:  0.0001824453853104503
custom OneCycleLR:  0.00027394888170857826 pytorch OneCycleLR:  0.0001767034925913027
custom OneCycleLR:  0.00027154031634651834 pytorch OneCycleLR:  0.0001710339529075956
custom OneCycleLR:  0.00026916777029017513 pytorch OneCycleLR:  0.00016543803508178376
custom OneCycleLR:  0.0002668315037159602 pytorch OneCycleLR:  0.00015991699146001575
custom OneCycleLR:  0.0002645317728218304 pytorch OneCycleLR:  0.00015447205763186565
custom OneCycleLR:  0.000262268829799194 pytorch OneCycleLR:  0.00014910445215381003
custom OneCycleLR:  0.00026004292280525445 pytorch OneCycleLR:  0.0001438153762765216
custom OneCycleLR:  0.00025785429593579726 pytorch OneCycleLR:  0.00013860601367603193
custom OneCycleLR:  0.00025570318919842253 pytorch OneCycleLR:  0.0001334775301888306
custom OneCycleLR:  0.00025358983848622465 pytorch OneCycleLR:  0.00012843107355095377
custom OneCycleLR:  0.00025151447555192406 pytorch OneCycleLR:  0.00012346777314112658
custom OneCycleLR:  0.0002494773279824546 pytorch OneCycleLR:  0.00011858873972801045
custom OneCycleLR:  0.00024747861917400317 pytorch OneCycleLR:  0.00011379506522161873
custom OneCycleLR:  0.0002455185683075141 pytorch OneCycleLR:  0.00010908782242895002
custom OneCycleLR:  0.00024359739032465288 pytorch OneCycleLR:  0.00010446806481389881
custom OneCycleLR:  0.0002417152959042349 pytorch OneCycleLR:  9.993682626149293e-05
custom OneCycleLR:  0.0002398724914391225 pytorch OneCycleLR:  9.549512084651507e-05
custom OneCycleLR:  0.00023806917901359227 pytorch OneCycleLR:  9.114394260655564e-05
custom OneCycleLR:  0.00023630555638117247 pytorch OneCycleLR:  8.688426531955129e-05
custom OneCycleLR:  0.00023458181694295966 pytorch OneCycleLR:  8.271704228585612e-05
custom OneCycleLR:  0.00023289814972640756 pytorch OneCycleLR:  7.864320611489772e-05
custom OneCycleLR:  0.00023125473936459973 pytorch OneCycleLR:  7.466366851646164e-05
custom OneCycleLR:  0.00022965176607600202 pytorch OneCycleLR:  7.077932009665415e-05
custom OneCycleLR:  0.00022808940564469937 pytorch OneCycleLR:  6.69910301585882e-05
custom OneCycleLR:  0.00022656782940111932 pytorch OneCycleLR:  6.32996465078364e-05
custom OneCycleLR:  0.00022508720420324335 pytorch OneCycleLR:  5.970599526269471e-05
custom OneCycleLR:  0.00022364769241830972 pytorch OneCycleLR:  5.6210880669300965e-05
custom OneCycleLR:  0.00022224945190500755 pytorch OneCycleLR:  5.2815084921647e-05
custom OneCycleLR:  0.0002208926359961657 pytorch OneCycleLR:  4.9519367986526286e-05
custom OneCycleLR:  0.0002195773934819385 pytorch OneCycleLR:  4.632446743345669e-05
custom OneCycleLR:  0.00021830386859348928 pytorch OneCycleLR:  4.3231098269614797e-05
custom OneCycleLR:  0.00021707220098717307 pytorch OneCycleLR:  4.0239952779819815e-05
custom OneCycleLR:  0.00021588252572922275 pytorch OneCycleLR:  3.73517003716026e-05
custom OneCycleLR:  0.00021473497328093673 pytorch OneCycleLR:  3.456698742539522e-05
custom OneCycleLR:  0.00021362966948437268 pytorch OneCycleLR:  3.1886437149872676e-05
custom OneCycleLR:  0.00021256673554854755 pytorch OneCycleLR:  2.9310649442481776e-05
custom OneCycleLR:  0.0002115462880361455 pytorch OneCycleLR:  2.6840200755185466e-05
custom OneCycleLR:  0.00021056843885073584 pytorch OneCycleLR:  2.447564396545582e-05
custom OneCycleLR:  0.00020963329522450102 pytorch OneCycleLR:  2.2217508252541176e-05
custom OneCycleLR:  0.00020874095970647766 pytorch OneCycleLR:  2.006629897903861e-05
custom OneCycleLR:  0.00020789153015131134 pytorch OneCycleLR:  1.8022497577794767e-05
custom OneCycleLR:  0.0002070850997085245 pytorch OneCycleLR:  1.608656144416369e-05
custom OneCycleLR:  0.00020632175681230326 pytorch OneCycleLR:  1.4258923833642831e-05
custom OneCycleLR:  0.00020560158517179795 pytorch OneCycleLR:  1.2539993764912554e-05
custom OneCycleLR:  0.00020492466376194484 pytorch OneCycleLR:  1.0930155928298623e-05
custom OneCycleLR:  0.00020429106681480458 pytorch OneCycleLR:  9.429770599680518e-06
custom OneCycleLR:  0.0002037008638114225 pytorch OneCycleLR:  8.039173559862363e-06
custom OneCycleLR:  0.0002031541194742088 pytorch OneCycleLR:  6.758676019427176e-06
custom OneCycleLR:  0.00020265089375984147 pytorch OneCycleLR:  5.588564549088189e-06
custom OneCycleLR:  0.00020219124185269065 pytorch OneCycleLR:  4.529101015556186e-06
custom OneCycleLR:  0.000201775214158768 pytorch OneCycleLR:  3.580522522934006e-06
custom OneCycleLR:  0.00020140285630019817 pytorch OneCycleLR:  2.7430413596540916e-06
custom OneCycleLR:  0.00020107420911021678 pytorch OneCycleLR:  2.016844950968456e-06
custom OneCycleLR:  0.0002007893086286914 pytorch OneCycleLR:  1.4020958170042977e-06
custom OneCycleLR:  0.0002005481860981705 pytorch OneCycleLR:  8.989315363919581e-07
custom OneCycleLR:  0.00020035086796045665 pytorch OneCycleLR:  5.074647154757879e-07
custom OneCycleLR:  0.00020019737585370734 pytorch OneCycleLR:  2.277829631129057e-07
custom OneCycleLR:  0.0002000877266100618 pytorch OneCycleLR:  5.994887106690402e-08
custom OneCycleLR:  0.00020002193225379506 pytorch OneCycleLR:  4e-09
custom OneCycleLR:  0.00019999999999999996 pytorch OneCycleLR:  5.994887106690402e-08

Python 中的 torch.optim.lr_scheduler 模块
Z2572862506的博客
01-31 1594
Python 中的 torch.optim.lr_scheduler 模块
torch.optim.lr_scheduler--学习率调整总结
weixin_44543648的博客
01-21 6309
参考链接：https://blog.csdn.net/qyhaill/article/details/103043637 调用例子： import torch import torch.nn as nn from torch.optim.lr_scheduler import LambdaLR initial_lr = 5 class model(nn.Module): def __init__(self): super().__init__() self.con
torch.optim.lr_scheduler.LambdaLR（ ）函数的用法
Kelly_Ai_Bai的博客
12-19 1076
torch.optim.lr_scheduler.LambdaLR()函数的用法
torch.optim.lr_scheduler.LambdaLR 的用法
weixin_39800806的博客
07-10 700
lr在第100个到第400个epoch中，指数衰减.
class torch.optim.lr_scheduler.LambdaLR
敲代码的小风
04-26 2302
参考链接: class torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda, last_epoch=-1, verbose=False)
Pytorch lr_scheduler.LambdaLR()的简单理解与用法
xiongxyowo的博客
08-11 1万+
在python中，有个东西叫做匿名函数(lambda表达式)，能够用于很方便的定义各种规则，这个LambdaLR也就可以理解成自定义规则去调整网络的学习率。其次，last_epoch值修改为已完成的epoch数。理解LambdaLR的核心在于，自定义规则函数只有一个参数，即当前的epoch，这个参数是scheduler自己传进去的。开始(因为外面的超参last_epoch-1表示-1已结束，因此从0开始)，每step一次加1；，比如我们想每3个epoch(即在第2，5，8个epoch。...
torch.optim.lr_scheduler：调整学习率之CosineAnnealingWarmRestarts()参数说明
qq_37612828的博客
08-25 1万+
参考博主关于torch学习率调整的文章，写的非常好。https://blog.csdn.net/qyhaill/article/details/103043637，但是CosineAnnealingWarmRestarts()学习率调整博主没有写，在此借用博主的代码，补充一下此函数的学习率调整曲线。对于此函数，官方解释如下：链接地址https://pytorch.org/docs/stable/optim.html 函数名跟CosinAnnealingLR()相近，参数有两个不一样，T_0就是初始re
Pytorch笔记——学习率的调整torch.optim.lr_scheduler
神仙院B栋4楼保安的博客
09-19 1128
参考资料：官方文档 在训练网络的时候需要根据不同的训练阶段对网络的学习率进行调整，以得到较好的训练效果，有时候会觉得手动调整学习率的过程比较枯燥繁重，可以考虑尝试一下pytorch中的orch.optim.lr_scheduler的模块对学习率进行调整。这里介绍几个个人目前用的比较多的。 ReduceLROnPlateau 这是我目前用的最多的学习率调度器。这一调度器的作用是在某一指标没有被改善的时候降低学习率。 torch.optim.lr_scheduler.ReduceLROnPlateau(opti
pytorch torch.optim.lr_scheduler 调整学习率的六种策略
baoxin1100的博客
07-19 4127
pytorch torch.optim.lr_scheduler 调整学习率的六种策略 1. 为什么需要调整学习率 在深度学习训练过程中，最重要的参数就是学习率，通常来说，在整个训练过层中，学习率不会一直保持不变，为了让模型能够在训练初期快速收敛，学习率通常比较大，在训练末期，为了让模型收敛在更小的局部最优点，学习率通常要比较小。 2. 学习率的初始值设置 其实，不同的任务学习率的初始值是需要试验几次来获得的，使用的优化器不同，mini-batch 的 batch_size 大小不同，学习率的初始值也不太相
学习率调度器，torch.optim.lr_scheduler.LambdaLR（）
qq_40397392的博客
04-01 175
optimizer.param_groups[0]["lr"] #优化器目前的学习率。
torch.optim.lr_scheduler.OneCycleLR 学习与理解
jiongta9473的博客
06-16 9238
torch.optim.lr_scheduler.OneCycleLR 详解
torch.optim.lr_scheduler.OneCycleLR
m0_49133355的博客
07-17 819
torch.optim.lr_scheduler.OneCycleLR 是 PyTorch 中的一个学习率调度器，它根据 “1cycle” 策略在训练期间调整学习率，可以提高训练性能和收敛性。在这里，它设置为 0.4，表示学习率将在前 40% 的训练步骤中增加，后 60% 的训练步骤中减少。在这种情况下，它设置为 init_lr*10，其中 init_lr 是提供给优化器的初始学习率。在这里，它设置为 10。这个学习率调度器将在训练过程中根据指定的参数动态调整学习率和动量，从而帮助模型更好地收敛。
【笔记】pytorch 中学习率调整函数（手动定义一个学习率衰减函数） ： torch.optim.lr_scheduler ...
nyist_yangguang的博客
10-12 598
注1： 注2： 注3： 正文： import torch import torch.nn as nn from torch.optim.lr_scheduler import LambdaLR import itertools initial_lr = 0.1 class model(nn.Module): def __init__(self): super().__init__() self.conv1...
pytorch-lr_scheduler.LambdaLR函数，更新学习率的管理工具
xingghaoyuxitong的博客
08-27 5431
理论： LambdaLR更新学习率方式是 lr = lr*lr_lambda 其中，lr由optim系列优化器提供，lr_lambda由lr_scheduler>lambdaLR提供 假设，lr初始值为0.4， 更新学习率函数lambda表达式为：lr_lambda = lambda epoch:0.1*epoch) epoch的初始值为0 则，lr的变化规律应该是 0.4*0.1*0=0 0.4*0.1*1=0.04 0.4*0.1*2=0.08 ...... 实验： i.
torch.optim.lr_scheduler.OneCycleLR用法
so
01-16 1万+
代码： import cv2 import torch.nn as nn import torch from torchvision.models import AlexNet import matplotlib.pyplot as plt #定义2分类网络 steps = [] lrs = [] model = AlexNet(num_classes=2) lr = 0.9 optimizer = torch.optim.SGD(model.parameters(), lr=lr, momentum=
class torch.optim.lr_scheduler.ReduceLROnPlateau
敲代码的小风
04-26 984
参考链接: class torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode=‘min’, factor=0.1, patience=10, threshold=0.0001, threshold_mode=‘rel’, cooldown=0, min_lr=0, eps=1e-08, verbose=False)
pytorch笔记：调整学习率（torch.optim.lr_scheduler）
qq_40206371的博客
08-25 1万+
1 概述 torch.optim.lr_scheduler模块提供了一些根据epoch训练次数来调整学习率（learning rate）的方法。一般情况下我们会设置随着epoch的增大而逐渐减小学习率从而达到更好的训练效果。 2lr_scheduler调整策略举例 2.1torch.optim.lr_scheduler.LambdaLR torch.optim.lr_scheduler.LambdaLR( optimizer, lr_lambda, last_ep...
PyTorch torch.optim.lr_scheduler 学习率 - LambdaLR;StepLR;MultiStepLR;ExponentialLR
热门推荐
zisuina_2的博客
11-26 2万+
PyTorch学习率调整策略通过torch.optim.lr_scheduler接口实现。PyTorch提供的学习率调整策略分为三大类，分别是 有序调整：等间隔调整(Step)，按需调整学习率(MultiStep)，指数衰减调整(Exponential)和余弦退火CosineAnnealing。 自适应调整：自适应调整学习率 ReduceLROnPlateau。 自定义调整：自定义调整学习率 L...
torch.optim.lr_scheduler替换
最新发布
03-08
### 替换 PyTorch 中的学习率调度器 在 PyTorch 中，除了 `torch.optim.lr_scheduler` 提供的标准学习率调度器外，还可以通过自定义函数或使用其他库来实现学习率调整。 #### 方法一：手动修改学习率 可以直接访问优化器的参数组并更新学习率： ```python for param_group in optimizer.param_groups: param_group['lr'] = new_learning_rate ``` 这种方法提供了最大的灵活性，允许在训练循环内的任何位置动态更改学习率[^1]。 #### 方法二：使用 LambdaLR 调度器 LambdaLR 是一种灵活的方式，可以通过定义 lambda 函数来自定义学习率的变化规律。例如线性衰减： ```python lambda1 = lambda epoch: 0.95 ** epoch scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda1) ``` 这使得可以轻松创建复杂的学习率变化模式而无需编写额外的逻辑代码[^2]。 #### 方法三：CosineAnnealingLR 和 CosineAnnealingWarmRestarts 这两种调度策略模仿余弦波形改变学习率，在某些情况下能带来更好的泛化性能: ```python # 单周期余弦退火 scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=10) # 多重重启版本 scheduler = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10) ``` 这些方法特别适合于长时间训练的任务，因为它们可以在多个时期内平滑地降低学习率. #### 自定义调度器 如果上述选项都不能满足需求，则可以选择继承 `_LRScheduler` 类构建自己的调度算法。这种方式最具有定制性和适应特殊场景的能力。 ```python from torch.optim.lr_scheduler import _LRScheduler class CustomScheduler(_LRScheduler): def __init__(self, optimizer, last_epoch=-1): super(CustomScheduler, self).__init__(optimizer, last_epoch) def get_lr(self): # 定义新的学习率计算方式 pass ``` 这种做法适用于那些希望完全控制学习率调整过程的研究人员和技术专家.
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值