backward-pytroch

最新推荐文章于 2024-10-17 12:55:45 发布

Sag_ittarius

最新推荐文章于 2024-10-17 12:55:45 发布

阅读量194

点赞数

分类专栏： Pytorch

Pytorch 专栏收录该内容

10 篇文章

订阅专栏

本文详细介绍了在PyTorch框架下如何实现多任务学习中的梯度累积过程。通过一个实例，展示了如何在一个特征提取器的基础上进行两个不同任务的学习，并将每个任务的损失函数梯度累加到共享的特征提取器参数上。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

input = Variable(data)
# Get the features
features = feature_extractor(input)

# Compute first loss and get the gradients for it
loss1 = task1(features)
loss1.backward(retain_graph=True)
# This add the gradients wrt loss1 in both the "task1" net and the "feature_extractor" net
# So each parameter "w" in "feature_extractor" has it gradient d(loss1)/dw

# Perform the second task and get the gradients for it as well
loss2 = task2(features)
loss2.backward()
# This will add gradients in "task2" and accumulate in "feature_extractor"
# Now each parameter in "feature_extractor" contains d(loss1)/dw + d(loss2)/dw

Reference:

https://discuss.pytorch.org/t/why-do-we-need-to-set-the-gradients-manually-to-zero-in-pytorch/4903/8