Pytorch训练时重要步骤

目录

optimizer.zero_grad()

代码解读

调用频率和结果

loss.backward()

optimizer.step()

---

在学习pytorch时，官方文档 有一段示例代码

def train_loop(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    for batch, (X, y) in enumerate(dataloader):
        # Compute prediction and loss
        pred = model(X)
        loss = loss_fn(pred, y)

        # Backpropagation
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if batch % 100 == 0:
            loss, current = loss.item(), batch * len(X)
            print(f"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]")

有三个关键步骤：

1. 梯度归零

2. 反向传播，得到每个参数的梯度

3. 通过梯度下降执行一步参数更新

下面详细了解一下这几个步骤

optimizer.zero_grad()