解决PyTorch DDP: Finding the cause of “Expected to mark a variable ready only once“

早上做消融实验的时候需要复现俩月前的实验结果，但是莫名其妙同样的代码和环境却跑不通了，会在loss.backward()的时候报如下错误：
RuntimeError: Expected to mark a variable ready only once. This error is caused by one of the following reasons: 1) Use of a module parameter outside the ``forward`` function. Please make sure model parameters are not shared across multiple concurrent forward-backward passes. 2) Reused parameters in multiple reentrant backward passes. For example, if you use multiple ``checkpoint`` functions to wrap the same part of your model, it would result in the same set of parameters been used by different reentrant backward passes multiple times, and hence marking a variable ready multiple times. DDP does not support such use cases yet.

在网上找了一圈发现一共也没几个人问过这个报错，其中stackoverflow上有人解决了这问题，说是把find_unused_parameters设置为false就莫名其妙好了，但是我这么设置之后在固定D训练G的时候又报错：之前写代码时碰到了这样一个错误：
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your mod