分布式训练Warning: Grad strides do not match bucket view strides. This may indicate grad was not

最新推荐文章于 2024-11-22 09:08:36 发布

xiangyong58

最新推荐文章于 2024-11-22 09:08:36 发布

阅读量3.2k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： Machine & Deep Learning 文章标签：人工智能计算机视觉

本文链接：https://blog.youkuaiyun.com/xiangyong58/article/details/132158277

Machine & Deep Learning 专栏收录该内容

78 篇文章 ¥9.90 ¥99.00

订阅专栏

超级会员免费看

本文探讨了在分布式训练过程中遇到的'Grad strides do not match bucket view strides'警告，指出该警告通常由数据转换引起。提供两个示例，eg1说明transpose或permute后进行reshape操作时，应使用.contiguous()，eg2则强调rearrange操作后同样需要加上.contiguous()来避免警告。

warning信息如下：

Warning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed.  This is not an error, but may impair performance.