ValueError: Caught ValueError in DataLoader worker process 0.

最新推荐文章于 2025-01-08 19:52:50 发布

砸吧砸吧

最新推荐文章于 2025-01-08 19:52:50 发布

阅读量7.8k

点赞数 2

文章标签： pytorch 深度学习 python

本文链接：https://blog.youkuaiyun.com/m0_49629753/article/details/120678129

版权

这篇博客主要讨论了在使用PyTorch训练模型时遇到的数据加载错误，特别是ValueError，由于尝试加载包含pickle数据的.npy文件而引发。作者首先尝试修改`allow_pickle`参数，然后详细描述了错误的演变，最终确定问题源于数据集本身。解决方法强调了在复现代码时确保正确数据集路径的重要性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Traceback (most recent call last):
  File "G:/wjy/Eye_VR_Segmentation-master/src/test.py", line 258, in <module>
    print('Val set: ', utils.eval(loaders['val'], model, utils.generalised_dice_loss_ce))
  File "G:\wjy\Eye_VR_Segmentation-master\src\utils.py", line 241, in eval
    for i, sample_batched in enumerate(loader):
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\dataloader.py", line 345, in __next__
    data = self._next_data()
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\dataloader.py", line 856, in _next_data
    return self._process_data(data)
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\dataloader.py", line 881, in _process_data
    data.reraise()
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\_utils.py", line 394, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\_utils\worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "G:\wjy\Eye_VR_Segmentation-master\src\utils.py", line 126, in __getitem__
    npy = np.load(npy_path, allow_pickle=False)
  File "C:\Users\06\.conda\envs\vr-torch\lib\site-packages\numpy\lib\npyio.py", line 445, in load
    raise ValueError("Cannot load file containing pickled data "
ValueError: Cannot load file containing pickled data when allow_pickle=False

最后的错误为：ValueError: Cannot load file containing pickled data when allow_pickle=False

出错代码为：

npy = np.load(npy_path, allow_pickle=False)

首先尝试将allow_pickle=False的False改为True

然后再次报错Failed to interpret file 'XXXXX' as a pickle

最后发现其实还是数据集的问题，复现代码的时候需要关注数据集地址。