IndexError: Caught IndexError in DataLoader worker process 0.

数据加载错误与解决方案
博客内容涉及一个常见的编程错误,即在训练模型时数据集加载出现问题,导致数据不成对。错误的根本原因是数据集的图像数量与对应的目标(ground_truth)不匹配。为了解决这个问题,建议重新导入数据集,确保图像与它们对应的标签数量一致。这有助于避免因数据不匹配而引发的错误,从而能够正常进行模型训练。

在这里插入图片描述
问题分析:
该错误形成的原因是数据集加载错误,导致数据不是成对的。比如data有1000幅图像,而target(即ground_truth)有1200幅图像,由于1000不等于1200所以出现上述错误。

解决方法:
重新导入数据集,确保数据成对儿。

Traceback (most recent call last): File "main_pretrain.py", line 234, in <module> main(args) File "main_pretrain.py", line 204, in main train_stats = train_one_epoch( File "/2024110148/mymae/mae/engine_pretrain.py", line 43, in train_one_epoch for alpha_data,_ in alpha_loader: File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__ data = self._next_data() File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data return self._process_data(data) File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data data.reraise() File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in DataLoader worker process 0. Original Traceback (most recent call last): File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop data = fetcher.fetch(index) File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/envs/mae/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/2024110148/mymae/mae/util/alpha_image_loader.py", line 19, in __getitem__ raise IndexError(f"idx={idx} 超过了图片数量 {len(self.image_paths)}") IndexError: idx=1668 超过了图片数量 100
05-14
File "train_vit.py", line 204, in <module> main(args) File "train_vit.py", line 99, in main test(student, test_loader, epoch, args) File "train_vit.py", line 150, in test for episode in test_loader: File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in __next__ data = self._next_data() File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data return self._process_data(data) File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data data.reraise() File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise raise exception IndexError: Caught IndexError in DataLoader worker process 0. Original Traceback (most recent call last): File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/public/home/wl_ligexian/SemanticPrompt-main/data/dataset.py", line 103, in __getitem__ image, label = self.dataset[i] File "/public/home/wl_ligexian/anaconda3/envs/python38/lib/python3.8/site-packages/torchvision/datasets/folder.py", line 228, in __getitem__ path, target = self.samples[index] IndexError: list index out of range
最新发布
07-01
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值