训练时workers和 batch多大合适 cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
132/180 10.1G 0.3115 0.423 0.08586 10 640: 100%|██████████| 278/278 [03:27<00:00, 1.34it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.62it/s]
all 832 5918 0.866 0.72 0.807 0.566
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
133/180 10.3G 0.3097 0.4263 0.0904 15 640: 100%|██████████| 278/278 [03:22<00:00, 1.37it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.56it/s]
all 832 5918 0.863 0.722 0.807 0.565
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
134/180 10.1G 0.3027 0.4206 0.08962 12 640: 100%|██████████| 278/278 [03:30<00:00, 1.32it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.60it/s]
all 832 5918 0.863 0.724 0.807 0.565
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
135/180 10.3G 0.3028 0.4213 0.08694 10 640: 100%|██████████| 278/278 [03:26<00:00, 1.34it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.56it/s]
all 832 5918 0.859 0.727 0.806 0.565
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
136/180 10.4G 0.3089 0.4218 0.08623 31 640: 100%|██████████| 278/278 [03:24<00:00, 1.36it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.50it/s]
all 832 5918 0.86 0.724 0.805 0.564
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
137/180 10.3G 0.311 0.4224 0.08655 25 640: 100%|██████████| 278/278 [03:25<00:00, 1.35it/s]
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 52/52 [00:14<00:00, 3.54it/s]
all 832 5918 0.858 0.725 0.805 0.565
Epoch GPU_mem giou_loss cls_loss l1_loss Instances Size
0%| | 0/278 [00:00<?, ?it/s]/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/home/l808/.conda/envs/rtdetr_ultralytics/lib/python3.10/site-packages/torch/autograd/__init__.py:266: UserWarning: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True, warn_only=True)'. You can file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation. (Triggered internally at /opt/conda/conda-bld/pytorch_1704987394225/work/aten/src/ATen/Context.cpp:79.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
为什么训练速度这么慢