torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 214.00 MiB. GPU 0 has a total capacity

本地有两张GPU,NVIDIA GeForce RTX 4080 和 NVIDIA GeForce RTX 4080 SUPER但是

启动报错

oading checkpoint shards: 80%|████████████████████████████████████████████████████████████████████████████▊ | 8/10 [01:38<00:24, 12.30s/it] 2024-11-16 08:46:26.929 Uncaught app exception Traceback (most recent call last): File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 575, in _run_script self._session_state.on_script_will_rerun( File "/home/xx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/state/safe_session_state.py", line 65, in on_script_will_rerun self._state.on_script_will_rerun(latest_widget_states) File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/state/session_state.py", line 517, in on_script_will_rerun self._call_callbacks() File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/state/session_state.py", line 530, in _call_callbacks self._new_widget_state.call_callback(wid) File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/state/session_state.py", line 274, in call_callback callback(*args, **kwargs) File "/server/repo/GLM-4/composite_demo/src/main.py", line 130, in page_changed st.session_state.client = build_client(Mode(new_page)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/server/repo/GLM-4/composite_demo/src/main.py", line 114, in build_client return get_client(CHAT_MODEL_PATH, typ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 165, in wrapper return cached_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 194, in __call__ return self._get_or_create_cached_value(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 221, in _get_or_create_cached_value return self._handle_cache_miss(cache, value_key, func_args, func_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 277, in _handle_cache_miss computed_value = self._info.func(*func_args, **func_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/server/repo/GLM-4/composite_demo/src/client.py", line 89, in get_client return HFClient(model_path) ^^^^^^^^^^^^^^^^^^^^ File "/server/repo/GLM-4/composite_demo/src/clients/hf.py", line 21, in __init__ self.model = AutoModelForCausalLM.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4225, in from_pretrained ) = cls._load_pretrained_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4728, in _load_pretrained_model new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xxx/anaconda3/envs/glm-4-demo/lib/python3.12/site-packages/transformers/modeling_utils.py", line

torch.cuda.OutOfMemoryError: CUDA out of memory是指在使用PyTorch进行GPU计算时,出现了内存不足的错误。具体来说,在尝试分配810.00 MiB的内存时,由于已经分配了一部分内存并没有释放,导致了内存不足的情况。 解决这个问题的方法之一是调整max_split_size_mb参数,以避免内存碎片化。这个参数可以在代码中设置,具体的值可以根据情况进行调整。另外,可以查阅PyTorch的内存管理文档以及PYTORCH_CUDA_ALLOC_CONF文档,获取更多关于内存管理和配置的信息。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* [报错记录torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 90.00 MiB (GPU 0; 7.93 ...](https://blog.csdn.net/gongkeguo/article/details/127674524)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] - *2* *3* [成功解决torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 2.00 ...](https://blog.csdn.net/qq_41185868/article/details/131015210)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值