Keras/TensorFlow报错CUDA_ERROR_OUT_OF_MEMORY:

最新推荐文章于 2023-06-26 16:38:50 发布
最新推荐文章于 2023-06-26 16:38:50 发布
阅读量6.3k 收藏 10
点赞数 4
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/Kyrielong/article/details/88288440
在使用Keras的model.predict时遇到CUDA错误：CUDA_ERROR_OUT_OF_MEMORY。即使GPU有8GB显存，依然报内存不足。错误日志显示多次尝试分配内存失败。解决方法是通过代码限制GPU内存占用率。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

用keras的model.predict时报错如下：

failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-03-07 10:24:00.610883: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-03-07 10:24:00.613876: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-03-07 10:24:00.617918: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-03-07 10:24:00.621197: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-03-07 10:24:00.624652: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 2.36G (2536134912 bytes) fro

最低0.47元/天 解锁文章

200万优质内容无限畅学
Kyrielong
关注
failed to allocate 18.41M (19300352 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
qq_36362060的博客
06-13 1万+
tensorflow 训练时出现failed to allocate 18.41M (19300352 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY 我在ubuntu16.04中安装的是tensorflow-gpu1.6.0版本的，在使用jupyter notebook训练时出现了failed to allocate 18.41M (19300352 ...
解决Tensorflow error(CUBLAS_STATUS_EXECUTION_FAILED)报错问题
SpadgerZ的博客
10-27 2871
显卡CUDA tensorflow-gpu
failed to allocate 192.19M (201523200 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
mindfusion的博客
02-15 2381
解决方式： import os os.environ['CUDA_VISIBLE_DEVICES'] = '1'
failed to allocate 5.91G (6347372032 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
Kp0fS的草稿纸
04-09 1118
import tensorflow as tf gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333) sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options)) 试试
【已解决】深度学习failed to allocate 4.67G (5019415296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
qq_43681154的博客
02-04 1242
出现这种问题的情况有两种，一种是没有指定GPU，导致内存不足，第二种是自己的设置导致内存超出。
【Tensorflow-Error】CUDA_ERROR_OUT_OF_MEMORY: out of memory
luckynote
03-20 1万+
在Tensorflow 训练模型时报错提示： failed to allocate 3.77G (4046333952 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 虽然会报出显存溢出问题，但不影响正常训练，不过笔者还是想知道这个问题是怎么来的。 废话不多说，先上session初始化的代码 gpu_optio...
2025-05-30 20:05:08.369514: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll 2025-05-30 20:05:08.784492: E tensorflow/stream_executor/cuda/cuda_blas.cc:428] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED 怎么解决？
05-31
2. TensorFlow出现`CUDA_ERROR_OUT_OF_MEMORY`错误应如何解决？ 3. 如何在不同版本的CUDA之间切换？ 4. 混合精度训练对cuBLAS性能有何影响？ --- ### 引用说明 - 环境变量非持久性问题可能导致CUDA路径未正确...
WARNING:tensorflow:From /root/Python/conda_lit/kind/lib/python3.6/site-packages/tensorflow/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. Instructions for updating: non-resource variables are not supported in the long term 2025-07-26 19:44:10.231806: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-07-26 19:44:10.313436: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:39:00.0 name: NVIDIA GeForce RTX 4090 computeCapability: 8.9 coreClock: 2.52GHz coreCount: 128 deviceMemorySize: 23.55GiB deviceMemoryBandwidth: 938.86GiB/s 2025-07-26 19:44:10.313630: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory 2025-07-26 19:44:10.313667: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10'; dlerror: libcublas.so.10: cannot open shared object file: No such file or directory 2025-07-26 19:44:10.314718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-07-26 19:44:10.314970: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-07-26 19:44:10.315017: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10'; dlerror: libcusolver.so.10: cannot open shared object file: No such file or directory 2025-07-26 19:44:10.315049: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10'; dlerror: libcusparse.so.10: cannot open shared object file: No such file or directory 2025-07-26 19:44:10.315080: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory 2025-07-26 19:44:10.315086: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1598] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2025-07-26 19:44:10.315433: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA 2025-07-26 19:44:10.342984: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2000000000 Hz 2025-07-26 19:44:10.349020: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f67d0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-07-26 19:44:10.349043: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-07-26 19:44:10.350861: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-07-26 19:44:10.350875: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] WARNING:tensorflow:From /root/Python/conda_lit/kind/lib/python3.6/site-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. loaded ./checkpoint/decom_net_train/model.ckpt loaded ./checkpoint/illumination_adjust_net_train/model.ckpt No restoration pre model! (480, 640, 3) (680, 720, 3) (415, 370, 3) Traceback (most recent call last): File "evaluate.py", line 73, in <module> eval_low_im = load_images(eval_low_data_name[idx]) File "/root/Python/KinD-master/KinD-master/utils.py", line 63, in load_images im = Image.open(file) File "/root/Python/conda_lit/kind/lib/python3.6/site-packages/PIL/Image.py", line 2975, in open fp = builtins.open(filename, "rb") IsADirectoryError: [Errno 21] Is a directory: './test/results'
最新发布
07-27
2. TensorFlow GPU 环境配置问题：缺少必要的 CUDA 库文件 ### 完整解决方案 #### 问题1：图像加载路径错误 (IsADirectoryError) ```python # utils.py (完整修复版) import os import numpy as np from PIL ...
tensorflow-gpu报错：tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found
laxyer的博客
03-27 1776
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{no
CUDA_ERROR_OUT_OF_MEMORY
weixin_30906185的博客
01-29 331
E tensorflow/stream_executor/cuda/cuda_driver.cc:1002] failed to allocate 198.83M (208486400 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY In case it's still relevant for someone, I encountered...
Tensorflow 错误001：CUDA_ERROR_OUT_OF_MEMORY
大可的杨先森的博客
11-14 1396
代码出现tensorflow.python.framework.errors_impl.InternalError，from device: CUDA_ERROR_OUT_OF_MEMORY 此文为解决机器学习中使用tensorflow，在运行代码出现上述报错情况 1.先运行nvidia-smi 检查GPU运行情况，若内存够用进入2 2.代码应作已下修改 import tensorflo...
TensorFlow2.0 CUDA_ERROR_OUT_OF_MEMORY out of memory 解决办法
OUTMAN80的博客
06-19 3138
今天头一次使用TensorFlow2.0的GPU版本进行训练，训练中遇到【CUDA_ERROR_OUT_OF_MEMORY out of memory】，于是百度各种解决办法，基本能查到的都是TensorFlow1.0的用法，但此方法在TensorFlow2.0中不适用。于是经过多方查找，终于找到设置方法，在此进行记录，一是方便日后回看，二是弥补百度中各种查不到。 TensorFlow1.0用法 from keras import backend as K config = tf.ConfigProto
pytorch遇见RuntimeError: CUDA out of memory的解决
引爆激情
07-24 3917
最近在用pytorch做项目时，本人遇到RuntimeError: CUDA out of memory的错误，下面就此问题做一个记录和分享，并谈谈今后遇到爆显存问题的解决思路。 目录 1.查看是否其他程序占用显存 2.查看pytorch和cuda是否匹配 3.cuda.empty_cache()方法 3.当前项目代码是否还使用了keras/tensorflow 4.调小batch_size和通过程序优化显存 1.查看是否其他程序占用显存 遇到此类错误后，对于py格式的文件来说，程序会进行终
Keras/TensorFlow 报错：CUDA_ERROR_OUT_OF_MEMORY 解决办法
Kyrielong的博客
03-05 8833
Keras/TensorFlow 报错如下： failed to alloc 2097152 bytes on host: CUDA_ERROR_OUT_OF_MEMORY could not allocate pinned host memory of size:xxxxx 解决办法： TensorFlow 默认贪婪的占用全部显存，所以有时候显存不够用，添加如下代码，让显存按需分配 ...
错误调试：failed to allocate 4.75G (5102921216 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
计算机视觉
10-18 4541
原文：https://blog.youkuaiyun.com/strive_for_future/article/details/81292865?utm_source=copy 1）需要指定GPU，代码头部添加如下代码： import os os.environ["CUDA_VISIBLE_DEVICES"] = "1" 2）限制当前脚本可用显存，代码头部添加第一行，session 语句进行如第二...
failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
hanshuning的专栏
03-01 4904
显存充足，但是却出现CUDA error:out of memory错误 添加： import os # 设置程序使用的GPU编号：我用6号和7号显卡 os.environ['CUDA_VISIBLE_DEVICES'] = '0,1' import numpy as np import keras import keras.backend as K import tensorflow.compat.v1 as tf tf.disable_v2_behavior() import keras.b
解决出现CUDA error:out of memory的问题
热门推荐
qq_46542320的博客
05-24 2万+
使用：CUDA_VISIBLE_DEVICES限制一下使用的GPU。CUDA_VISIBLE_DEVICES=0,1 则使用这两张GPU。CUDA_VISIBLE_DEVICES=0 则使用第一张GPU。CUDA_VISIBLE_DEVICES=1，则使用第二张GPU。如何设置CUDA_VISIBLE_DEVICES？例如：有两块GPU，即0,1号GPU，查阅资料，发现是显卡内存不足。
问题解决 |关于CUDA的代码错误总结以及解决方法
weixin_44649780的博客
06-26 1万+
本博客主要关于常见的CUDA的代码错误总结以及解决方法~
评论 5
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值