KPAC(ICCV 2021)代码单图片推理

原创 已于 2024-11-25 14:34:11 修改 · 1k 阅读 · 23 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#linux #WSL #ubuntu #图像恢复 #图像去模糊 #ICCV

于 2024-11-25 11:59:44 首次发布

文章目录

KPAC(ICCV 2021)代码单图片推理

论文链接：Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions

该论文研究的问题是散焦去模糊，是图像恢复中经常需要做的一个任务，经常需要找对比图片。如果可以运行该论文，那么论文中可以增加一个对比图片。现在论文中大家都不用TensorFlow，不巧的是该论文原始代码是TF写的。我们并不是改为torch形式，而依然通过安装tf1.15的gpu版本。原始的代码我没法进行测试，改了半天放弃了，结合chatgpt写了一个单图片推理的代码，可以成功运行，但是依然需要大概8.6G的显存。

我的环境：

系统：Ubuntu 22.04（WSL2）
显卡：4070 ti s
wsl2的cuda版本：11.5

论文github链接：https://github.com/HyeongseokSon1/KPAC

TensorFlow的版本：https://developer.download.nvidia.com/compute/redist/nvidia-tensorflow/

我使用的tf版本下载：https://developer.download.nvidia.com/compute/redist/nvidia-tensorflow/nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl

该tf1.15.5版本应该是支持30和40系显卡的。

论文的KPAC如下：

在这里插入图片描述

创建虚拟环境

通过conda新建虚拟环境，使用如下代码：

conda create -n KPAC python=3.8

激活环境：

conda activate KPAC

再KPAC虚拟环境下安装依赖包，切忌不要再base虚拟环境下操作，通过新建虚拟环境来操作。

安装依赖包

首先下载下来nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl，然后本地安装：

pip install nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl

安装tf之后，测试gpu是否可用。将下面代码保存为device_gpu.py，然后运行。

import tensorflow as tf

gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
    print("Available GPU(s):")
    for gpu in gpus:
        print(gpu)
else:
    print("No GPUs detected.")

输出大概如下：

2024-11-25 11:09:56.018231: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
2024-11-25 11:09:57.076884: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2024-11-25 11:09:57.303655: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.303730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1666] Found device 0 with properties: 
name: NVIDIA GeForce RTX 4070 Ti SUPER major: 8 minor: 9 memoryClockRate(GHz): 2.61
pciBusID: 0000:2b:00.0
2024-11-25 11:09:57.303762: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2024-11-25 11:09:57.330370: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2024-11-25 11:09:57.354441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2024-11-25 11:09:57.357430: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2024-11-25 11:09:57.363308: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2024-11-25 11:09:57.382721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2024-11-25 11:09:57.383752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2024-11-25 11:09:57.383854: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.383936: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.383970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1794] Adding visible gpu devices: 0
Available GPU(s):
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')

会有一个GPU:0，表示gpu可用。

之后可以安装下面依赖包：

easydict==1.13
matplotlib==3.5.3
numpy==1.18.5
opencv-python==4.10.0.84
protobuf==3.20.3
scikit-image==0.17.2
scipy==1.7.3

数据集路径设置

找到根目录下的config.py文件，修改如下路径：

# for dual
config.TEST.folder_path_c = 'DeblurDataset/DefocusDeblur/test_c/' # need to change
config.TEST.folder_path_l = 'DeblurDataset/DefocusDeblur/test_l/' # need to change
config.TEST.folder_path_r = 'DeblurDataset/DefocusDeblur/test_r/' # need to change

因为我是运行的双像素去看模糊，根据main_eval_dual.py修改的，我就只修改了这个路径。

运行测试

如果安装原论文的方法使用如下代码测试：

CUDA_VISIBLE_DEVICES=0 python main_eval_dual.py

会有如下错误，我是放弃了修改这个代码，scipy=1.2.*的版本好像在python 3.8版本中不可安装。

[*] Load pretrained/dual.npz SUCCESS!
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
    return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_c//source/
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
    return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_l//source/
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
    return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_r//source/
[!] ./Evaluations/dual_results/ exists ...
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
    x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
    x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Exception in thread threading_and_return:
Traceback (most recent call last):
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
    results[i] = fn(data, **kwargs)
  File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
    x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Traceback (most recent call last):
  File "main_eval_dual.py", line 109, in <module>
    DefocusDeblur()
  File "main_eval_dual.py", line 93, in DefocusDeblur
    out = sess.run(result, {t_image_c: valid_ref_img_c, t_image_l: valid_ref_img_l,t_image_r: valid_ref_img_r})                
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/site-packages/tensorflow_core/python/client/session.py", line 955, in run
    result = self._run(None, fetches, feed_dict, options_ptr,
  File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/site-packages/tensorflow_core/python/client/session.py", line 1153, in _run
    raise ValueError(
ValueError: Cannot feed value of shape (1,) for Tensor 't_image_c:0', which has shape '(1, 1120, 1680, 3)'

虽说默认的代码无法运行，我们主要为了测试，生成一个去模糊的图片，我们也不是用于训练的，非常不推荐运行。

如果你想复现图像恢复相关的论文，我推荐你NAFNet、Restormer，这两个效果又好，又容易，缺点就是需要显卡，如果你想使用自己的数据集，修改修改batch_size，调小一点。

单图片推理

回到KPAC，默认的代码无法使用，我们修改一下，改为单图片去模糊，使用如下代码，可以随便起一个文件名，例如single_img.py。

import os
import time
import numpy as np
import tensorflow as tf
import tensorlayer as tl
from model import Defocus_Deblur_Net6_ds_dual
from utils import scale_imgs_fn
from PIL import Image


# 加载并预处理图片
def load_and_preprocess_image(img_path):
    """
    加载并预处理图片。
    :param img_path: 图片路径
    :return: 归一化后的图片数组
    """
    if not os.path.exists(img_path):
        raise FileNotFoundError(f"[Error] Image not found: {img_path}")
    
    img = Image.open(img_path).convert("RGB")  # 确保图片为 RGB 格式
    img = np.array(img, dtype=np.float32)
    img = np.expand_dims(img, axis=0)  # 添加 batch 维度
    img = tl.prepro.threading_data(img, fn=scale_imgs_fn)  # 归一化到 [-1, 1]
    return img


# 保存图片
def save_image(image_array, save_path):
    """
    保存图片到文件。
    :param image_array: 图像数组 ([-1, 1])
    :param save_path: 保存路径
    """
    # 将 [-1, 1] 转换为 [0, 255]
    image_array = ((image_array + 1) * 127.5).astype(np.uint8)
    image = Image.fromarray(image_array)
    image.save(save_path)
    print(f"Deblurred image saved to {save_path}")


# 测试单张图片的去模糊
def test_single_image(image_path_c, image_path_l, image_path_r, save_path):
    """
    测试单张图片的去模糊。
    :param image_path_c: 中心图像路径
    :param image_path_l: 左图像路径
    :param image_path_r: 右图像路径
    :param save_path: 输出结果保存路径
    """
    # 参数定义
    H, W = 1120, 1680
    checkpoint_dir = "pretrained"

    # 加载图片
    try:
        img_c = load_and_preprocess_image(image_path_c)
        img_l = load_and_preprocess_image(image_path_l)
        img_r = load_and_preprocess_image(image_path_r)
    except FileNotFoundError as e:
        print(e)
        return

    # 构建网络
    with tf.device('/device:GPU:0'):
        t_image_c = tf.placeholder('float32', [1, H, W, 3], name='t_image_c')
        t_image_l = tf.placeholder('float32', [1, H, W, 3], name='t_image_l')
        t_image_r = tf.placeholder('float32', [1, H, W, 3], name='t_image_r')
        net_g = Defocus_Deblur_Net6_ds_dual(t_image_c, t_image_l, t_image_r, is_train=False, hrg=H, wrg=W)
        result = net_g.outputs

    # TensorFlow 会话配置
    configg = tf.ConfigProto()
    configg.gpu_options.allow_growth = True
    sess = tf.Session(config=configg)
    tl.layers.initialize_global_variables(sess)
    tl.files.load_and_assign_npz(sess=sess, name=f'{checkpoint_dir}/dual.npz', network=net_g)

    # 运行模型
    print("[*] Running deblurring...")
    start_time = time.time()
    out = sess.run(result, feed_dict={t_image_c: img_c, t_image_l: img_l, t_image_r: img_r})
    print("Processing time: %.4fs" % (time.time() - start_time))

    # 保存结果
    save_image(out[0], save_path)


# 主函数
if __name__ == '__main__':
    # 测试单张图片的输入路径
    image_path_c = "DeblurDataset/DefocusDeblur/test_c/source/1P0A2417.png"  # 替换为实际图片路径
    image_path_l = "DeblurDataset/DefocusDeblur/test_l/source/1P0A2417_L.png"    # 替换为实际图片路径
    image_path_r = "DeblurDataset/DefocusDeblur/test_r/source/1P0A2417_R.png"   # 替换为实际图片路径

    # 保存结果的路径
    save_path = "./deblurred_result.png"

    # 调用测试函数
    test_single_image(image_path_c, image_path_l, image_path_r, save_path)

之后我们运行该代码：

python single_img.py

大概输出如下内容：

[*] Load pretrained/dual.npz SUCCESS!
[*] Running deblurring...
2024-11-25 11:43:55.798790: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2024-11-25 11:43:58.611393: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_89' is not defined for option 'gpu-name'

Relying on driver to perform ptx compilation. This message will be only logged once.
2024-11-25 11:43:59.507653: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
Processing time: 33.1819s
Deblurred image saved to ./deblurred_result.png

可以成功使用gpu预测，但是不巧的是，需要显存大概8.6G，所以最好使用12G显存以上的显卡，否则可能无法使用gpu预测，或者直接使用cpu预测。

评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

百年孤独百年

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值