Towards Efficient and Scale-Robust

论文名字:Towards Efficient and Scale-Robust Ultra-High-Definition Image Demoir´eing
论文下载地址:https://arxiv.org/abs/2207.09935
论文代码地址: https://xinyu-andy.github.io/uhdm-page

论文内容:获取多语义信息并交互融合以处理超高清图像去摩尔纹。

具体内容:随着拍照设备的进步,超高清图像的获取更加容易,也对超高清图像的去摩尔纹技术提出了新的要求:处理负担小、处理分辨率高。

现有方法都是从低分辨率图像上训练和测试,在4K图像上难以去除摩尔纹,但他们也能承受对应的计算成本(即方法的计算成本可负担但效果欠佳),作者认为这些方法欠缺多尺度特征的有效提取策略(可能还有融合策略)。

**笔者观点:**去摩尔纹有种常用的方法是将输入下采样(通常是两次)放到对应的三个分支中,分辨率大小的改变意味着语义信息的不同,最直观的感受是同样10×10大小的感受野,缩略图看到的是一个结构而高清图看到的只是细节,此外同一分支中随着卷积的增加,语义信息也在发生改变。而分支之间的交互也即语义之间的交互都放在输出阶段,缺少有效交互和融合。因此作者想从这一点下手解决问题。

提出的方法一共分三点:DRDB模块、SAM模块、损失函数。整体结构框架如下(图画的真不赖):

请添加图片描述

DRDB(Dilated Residual Dense Block)

老面孔了,就是简单的密集块加空洞卷积,最后使用残差链接,感兴趣的可以查看这篇内容:残差密集块

SAM(Semantic-Aligned Scale-Aware Module)

这是作者提出的模块,也是解决多尺度(语义)的核心。由两个小模块组成,分别是: pyramid feature extraction 和cross-scale dynamic fusion

Pyramid feature extraction

请添加图片描述

首先对输入使用双线性差值进行下采样,然后经过扩展密集块(密集块+空洞卷积)再经过一个卷积层,由于这里的模块结构一样,因此参数也可以共享。为了保持输出的大小一致,因此增加了双线性差值上采样得到结果 Y 0 , Y 1 , Y 2 Y_0,Y_1,Y_2 Y0,Y1,Y2

Cross-scale dynamic fusion

请添加图片描述

对上一个模块的输出在空间维度做一个GAP,得到每一个channel一个值,拼接起来,过三层全连接层,分解回三个尺度,再对上一个模块的输出做一个相乘的操作,最后是一个残差链接。

**笔者观点:**这里其实就是一个channel-attention的操作。因为4K图像的分辨率很高,而摩尔纹的尺度范围也很大,可以是很大范围的摩尔纹也可以是一个很小范围内的摩尔纹,每张图像的侧重都不一样,因此需要做一个attention的操作,这点在论文中可以得到印证。但是我感到疑惑的是为什么不是把三个尺度各自作为一个整体而是全部channel都看作是单独的元素(如果这里不是做3分支而是做成5分支甚至50分支,操作都是一样的,并没有体现出三分支的特点)可能是模型足够强大可以学到最优的参数?此外,对齐不清楚体现在哪里?

损失函数

采用深度监管策略,简单讲就是在中间层加入分支对主干网络进行监督,感兴趣的可以查看 深度监管策略,在该模型中则是获得三种尺度的输出图像,具体见整体结构框架图像。同时对语义信息采取感知损失。

Experiments

数据集:FHDM i、TIP2018、LCDMoire、UHDM
评价指标:PSNR、SSIM、LPIPS

定量/定性

请添加图片描述
请添加图片描述

消融实验

请添加图片描述

A代表SAM中两个模块都没有
A+代表SAM中两个模块都没有,但是参数两调整到和E一致
B代表有金字塔上下文提取,且共享参数
D代表有金字塔上下文提取,不共享参数
C代表有跨尺度动态融合
E代表拥有全部组件的模型

金字塔上下文提取,且共享参数
D代表有金字塔上下文提取,不共享参数
C代表有跨尺度动态融合
E代表拥有全部组件的模型

### KAZE-SLAM Algorithm Advantages and Disadvantages The KAZE-SLAM algorithm leverages the KAZE feature detector, which is a scale-invariant feature detection method designed to identify robust keypoints across different scales and lighting conditions. Below are its key advantages and disadvantages: #### Advantages 1. **Robustness Across Scales**: The KAZE feature detector uses nonlinear diffusion filtering techniques that allow it to detect features at multiple scales effectively while preserving edge information[^3]. This makes KAZE particularly suitable for scenarios where scaling variations occur frequently. 2. **Invariance Properties**: Features detected using KAZE exhibit strong invariance properties against changes such as illumination, rotation, and affine transformations, making them highly reliable for visual SLAM applications. 3. **High Accuracy with Sparse Reconstructions**: When integrated into SLAM systems like KAZE-SLAM, these invariant features contribute significantly towards accurate camera pose estimation and sparse geometric reconstructions without excessive computational overhead compared to other dense methods[^1]. 4. **Efficient Matching Performance**: Due to their distinctive nature, matching between frames can be performed efficiently even under challenging imaging conditions or dynamic scenes containing repetitive patterns or textureless areas. #### Disadvantages 1. **Computational Complexity**: Although less computationally intensive than some alternatives (e.g., Bundle Adjustment), extracting KAZE descriptors remains relatively expensive due primarily to non-linear processing steps involved during descriptor generation. Consequently, this may limit real-time performance on resource-constrained devices unless optimized implementations exist. 2. **Memory Requirements**: Storing large numbers of high-dimensional KAZE descriptors could lead to increased memory usage over time within long-running sessions typical of many autonomous navigation tasks involving continuous mapping processes. 3. **Limited Applicability Under Certain Conditions**: While generally effective, there might still remain specific situations—such as extremely low-light environments—that challenge both detection reliability and subsequent tracking success rates depending upon how well tuned parameters have been set beforehand according user needs/preferences[^4]. ```python import cv2 from matplotlib import pyplot as plt def kaze_feature_detection(image_path): img = cv2.imread(image_path, 0) # Initialize KAZE detector kaze = cv2.KAZE_create() # Detect keypoints and compute descriptors kp, des = kaze.detectAndCompute(img, None) return kp, des # Example Usage keypoints, descriptors = kaze_feature_detection('example_image.jpg') print(f"Number of Keypoints Detected: {len(keypoints)}") ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值