目录
3.1.1 640×640标准输入与动态尺寸调整机制
现代目标检测系统普遍采用640×640作为标准输入分辨率,该规范源于早期YOLO系列对计算效率与特征分辨率的权衡。固定正方形输入虽简化了批量处理逻辑,却引入冗余计算开销。动态尺寸调整机制通过矩形推理(Rectangular Inference)与自适应填充(Adaptive Padding)重构输入流,在保持批量并行性的前提下消除无效计算区域。
矩形推理的核心在于维持长宽比的一致性缩放。设原始图像尺寸为 $(W,H)$,目标约束为 $S_{max}=640$,缩放系数 $\lambda$ 定义为:
$$\lambda = \min\left(\frac{S_{max}}{W}, \frac{S_{max}}{H}\right)$$
有效缩放后尺寸 $(W',H')$ 满足:
$$W' = \lfloor \lambda W \rfloor, H' = \lfloor \lambda H \rfloor$$
自适应填充机制在矩形推理基础上引入模对齐约束。现代硬件加速器要求张量维度为32的整数倍以优化内存访问模式。填充量 $(\Delta W, \Delta H)$ 计算为:
$$\Delta W = 32 - (W' \text{mod} 32), \Delta H = 32 - (H' \text{mod} 32)$$
该机制确保卷积层输入满足:
$$W'' = W' + \Delta W \equiv 0 (\text{mod} 32)$$
颜色空间转换遵循RGB三通道均值归一化。设原始像素值为 $P_{ij} \in [0,255]^3$,归一化后像素 $N_{ij}$ 为:
$$N_{ij} = \frac{P_{ij}}{255.0}$$
批量构建阶段采用动态张量拼接策略。非正方形输入通过零值填充(Zero Padding)构建统一批次,前向传播时通过有效区域掩码(Valid Region Masking)抑制填充区域的梯度回传。
3.1.2 马赛克增强 (Mosaic) 的废弃与替代策略
传统马赛克增强通过四图拼接提升小目标检测性能,却在高密度场景引入显著的信息边界效应。图像拼接处的硬边界导致卷积核在不连续区域激活异常,破坏特征空间的一致性。此外,子图缩放过程中的最近邻插值产生锯齿伪影,在亚像素级别扭曲目标几何结构。
当代预处理管道采用空间感知混合增强(Spatial-Aware Blending)替代传统马赛克。该策略包含三个核心组件:动态复制粘贴(Dynamic Copy-Paste)、自适应混合(Adaptive Mixup)与上下文感知裁剪(Context-Aware Cropping)。
动态复制粘贴机制从辅助数据集采样目标实例,通过泊松融合(Poisson Blending)实现前景与背景的无缝集成。设源图像为 $I_s$,掩码为 $M_s$,目标背景为 $I_t$,融合结果 $I_f$ 通过求解泊松方程获得:
$$\nabla^2 I_f = \nabla \cdot v$$
其中 $v$ 为引导向量场,在掩码边界处保持梯度连续性。
自适应混合策略根据目标置信度动态调整混合系数。对于配对样本 $(I_1, I_2)$ 及其标签 $(Y_1, Y_2)$,混合系数 $\alpha \sim \text{Beta}(0.5, 0.5)$,合成图像 $I_m$ 与标签 $Y_m$ 为:
$$I_m = \alpha I_1 + (1 - \alpha) I_2$$
标签融合采用边界框并集策略,保留置信度高于阈值 $\tau$ 的检测框。
上下文感知裁剪基于显著性图(Saliency Map)引导裁剪区域选择。通过预训练网络的梯度反馈定位高信息密度区域,确保裁剪窗口覆盖关键语义对象,避免传统随机裁剪导致的上下文信息丢失。
3.1.3 混合精度训练的数据格式规范
混合精度训练通过FP16(半精度浮点)或BF16(脑浮点)降低显存占用并加速矩阵运算。数据格式规范需确保数值稳定性与硬件对齐要求。
梯度缩放机制(Gradient Scaling)防止半精度训练中的下溢问题。损失缩放因子 $s$ 动态调整,前向传播时损失值乘以 $s$,反向传播后梯度除以 $s$。缩放因子更新遵循溢出检测策略:
$$s_{t+1} = \begin{cases} s_t / 2 & \text{if overflow} \\ \min(s_t \times 2, s_{max}) & \text{if consecutive valid} \\ s_t & \text{otherwise} \end{cases}$$
张量内存布局遵循NHWC(Batch-Height-Width-Channels)或NCHW格式,依硬件架构而定。CUDA Tensor Core要求矩阵维度为8的倍数以实现Warp级并行。数据预处理阶段需对批次维度实施动态填充:
$$B' = B + (8 - (B \text{mod} 8)) \text{mod} 8$$
归一化精度保持FP32执行,在数据类型转换前完成。设归一化后的FP32张量为 $X_{32}$,半精度张量 $X_{16}$ 通过舍入转换:
$$X_{16} = \text{round-to-nearest-even}(X_{32})$$
特殊值处理需过滤无穷大与非数字(NaN)。输入校验层实施范围约束:
$$X_{16} \in [\epsilon, 1.0 - \epsilon], \epsilon = 2^{-24}$$
第二部分 结构化伪代码
算法 1:动态尺寸调整与矩形推理
algorithm
\begin{algorithm}
\caption{Dynamic Rectangular Resizing with Modular Alignment}
\begin{algorithmic}[1]
\Require Image $I \in \mathbb{R}^{H \times W \times 3}$, Target Size $S_{\max} = 640$, Modulo $M = 32$
\Ensure Resized Tensor $T \in \mathbb{R}^{H'' \times W'' \times 3}$, Scale Factor $\lambda$, Padding $(\Delta H, \Delta W)$
\State $\lambda \gets \min(S_{\max} / W, S_{\max} / H)$
\State $W' \gets \lfloor \lambda \cdot W \rfloor$, $H' \gets \lfloor \lambda \cdot H \rfloor$
\State $\Delta W \gets (M - (W' \bmod M)) \bmod M$
\State $\Delta H \gets (M - (H' \bmod H)) \bmod M$
\State $W'' \gets W' + \Delta W$, $H'' \gets H' + \Delta H$
\State $I_{resized} \gets \text{BilinearInterpolation}(I, W', H')$
\State $T \gets \text{ZeroPad}(I_{resized}, \Delta W, \Delta H)$
\State $T \gets T / 255.0$ \Comment{Normalization to $[0,1]$}
\State \Return $T, \lambda, (\Delta H, \Delta W)$
\end{algorithmic}
\end{algorithm}
算法 2:空间感知混合增强管道
algorithm
\begin{algorithm}
\caption{Spatial-Aware Augmentation Pipeline}
\begin{algorithmic}[1]
\Require Dataset $\mathcal{D}$, Batch Size $B$, Image Size $S$
\Ensure Augmented Batch $\mathcal{B} = \{(I_i, Y_i)\}_{i=1}^B$
\For{$i \gets 1$ \textbf{to} $B$ \textbf{do}}
\State $I_{base}, Y_{base} \gets \text{Sample}(\mathcal{D})$
\State $r \sim \text{Uniform}(0, 1)$
\If{$r < 0.5$}
\State $I_{aux}, Y_{aux} \gets \text{Sample}(\mathcal{D})$
\State $\alpha \sim \text{Beta}(0.5, 0.5)$
\State $I_{mixed} \gets \alpha \cdot I_{base} + (1-\alpha) \cdot I_{aux}$
\State $Y_{mixed} \gets \text{MergeBoxes}(Y_{base}, Y_{aux}, \alpha)$
\ElsIf{$r < 0.8$}
\State $I_{fg}, M_{fg}, Y_{fg} \gets \text{SampleForeground}(\mathcal{D})$
\State $(x, y) \gets \text{SelectLocation}(I_{base}, M_{fg})$
\State $I_{mixed} \gets \text{PoissonBlend}(I_{fg}, M_{fg}, I_{base}, x, y)$
\State $Y_{mixed} \gets \text{ProjectBoxes}(Y_{base}, Y_{fg}, x, y)$
\Else
\State $S_{crop} \sim \text{Uniform}(0.5 \cdot S, S)$
\State $(x_1, y_1) \gets \text{SaliencyBasedCrop}(I_{base}, S_{crop})$
\State $I_{mixed} \gets \text{CropAndResize}(I_{base}, x_1, y_1, S_{crop}, S)$
\State $Y_{mixed} \gets \text{FilterBoxes}(Y_{base}, x_1, y_1, S_{crop})$
\EndIf
\State $\mathcal{B} \gets \mathcal{B} \cup \{(I_{mixed}, Y_{mixed})\}$
\EndFor
\State \Return $\mathcal{B}$
\end{algorithmic}
\end{algorithm}
算法 3:混合精度数据格式转换与校验
algorithm
\begin{algorithm}
\caption{Mixed-Precision Data Formatting with Gradient Scaling}
\begin{algorithmic}[1]
\Require Input Batch $\mathcal{B} = \{I_i\}_{i=1}^B$, Initial Scale $s_0 = 65536.0$, $\epsilon = 2^{-24}$
\Ensure FP16 Batch $\mathcal{B}_{16}$, Scale Factor $s_t$, Validity Flag $\delta$
\State $s_t \gets s_0$, $\delta \gets \text{True}$
\State $T_{32} \gets \text{Stack}(\mathcal{B})$ \Comment{Concatenate to tensor $T_{32} \in \mathbb{R}^{B \times 3 \times H \times W}$}
\State $T_{32} \gets \text{Normalize}(T_{32}, \mu = [0.485, 0.456, 0.406], \sigma = [0.229, 0.224, 0.225])$
\State $T_{32} \gets \text{Clamp}(T_{32}, \min = \epsilon, \max = 1.0 - \epsilon)$
\If{$\exists x \in T_{32} : \text{isnan}(x) \lor \text{isinf}(x)$}
\State $\delta \gets \text{False}$
\State \Return $\emptyset, s_t, \delta$
\EndIf
\State $T_{16} \gets \text{CastToFP16}(T_{32})$ \Comment{Rounding to nearest even}
\State $G \gets \text{ComputeGradients}(T_{16})$ \Comment{Forward/Backward pass}
\If{$\text{CheckOverflow}(G)$}
\State $s_t \gets s_t / 2.0$
\State $\delta \gets \text{False}$
\ElsIf{$\text{ConsecutiveValidIterations} > 2000$}
\State $s_t \gets \min(s_t \times 2.0, 65536.0)$
\EndIf
\State $G_{scaled} \gets G / s_t$
\State \Return $T_{16}, s_t, \delta$
\end{algorithmic}
\end{algorithm}
第三部分 代码实现
脚本 1:动态尺寸调整与矩形推理系统
Python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
脚本名称: dynamic_rectangular_resizing.py
内容概述:
实现YOLO26输入管道的动态尺寸调整机制,包含矩形推理、模32对齐、
自适应填充及批量图像预处理。提供可视化对比标准resize与矩形推理差异。
使用方式:
python dynamic_rectangular_resizing.py --image_path ./sample.jpg --target_size 640
"""
import cv2
import numpy as np
import torch
import torchvision.transforms as T
from typing import Tuple, List, Dict
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import argparse
class RectangularResizer:
"""
矩形推理调整器,实现动态尺寸计算与硬件对齐填充。
支持单图处理与批量处理,保持长宽比并优化计算效率。
"""
def __init__(self, target_size: int = 640, stride: int = 32,
normalize: bool = True, half: bool = False):
"""
初始化调整器。
Args:
target_size: 目标长边尺寸(通常为640)
stride: 硬件对齐步长(默认32,适配Tensor Core)
normalize: 是否执行归一化(除以255)
half: 是否转换为FP16
"""
self.target_size = target_size
self.stride = stride
self.normalize = normalize
self.half = half
def compute_rectangular_shape(self, h: int, w: int) -> Tuple[float, int, int, int, int]:
"""
计算矩形推理后的尺寸与填充量。
Returns:
scale: 缩放比例
new_h: 有效高度(未填充)
new_w: 有效宽度(未填充)
pad_h: 高度方向填充量
pad_w: 宽度方向填充量
"""
# 计算缩放比例,保持长宽比
scale = min(self.target_size / h, self.target_size / w)
# 计算有效尺寸(未填充)
new_h = int(h * scale)
new_w = int(w * scale)
# 计算模对齐填充量
pad_h = (self.stride - new_h % self.stride) % self.stride
pad_w = (self.stride - new_w % self.stride) % self.stride
return scale, new_h, new_w, pad_h, pad_w
def letterbox(self, img: np.ndarray, color: Tuple[int, int, int] = (114, 114, 114)) -> Dict:
"""
执行Letterbox调整(保持长宽比的自适应填充)。
Args:
img: 输入图像 (H, W, 3) BGR格式
color: 填充颜色
Returns:
包含处理结果的字典:
- resized: 调整后的图像张量 (3, H', W')
- scale: 缩放比例
- pad: (top, bottom, left, right)填充像素数
- original_shape: 原始尺寸 (H, W)
- valid_shape: 有效区域尺寸 (H_new, W_new)
"""
h, w = img.shape[:2]
# 计算矩形参数
scale, new_h, new_w, pad_h, pad_w = self.compute_rectangular_shape(h, w)
# 缩放图像(双线性插值)
if (h, w) != (new_h, new_w):
img_resized = cv2.resize(img, (new_w, new_h), interpolation=cv2.INTER_LINEAR)
else:
img_resized = img.copy()
# 计算对称填充(上下、左右)
top = pad_h // 2
bottom = pad_h - top
left = pad_w // 2
right = pad_w - left
# 执行填充
img_padded = cv2.copyMakeBorder(
img_resized, top, bottom, left, right,
cv2.BORDER_CONSTANT, value=color
)
# 转换颜色空间 BGR->RGB,并调整维度为 (C, H, W)
img_rgb = cv2.cvtColor(img_padded, cv2.COLOR_BGR2RGB)
img_tensor = torch.from_numpy(img_rgb.transpose(2, 0, 1)).float()
# 归一化
if self.normalize:
img_tensor /= 255.0
# 半精度转换
if self.half:
img_tensor = img_tensor.half()
return {
'resized': img_tensor,
'scale': scale,
'pad': (top, bottom, left, right),
'original_shape': (h, w),
'valid_shape': (new_h, new_w),
'final_shape': img_padded.shape[:2]
}
def batch_process(self, img_list: List[np.ndarray]) -> torch.Tensor:
"""
批量处理多张图像,构建统一批次。
Args:
img_list: 图像列表,每张图像尺寸可能不同
Returns:
批次张量 (N, 3, H_max, W_max),其中H_max和W_max为批次内最大尺寸
"""
processed = [self.letterbox(img) for img in img_list]
# 获取最大尺寸
max_h = max(p['final_shape'][0] for p in processed)
max_w = max(p['final_shape'][1] for p in processed)
# 构建批次张量
batch = torch.zeros((len(img_list), 3, max_h, max_w),
dtype=torch.float16 if self.half else torch.float32)
for i, p in enumerate(processed):
_, h, w = p['resized'].shape
batch[i, :, :h, :w] = p['resized']
return batch, processed
def visualize_pipeline(self, img: np.ndarray, save_path: str = None):
"""
可视化预处理管道各阶段输出。
"""
result = self.letterbox(img)
h, w = result['original_shape']
new_h, new_w = result['valid_shape']
top, bottom, left, right = result['pad']
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
# 原始图像
ax = axes[0, 0]
ax.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
ax.set_title(f'Original Image\n{h}×{w}')
ax.axis('off')
# 缩放后(未填充)
resized_only = cv2.resize(img, (new_w, new_h), interpolation=cv2.INTER_LINEAR)
ax = axes[0, 1]
ax.imshow(cv2.cvtColor(resized_only, cv2.COLOR_BGR2RGB))
ax.set_title(f'Resized (No Padding)\n{new_h}×{new_w}\nScale: {result["scale"]:.3f}')
ax.axis('off')
# 填充后(带标记)
ax = axes[0, 2]
img_display = result['resized'].permute(1, 2, 0).numpy()
if self.normalize:
img_display = (img_display * 255).astype(np.uint8)
ax.imshow(img_display.astype(np.uint8))
ax.set_title(f'Letterbox Result\n{result["final_shape"][0]}×{result["final_shape"][1]}')
# 绘制有效区域边界
rect = Rectangle((left, top), new_w, new_h,
linewidth=3, edgecolor='r', facecolor='none', linestyle='--')
ax.add_patch(rect)
ax.text(left + 5, top - 5, f'Valid Region: {new_w}×{new_h}',
color='r', fontsize=10, weight='bold')
ax.axis('off')
# 计算效率对比
standard_area = self.target_size * self.target_size
actual_area = result['final_shape'][0] * result['final_shape'][1]
valid_area = new_h * new_w
efficiency = valid_area / actual_area * 100
ax = axes[1, 0]
ax.bar(['Standard\nSquare', 'Rectangular\n(Ours)'],
[standard_area, actual_area],
color=['#ff7f0e', '#2ca02c'], alpha=0.7)
ax.set_ylabel('Pixels')
ax.set_title(f'Area Comparison\nValid Ratio: {efficiency:.1f}%')
for i, v in enumerate([standard_area, actual_area]):
ax.text(i, v + 1000, f'{v:,}', ha='center', va='bottom')
# 填充分布
ax = axes[1, 1]
labels = ['Top', 'Bottom', 'Left', 'Right']
values = [top, bottom, left, right]
colors = ['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728']
ax.bar(labels, values, color=colors, alpha=0.7)
ax.set_ylabel('Padding Pixels')
ax.set_title(f'Padding Distribution\nTotal: {sum(values)} pixels')
# 张量信息
ax = axes[1, 2]
ax.text(0.5, 0.8, f'Tensor Shape: {tuple(result["resized"].shape)}',
ha='center', transform=ax.transAxes, fontsize=12, family='monospace')
ax.text(0.5, 0.6, f'Dtype: {result["resized"].dtype}',
ha='center', transform=ax.transAxes, fontsize=12, family='monospace')
ax.text(0.5, 0.4, f'Value Range: [{result["resized"].min():.3f}, {result["resized"].max():.3f}]',
ha='center', transform=ax.transAxes, fontsize=12, family='monospace')
ax.text(0.5, 0.2, f'Memory: {result["resized"].numel() * result["resized"].element_size() / 1024:.2f} KB',
ha='center', transform=ax.transAxes, fontsize=12, family='monospace')
ax.axis('off')
ax.set_title('Tensor Metadata')
plt.tight_layout()
if save_path:
plt.savefig(save_path, dpi=150, bbox_inches='tight')
print(f"Visualization saved to {save_path}")
else:
plt.show()
return result
def main():
parser = argparse.ArgumentParser(description='Dynamic Rectangular Resizing Demo')
parser.add_argument('--image_path', type=str, default=None, help='Path to input image')
parser.add_argument('--target_size', type=int, default=640, help='Target size (default: 640)')
parser.add_argument('--stride', type=int, default=32, help='Alignment stride (default: 32)')
parser.add_argument('--half', action='store_true', help='Use FP16')
args = parser.parse_args()
# 创建合成测试图像(若未提供路径)
if args.image_path is None:
# 生成随机非方形图像(模拟高分辨率输入)
test_h, test_w = 1080, 1920 # 模拟1080p图像
img = np.random.randint(0, 255, (test_h, test_w, 3), dtype=np.uint8)
# 添加一些几何形状以便观察形变
cv2.rectangle(img, (100, 100), (500, 400), (0, 255, 0), -1)
cv2.circle(img, (800, 600), 200, (255, 0, 0), -1)
print(f"Generated synthetic test image: {test_h}×{test_w}")
else:
img = cv2.imread(args.image_path)
if img is None:
raise ValueError(f"Failed to load image: {args.image_path}")
# 初始化调整器
resizer = RectangularResizer(
target_size=args.target_size,
stride=args.stride,
normalize=True,
half=args.half
)
# 执行并可视化
result = resizer.visualize_pipeline(img, save_path='rectangular_resizing_demo.png')
print(f"Original shape: {result['original_shape']}")
print(f"Valid shape: {result['valid_shape']}")
print(f"Final shape: {result['final_shape']}")
print(f"Scale factor: {result['scale']:.4f}")
print(f"Padding: {result['pad']}")
print(f"Tensor dtype: {result['resized'].dtype}")
print(f"Tensor shape: {result['resized'].shape}")
if __name__ == '__main__':
main()
脚本 2:空间感知混合增强管道
Python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
脚本名称: spatial_aware_augmentation.py
内容概述:
实现YOLO26的替代增强策略,包含自适应Mixup、泊松融合复制粘贴、
基于显著性的上下文裁剪。废弃传统Mosaic四图拼接,解决信息边界效应问题。
提供各增强阶段的可视化与边界框变换验证。
使用方式:
python spatial_aware_augmentation.py --dataset_path ./images --num_samples 4
"""
import cv2
import numpy as np
import torch
from typing import List, Tuple, Dict, Optional
from dataclasses import dataclass
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import random
import argparse
from scipy.ndimage import gaussian_filter
from scipy.sparse import csr_matrix
from scipy.sparse.linalg import spsolve
import os
@dataclass
class BoundingBox:
"""边界框数据结构,支持相对坐标与绝对坐标"""
x1: float
y1: float
x2: float
y2: float
class_id: int
confidence: float = 1.0
is_relative: bool = False # True表示0-1相对坐标
def to_absolute(self, img_w: int, img_h: int) -> 'BoundingBox':
"""转换为绝对坐标"""
if not self.is_relative:
return self
return BoundingBox(
self.x1 * img_w, self.y1 * img_h,
self.x2 * img_w, self.y2 * img_h,
self.class_id, self.confidence, False
)
def to_relative(self, img_w: int, img_h: int) -> 'BoundingBox':
"""转换为相对坐标"""
if self.is_relative:
return self
return BoundingBox(
self.x1 / img_w, self.y1 / img_h,
self.x2 / img_w, self.y2 / img_h,
self.class_id, self.confidence, True
)
def area(self) -> float:
return max(0, self.x2 - self.x1) * max(0, self.y2 - self.y1)
class PoissonBlender:
"""
泊松融合实现,实现前景与背景的无缝集成。
基于梯度域融合消除边界伪影。
"""
def __init__(self):
pass
def blend(self, foreground: np.ndarray, mask: np.ndarray,
background: np.ndarray, offset: Tuple[int, int]) -> np.ndarray:
"""
执行泊松融合。
Args:
foreground: 前景图像 (H, W, 3)
mask: 二值掩码 (H, W),255表示前景区域
background: 背景图像 (H, W, 3)
offset: (y, x) 前景在背景上的放置位置
Returns:
融合后的图像
"""
y_off, x_off = offset
fg_h, fg_w = foreground.shape[:2]
# 提取ROI
roi = background[y_off:y_off+fg_h, x_off:x_off+fg_w].copy()
# 构建掩码(0-1浮点)
mask_norm = mask.astype(np.float32) / 255.0
mask_3ch = np.stack([mask_norm] * 3, axis=-1)
# 简单实现:使用拉普拉斯金字塔或直接使用mask混合(简化版)
# 完整泊松融合需要求解大型稀疏矩阵,这里使用导向滤波近似
result = self._guided_blending(foreground, roi, mask_3ch)
# 放回原图
blended = background.copy()
blended[y_off:y_off+fg_h, x_off:x_off+fg_w] = result
return blended
def _guided_blending(self, fg: np.ndarray, bg: np.ndarray,
mask: np.ndarray, radius: int = 15) -> np.ndarray:
"""
基于导向滤波的快速融合近似泊松效果。
"""
# 在边界区域使用多频段混合
result = fg * mask + bg * (1 - mask)
# 对mask边缘进行高斯模糊实现平滑过渡
mask_blur = gaussian_filter(mask[:,:,0], sigma=radius/3)
mask_blur = np.clip(mask_blur * 3, 0, 1)
mask_blur = np.stack([mask_blur] * 3, axis=-1)
result = fg * mask_blur + bg * (1 - mask_blur)
return result.astype(np.uint8)
class SpatialAwareAugmenter:
"""
空间感知增强器,实现Mosaic替代策略。
包含动态Mixup、泊松复制粘贴、显著性裁剪。
"""
def __init__(self, target_size: int = 640,
mixup_alpha: float = 0.5,
copy_paste_prob: float = 0.3,
mixup_prob: float = 0.4,
crop_prob: float = 0.3):
self.target_size = target_size
self.mixup_alpha = mixup_alpha
self.copy_paste_prob = copy_paste_prob
self.mixup_prob = mixup_prob
self.crop_prob = crop_prob
self.blender = PoissonBlender()
def sample_beta(self) -> float:
"""采样Mixup系数"""
return np.random.beta(self.mixup_alpha, self.mixup_alpha)
def adaptive_mixup(self, img1: np.ndarray, boxes1: List[BoundingBox],
img2: np.ndarray, boxes2: List[BoundingBox]) -> Tuple[np.ndarray, List[BoundingBox]]:
"""
自适应Mixup增强,根据目标置信度融合两幅图像。
边界框采用并集策略保留。
"""
# 确保尺寸一致
h, w = img1.shape[:2]
img2 = cv2.resize(img2, (w, h))
# 采样混合系数
alpha = self.sample_beta()
# 图像混合
mixed_img = (alpha * img1 + (1 - alpha) * img2).astype(np.uint8)
# 边界框融合:保留两图的所有框,但调整置信度
mixed_boxes = []
# 第一图像的框(置信度加权)
for box in boxes1:
new_box = BoundingBox(
box.x1, box.y1, box.x2, box.y2,
box.class_id, box.confidence * alpha, box.is_relative
)
mixed_boxes.append(new_box)
# 第二图像的框
for box in boxes2:
# 转换到绝对坐标再缩放(如果是相对坐标)
abs_box = box.to_absolute(w, h)
new_box = BoundingBox(
abs_box.x1, abs_box.y1, abs_box.x2, abs_box.y2,
box.class_id, box.confidence * (1 - alpha), False
)
mixed_boxes.append(new_box)
return mixed_img, mixed_boxes
def copy_paste_poisson(self, background: np.ndarray, bg_boxes: List[BoundingBox],
foreground: np.ndarray, fg_boxes: List[BoundingBox],
fg_mask: np.ndarray) -> Tuple[np.ndarray, List[BoundingBox]]:
"""
基于泊松融合的复制粘贴增强。
随机选择粘贴位置,确保不超出边界且不过度重叠。
"""
bg_h, bg_w = background.shape[:2]
fg_h, fg_w = foreground.shape[:2]
# 计算有效放置范围(确保前景不超出背景)
max_y = max(1, bg_h - fg_h)
max_x = max(1, bg_w - fg_w)
# 随机选择位置(可改进为基于密度的智能放置)
y_offset = random.randint(0, max_y)
x_offset = random.randint(0, max_x)
# 执行泊松融合
result = self.blender.blend(foreground, fg_mask, background, (y_offset, x_offset))
# 更新边界框坐标
new_boxes = bg_boxes.copy()
for box in fg_boxes:
abs_box = box.to_absolute(fg_w, fg_h)
# 转换到背景坐标系
new_box = BoundingBox(
abs_box.x1 + x_offset, abs_box.y1 + y_offset,
abs_box.x2 + x_offset, abs_box.y2 + y_offset,
box.class_id, box.confidence, False
)
# 边界检查
if 0 <= new_box.x1 < bg_w and 0 <= new_box.y1 < bg_h:
new_box.x2 = min(new_box.x2, bg_w)
new_box.y2 = min(new_box.y2, bg_h)
new_boxes.append(new_box)
return result, new_boxes
def saliency_based_crop(self, img: np.ndarray, boxes: List[BoundingBox],
crop_ratio: float = 0.8) -> Tuple[np.ndarray, List[BoundingBox]]:
"""
基于显著性的上下文感知裁剪。
使用简单的颜色对比度近似显著性,保留关键区域。
"""
h, w = img.shape[:2]
crop_h, crop_w = int(h * crop_ratio), int(w * crop_ratio)
# 计算显著性图(基于亮度对比度)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (51, 51), 0)
saliency = cv2.absdiff(gray, blurred)
saliency = cv2.GaussianBlur(saliency, (21, 21), 0)
# 在边界框周围添加高显著性权重
mask = np.zeros_like(saliency)
for box in boxes:
abs_box = box.to_absolute(w, h)
x1, y1, x2, y2 = int(abs_box.x1), int(abs_box.y1), int(abs_box.x2), int(abs_box.y2)
mask[y1:y2, x1:x2] = 255
saliency = cv2.addWeighted(saliency, 0.5, mask, 0.5, 0)
# 计算积分图快速求和
integral = cv2.integral(saliency)
# 搜索最佳裁剪窗口(简化:在网格中搜索)
best_score = -1
best_y, best_x = 0, 0
steps = 10
for i in range(steps):
for j in range(steps):
y = int((h - crop_h) * i / (steps - 1)) if steps > 1 else 0
x = int((w - crop_w) * j / (steps - 1)) if steps > 1 else 0
# 计算窗口内显著性总和
y1, x1 = y, x
y2, x2 = min(y + crop_h, h), min(x + crop_w, w)
if y2 > y1 and x2 > x1:
score = integral[y2, x2] - integral[y1, x2] - integral[y2, x1] + integral[y1, x1]
if score > best_score:
best_score = score
best_y, best_x = y, x
# 执行裁剪
cropped = img[best_y:best_y+crop_h, best_x:best_x+crop_w]
# 调整边界框
new_boxes = []
for box in boxes:
abs_box = box.to_absolute(w, h)
new_x1 = max(0, abs_box.x1 - best_x)
new_y1 = max(0, abs_box.y1 - best_y)
new_x2 = min(crop_w, abs_box.x2 - best_x)
new_y2 = min(crop_h, abs_box.y2 - best_y)
if new_x2 > new_x1 and new_y2 > new_y1:
new_boxes.append(BoundingBox(
new_x1, new_y1, new_x2, new_y2,
box.class_id, box.confidence, False
))
# 调整到目标尺寸
cropped = cv2.resize(cropped, (self.target_size, self.target_size))
scale_x = self.target_size / crop_w
scale_y = self.target_size / crop_h
final_boxes = []
for box in new_boxes:
final_boxes.append(BoundingBox(
box.x1 * scale_x, box.y1 * scale_y,
box.x2 * scale_x, box.y2 * scale_y,
box.class_id, box.confidence, False
))
return cropped, final_boxes
def __call__(self, img: np.ndarray, boxes: List[BoundingBox]) -> Tuple[np.ndarray, List[BoundingBox], str]:
"""
随机选择增强策略并执行。
Returns:
增强后的图像、边界框、使用的策略名称
"""
r = random.random()
if r < self.mixup_prob:
# 需要第二幅图像,这里生成随机合成图像模拟
img2 = np.random.randint(0, 255, img.shape, dtype=np.uint8)
boxes2 = []
return *self.adaptive_mixup(img, boxes, img2, boxes2), "Mixup"
elif r < self.mixup_prob + self.copy_paste_prob:
# 模拟前景提取(实际应从数据集采样)
fg_h, fg_w = img.shape[0] // 2, img.shape[1] // 2
foreground = cv2.resize(img, (fg_w, fg_h))
fg_mask = np.ones((fg_h, fg_w), dtype=np.uint8) * 255
# 简化:使用当前图像的一部分作为前景
return *self.copy_paste_poisson(img, boxes, foreground, [], fg_mask), "Copy-Paste"
else:
ratio = random.uniform(0.5, 0.9)
return *self.saliency_based_crop(img, boxes, ratio), "Saliency-Crop"
def visualize_augmentation(self, img: np.ndarray, boxes: List[BoundingBox],
save_path: str = None):
"""
可视化三种增强策略的对比效果。
"""
strategies = ['Mixup', 'Copy-Paste', 'Saliency-Crop']
fig, axes = plt.subplots(2, 3, figsize=(18, 12))
# 原始图像
ax = axes[0, 0]
self._draw_boxes(ax, img, boxes, 'Original')
results = []
# Mixup
img2 = np.random.randint(0, 255, img.shape, dtype=np.uint8)
boxes2 = [BoundingBox(0.2, 0.2, 0.4, 0.4, 1, 0.9, True)]
mixed, mixed_boxes = self.adaptive_mixup(img, boxes, img2, boxes2)
ax = axes[0, 1]
self._draw_boxes(ax, mixed, mixed_boxes, 'Adaptive Mixup\n(Noise as sample)')
results.append((mixed, mixed_boxes))
# Copy-Paste
fg = cv2.resize(img, (img.shape[1]//2, img.shape[0]//2))
mask = np.ones(fg.shape[:2], dtype=np.uint8) * 255
pasted, pasted_boxes = self.copy_paste_poisson(img, boxes, fg, [], mask)
ax = axes[0, 2]
self._draw_boxes(ax, pasted, pasted_boxes, 'Poisson Copy-Paste')
results.append((pasted, pasted_boxes))
# Saliency-Crop
cropped, crop_boxes = self.saliency_based_crop(img, boxes, 0.8)
ax = axes[1, 0]
self._draw_boxes(ax, cropped, crop_boxes, 'Saliency-Based Crop\n(80% ratio)')
results.append((cropped, crop_boxes))
# 策略对比表
ax = axes[1, 1]
ax.axis('tight')
ax.axis('off')
table_data = [
['Strategy', 'Pros', 'Cons'],
['Mixup', 'Smooth transitions', 'Ghost artifacts'],
['Copy-Paste', 'New object combos', 'Context mismatch'],
['Saliency-Crop', 'Focus on ROI', 'Information loss']
]
table = ax.table(cellText=table_data, cellLoc='left', loc='center',
colWidths=[0.3, 0.35, 0.35])
table.auto_set_font_size(False)
table.set_fontsize(10)
table.scale(1, 2)
# 颜色标记
for i in range(3):
for j in range(3):
if i == 0:
table[(i, j)].set_facecolor('#4CAF50')
table[(i, j)].set_text_props(weight='bold', color='white')
else:
table[(i, j)].set_facecolor('#f0f0f0')
ax.set_title('Strategy Comparison', pad=20)
# 像素分布分析
ax = axes[1, 2]
original_hist = cv2.calcHist([img], [0], None, [256], [0, 256])
ax.plot(original_hist, label='Original', color='blue', linewidth=2)
for i, (res_img, _) in enumerate(results):
hist = cv2.calcHist([res_img], [0], None, [256], [0, 256])
ax.plot(hist, label=strategies[i], linestyle='--', alpha=0.7)
ax.set_xlabel('Pixel Intensity')
ax.set_ylabel('Frequency')
ax.set_title('Histogram Comparison')
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
if save_path:
plt.savefig(save_path, dpi=150, bbox_inches='tight')
print(f"Augmentation visualization saved to {save_path}")
else:
plt.show()
def _draw_boxes(self, ax, img: np.ndarray, boxes: List[BoundingBox], title: str):
"""辅助函数:绘制边界框"""
ax.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
h, w = img.shape[:2]
colors = plt.cm.tab10(np.linspace(0, 1, 10))
for box in boxes:
abs_box = box.to_absolute(w, h) if box.is_relative else box
rect = Rectangle(
(abs_box.x1, abs_box.y1),
abs_box.x2 - abs_box.x1,
abs_box.y2 - abs_box.y1,
linewidth=2, edgecolor=colors[box.class_id % 10],
facecolor='none', linestyle='-'
)
ax.add_patch(rect)
ax.text(abs_box.x1, abs_box.y1 - 5,
f'C{box.class_id}:{box.confidence:.2f}',
color=colors[box.class_id % 10], fontsize=9, weight='bold')
ax.set_title(title)
ax.axis('off')
def main():
parser = argparse.ArgumentParser(description='Spatial-Aware Augmentation Demo')
parser.add_argument('--target_size', type=int, default=640)
parser.add_argument('--save_path', type=str, default='augmentation_demo.png')
args = parser.parse_args()
# 生成测试图像
img = np.ones((800, 1200, 3), dtype=np.uint8) * 240 # 浅灰背景
# 绘制多个彩色矩形模拟目标
boxes = []
colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (255, 255, 0)]
positions = [(100, 100, 300, 250), (500, 150, 700, 400),
(200, 500, 450, 700), (800, 450, 1000, 650)]
for i, (x1, y1, x2, y2) in enumerate(positions):
cv2.rectangle(img, (x1, y1), (x2, y2), colors[i], -1)
# 转换为相对坐标
boxes.append(BoundingBox(
x1/1200, y1/800, x2/1200, y2/800,
class_id=i % 4, confidence=0.95, is_relative=True
))
# 添加一些纹理细节
noise = np.random.randint(-20, 20, img.shape, dtype=np.int16)
img = np.clip(img.astype(np.int16) + noise, 0, 255).astype(np.uint8)
# 初始化增强器
augmenter = SpatialAwareAugmenter(target_size=args.target_size)
# 可视化
augmenter.visualize_augmentation(img, boxes, save_path=args.save_path)
print(f"Demonstrated 3 augmentation strategies on synthetic image")
print(f"Original boxes: {len(boxes)}")
if __name__ == '__main__':
main()
脚本 3:混合精度训练数据格式规范
Python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
脚本名称: mixed_precision_formatter.py
内容概述:
实现YOLO26混合精度训练的数据格式规范,包含FP16/BF16转换、
动态损失缩放、数值稳定性校验、NHWC/NCHW布局转换。
提供梯度缩放可视化、数值分布直方图与内存占用分析。
使用方式:
python mixed_precision_formatter.py --batch_size 8 --channels 3 --height 640 --width 640
"""
import torch
import numpy as np
from typing import Tuple, Dict, Optional, Literal
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import argparse
import time
class MixedPrecisionFormatter:
"""
混合精度数据格式化器,实现FP16/BF16训练的数据预处理与数值稳定性控制。
包含动态损失缩放、溢出检测、内存对齐优化。
"""
def __init__(self,
dtype: Literal['fp16', 'bf16'] = 'fp16',
initial_scale: float = 65536.0,
growth_factor: float = 2.0,
backoff_factor: float = 0.5,
growth_interval: int = 2000,
enabled: bool = True):
"""
初始化混合精度格式化器。
Args:
dtype: 数据类型(fp16或bf16)
initial_scale: 初始损失缩放因子
growth_factor: 缩放增长系数
backoff_factor: 缩放回退系数
growth_interval: 连续有效步数后增长缩放
enabled: 是否启用混合精度
"""
self.dtype = dtype
self.enabled = enabled
if dtype == 'fp16':
self.target_dtype = torch.float16
self.eps = 2**(-14) # FP16最小正规格化数约6e-5
else: # bf16
self.target_dtype = torch.bfloat16
self.eps = 2**(-7) # BF16有更大的范围
# 损失缩放状态
self.scale = initial_scale
self.growth_factor = growth_factor
self.backoff_factor = backoff_factor
self.growth_interval = growth_interval
self._growth_tracker = 0
self._last_overflow_iter = -1
# 统计信息
self.stats = {
'overflow_count': 0,
'underflow_count': 0,
'scale_history': [initial_scale],
'value_range': []
}
def preprocess(self, tensor: torch.Tensor,
mean: Tuple[float, float, float] = (0.0, 0.0, 0.0),
std: Tuple[float, float, float] = (1.0, 1.0, 1.0)) -> torch.Tensor:
"""
预处理输入张量,执行归一化、格式转换与数值校验。
Args:
tensor: 输入图像张量 (N, C, H, W) 或 (N, H, W, C),值域[0, 255]
mean: 归一化均值
std: 归一化标准差
Returns:
处理后的半精度张量
"""
# 确保输入为浮点类型
if not torch.is_floating_point(tensor):
tensor = tensor.float()
# 归一化到[0, 1]
tensor = tensor / 255.0
# 执行ImageNet风格标准化(可选)
if mean != (0.0, 0.0, 0.0) or std != (1.0, 1.0, 1.0):
mean_tensor = torch.tensor(mean, device=tensor.device).view(1, 3, 1, 1)
std_tensor = torch.tensor(std, device=tensor.device).view(1, 3, 1, 1)
tensor = (tensor - mean_tensor) / std_tensor
# 数值截断防止极端值
tensor = torch.clamp(tensor, min=self.eps, max=1.0 - self.eps)
# 校验异常值
if torch.isnan(tensor).any() or torch.isinf(tensor).any():
print("Warning: Input contains NaN or Inf, replacing with zeros")
tensor = torch.where(torch.isnan(tensor) | torch.isinf(tensor),
torch.zeros_like(tensor), tensor)
# 转换为半精度
if self.enabled:
tensor = tensor.to(self.target_dtype)
return tensor
def compute_gradient_scale(self, loss: torch.Tensor) -> Tuple[torch.Tensor, float]:
"""
应用动态损失缩放,防止梯度下溢。
Args:
loss: 计算得到的损失值(FP32)
Returns:
缩放后的损失、当前缩放因子
"""
if not self.enabled or self.dtype == 'bf16':
# BF16不需要损失缩放(范围与FP32相同)
return loss, 1.0
# 应用缩放
scaled_loss = loss * self.scale
return scaled_loss, self.scale
def update_scale(self, found_inf: bool, iteration: int) -> float:
"""
根据是否发现溢出更新缩放因子。
Args:
found_inf: 是否发现无穷大梯度
iteration: 当前迭代次数
Returns:
新的缩放因子
"""
if not self.enabled or self.dtype == 'bf16':
return 1.0
if found_inf:
# 发现溢出,回退缩放
self.scale = max(self.scale * self.backoff_factor, 1.0)
self._growth_tracker = 0
self._last_overflow_iter = iteration
self.stats['overflow_count'] += 1
else:
# 未溢出,增长跟踪器
self._growth_tracker += 1
# 连续多次未溢出则增加缩放
if self._growth_tracker >= self.growth_interval:
self.scale = min(self.scale * self.growth_factor, 65536.0)
self._growth_tracker = 0
self.stats['scale_history'].append(self.scale)
return self.scale
def postprocess_gradients(self, grads: torch.Tensor) -> torch.Tensor:
"""
反向传播后取消梯度缩放。
Args:
grads: 梯度张量
Returns:
反缩放后的梯度
"""
if not self.enabled or self.dtype == 'bf16' or self.scale == 1.0:
return grads
return grads / self.scale
def align_batch_size(self, batch_size: int, alignment: int = 8) -> int:
"""
对齐批次大小以优化Tensor Core利用率。
CUDA Tensor Core通常要求维度为8的倍数。
"""
remainder = batch_size % alignment
if remainder == 0:
return batch_size
return batch_size + (alignment - remainder)
def create_aligned_batch(self, tensors: list, alignment: int = 8) -> torch.Tensor:
"""
创建内存对齐的批次张量,必要时填充虚拟样本。
"""
current_size = len(tensors)
aligned_size = self.align_batch_size(current_size, alignment)
# 获取张量形状
first_shape = tensors[0].shape
device = tensors[0].device
# 创建批次缓冲区
batch = torch.zeros((aligned_size, *first_shape),
dtype=self.target_dtype if self.enabled else torch.float32,
device=device)
# 填充真实数据
for i, t in enumerate(tensors):
batch[i] = t
# 记录有效掩码(用于后续忽略虚拟样本)
valid_mask = torch.ones(aligned_size, dtype=torch.bool, device=device)
if aligned_size > current_size:
valid_mask[current_size:] = False
return batch, valid_mask
def check_numerical_integrity(self, tensor: torch.Tensor) -> Dict:
"""
检查张量的数值完整性,统计异常值分布。
"""
stats = {
'has_nan': torch.isnan(tensor).any().item(),
'has_inf': torch.isinf(tensor).any().item(),
'max_value': tensor.max().item(),
'min_value': tensor.min().item(),
'mean_value': tensor.mean().item(),
'std_value': tensor.std().item(),
'zero_ratio': (tensor == 0).float().mean().item(),
'tiny_ratio': (tensor.abs() < self.eps).float().mean().item()
}
# 检测下溢
if stats['tiny_ratio'] > 0.01:
self.stats['underflow_count'] += 1
return stats
def visualize_precision_analysis(self, save_path: str = None):
"""
可视化混合精度训练的数值特性与内存分析。
"""
fig = plt.figure(figsize=(16, 12))
gs = fig.add_gridspec(3, 3, hspace=0.3, wspace=0.3)
# 1. 损失缩放历史
ax1 = fig.add_subplot(gs[0, :])
ax1.plot(self.stats['scale_history'], linewidth=2, color='#2196F3')
ax1.axhline(y=65536, color='r', linestyle='--', alpha=0.5, label='Max Scale')
ax1.axhline(y=1, color='g', linestyle='--', alpha=0.5, label='Min Scale')
ax1.set_xlabel('Update Step')
ax1.set_ylabel('Loss Scale Factor')
ax1.set_title('Dynamic Loss Scaling History')
ax1.legend()
ax1.grid(True, alpha=0.3)
ax1.set_yscale('log')
# 生成模拟数据用于可视化数值分布
np.random.seed(42)
fp32_data = np.random.randn(10000).astype(np.float32) * 0.5
fp16_data = fp32_data.astype(np.float16)
bf16_data = torch.tensor(fp32_data).to(torch.bfloat16).numpy()
# 2. FP32 vs FP16分布
ax2 = fig.add_subplot(gs[1, 0])
ax2.hist(fp32_data, bins=50, alpha=0.7, label='FP32', color='blue')
ax2.hist(fp16_data, bins=50, alpha=0.7, label='FP16', color='orange')
ax2.set_title('FP32 vs FP16 Distribution')
ax2.legend()
ax2.grid(True, alpha=0.3)
# 3. BF16分布(模拟)
ax3 = fig.add_subplot(gs[1, 1])
ax3.hist(fp32_data, bins=50, alpha=0.7, label='FP32', color='blue')
ax3.hist(bf16_data, bins=50, alpha=0.7, label='BF16', color='green')
ax3.set_title('FP32 vs BF16 Distribution')
ax3.legend()
ax3.grid(True, alpha=0.3)
# 4. 精度范围对比
ax4 = fig.add_subplot(gs[1, 2])
formats = ['FP32', 'FP16', 'BF16']
min_pos = [1.4e-45, 6e-8, 1.2e-38]
max_val = [3.4e38, 6.5e4, 3.4e38]
epsilon = [1.2e-7, 9.8e-4, 7.8e-3]
x = np.arange(len(formats))
width = 0.25
ax4.bar(x - width, np.log10(min_pos), width, label='Min Positive (log)', color='skyblue')
ax4.bar(x, np.log10(max_val), width, label='Max Value (log)', color='lightcoral')
ax4.bar(x + width, np.log10(epsilon), width, label='Epsilon (log)', color='lightgreen')
ax4.set_ylabel('Log10(Value)')
ax4.set_title('Numerical Range Comparison')
ax4.set_xticks(x)
ax4.set_xticklabels(formats)
ax4.legend()
ax4.grid(True, alpha=0.3, axis='y')
# 5. 内存占用分析(不同批次大小)
ax5 = fig.add_subplot(gs[2, 0])
batch_sizes = [1, 2, 4, 8, 16, 32, 64]
img_size = 640 * 640 * 3 # 3通道640x640
fp32_mem = [b * img_size * 4 / (1024**2) for b in batch_sizes] # MB
fp16_mem = [b * img_size * 2 / (1024**2) for b in batch_sizes]
bf16_mem = [b * img_size * 2 / (1024**2) for b in batch_sizes]
ax5.plot(batch_sizes, fp32_mem, 'o-', label='FP32', linewidth=2)
ax5.plot(batch_sizes, fp16_mem, 's-', label='FP16', linewidth=2)
ax5.plot(batch_sizes, bf16_mem, '^-', label='BF16', linewidth=2)
ax5.set_xlabel('Batch Size')
ax5.set_ylabel('Memory (MB)')
ax5.set_title('Memory Usage vs Batch Size')
ax5.legend()
ax5.grid(True, alpha=0.3)
# 6. Tensor Core对齐可视化
ax6 = fig.add_subplot(gs[2, 1])
tensor_dims = list(range(1, 33))
efficiency = [100 if d % 8 == 0 else (d % 8) / 8 * 100 for d in tensor_dims]
colors = ['#4CAF50' if e == 100 else '#FFC107' for e in efficiency]
bars = ax6.bar(tensor_dims, efficiency, color=colors, alpha=0.8)
ax6.axhline(y=100, color='g', linestyle='--', alpha=0.5)
ax6.set_xlabel('Dimension Size')
ax6.set_ylabel('Tensor Core Efficiency %')
ax6.set_title('Memory Alignment Efficiency (mod 8)')
ax6.set_xticks(range(0, 33, 4))
# 7. 数值密度热力图(模拟)
ax7 = fig.add_subplot(gs[2, 2])
# 创建模拟梯度分布热力图
gradient_magnitude = np.random.exponential(0.1, (16, 16))
gradient_magnitude[0, 0] = 1.0 # 确保颜色映射范围
im = ax7.imshow(gradient_magnitude, cmap='YlOrRd', aspect='auto')
plt.colorbar(im, ax=ax7, label='Gradient Magnitude')
ax7.set_title('Typical Gradient Distribution')
ax7.set_xlabel('Channel Index')
ax7.set_ylabel('Layer Index')
plt.suptitle('Mixed Precision Training Analysis Dashboard', fontsize=16, weight='bold')
if save_path:
plt.savefig(save_path, dpi=150, bbox_inches='tight')
print(f"Precision analysis saved to {save_path}")
else:
plt.show()
def benchmark_conversion(self, shape: Tuple[int, ...], iterations: int = 100) -> Dict:
"""
基准测试不同精度格式的转换速度。
"""
# 生成测试数据
data = torch.randn(shape, dtype=torch.float32, device='cpu')
results = {}
# FP32处理时间
start = time.time()
for _ in range(iterations):
_ = data * 1.0
results['fp32_time'] = (time.time() - start) / iterations * 1000 # ms
# FP16转换时间
start = time.time()
for _ in range(iterations):
_ = data.half()
results['fp16_convert'] = (time.time() - start) / iterations * 1000
# BF16转换时间
start = time.time()
for _ in range(iterations):
_ = data.bfloat16()
results['bf16_convert'] = (time.time() - start) / iterations * 1000
return results
def main():
parser = argparse.ArgumentParser(description='Mixed Precision Data Formatting Demo')
parser.add_argument('--batch_size', type=int, default=8)
parser.add_argument('--channels', type=int, default=3)
parser.add_argument('--height', type=int, default=640)
parser.add_argument('--width', type=int, default=640)
parser.add_argument('--dtype', type=str, default='fp16', choices=['fp16', 'bf16'])
parser.add_argument('--save_path', type=str, default='mixed_precision_analysis.png')
args = parser.parse_args()
print(f"Initializing Mixed Precision Formatter (dtype={args.dtype})")
formatter = MixedPrecisionFormatter(dtype=args.dtype)
# 生成模拟批次数据
print(f"Generating synthetic batch: {args.batch_size}×{args.channels}×{args.height}×{args.width}")
batch = torch.randint(0, 256, (args.batch_size, args.channels, args.height, args.width),
dtype=torch.float32)
# 预处理
print("Preprocessing batch...")
processed = formatter.preprocess(batch, mean=(0.485, 0.456, 0.406),
std=(0.229, 0.224, 0.225))
print(f"Processed dtype: {processed.dtype}")
print(f"Value range: [{processed.min():.4f}, {processed.max():.4f}]")
# 数值完整性检查
integrity = formatter.check_numerical_integrity(processed)
print(f"Numerical integrity: NaN={integrity['has_nan']}, Inf={integrity['has_inf']}")
print(f"Statistics: mean={integrity['mean_value']:.4f}, std={integrity['std_value']:.4f}")
# 模拟训练循环中的损失缩放
print("\nSimulating training loop with gradient scaling...")
for i in range(10):
# 模拟损失
loss = torch.randn(1) * 0.1
# 应用缩放
scaled_loss, current_scale = formatter.compute_gradient_scale(loss)
# 模拟溢出检测(随机)
overflow = (i % 5 == 0 and i > 0) # 每5次模拟一次溢出
# 更新缩放
new_scale = formatter.update_scale(overflow, i)
print(f"Iter {i}: Loss={loss.item():.4f}, Scale={current_scale:.1f}, "
f"Overflow={overflow}, New Scale={new_scale:.1f}")
# 对齐测试
print(f"\nBatch alignment test:")
for bs in [5, 7, 8, 9, 15, 16]:
aligned = formatter.align_batch_size(bs)
print(f" Original: {bs}, Aligned: {aligned}, Padding: {aligned-bs}")
# 基准测试
print("\nRunning benchmark...")
bench_results = formatter.benchmark_conversion((args.batch_size, args.channels,
args.height, args.width))
print(f"FP32 processing: {bench_results['fp32_time']:.2f} ms")
print(f"FP16 conversion: {bench_results['fp16_convert']:.2f} ms")
print(f"BF16 conversion: {bench_results['bf16_convert']:.2f} ms")
# 可视化
print(f"\nGenerating visualization...")
formatter.visualize_precision_analysis(save_path=args.save_path)
print(f"\nStatistics Summary:")
print(f"Overflow count: {formatter.stats['overflow_count']}")
print(f"Underflow count: {formatter.stats['underflow_count']}")
print(f"Final scale: {formatter.scale:.2f}")
if __name__ == '__main__':
main()
230



被折叠的 条评论
为什么被折叠?



