目录
第八章 评估指标、基准测试与实验分析
8.1 评估指标体系
评估指标体系为抓取算法提供量化的性能度量与诊断工具。一套完整的评估体系需覆盖任务完成度、物理合理性、几何优化度、计算效率四个维度。仿真环境评估指标关注策略在虚拟环境中的执行可靠性；几何质量指标刻画抓取姿态的内在优化特性；计算效率指标衡量算法的部署可行性。
8.1.1 仿真环境评估指标
仿真环境评估在物理引擎中执行抓取动作序列，通过二进制或连续指标判断任务成功与否。评估过程标准化物体初始姿态、干扰施加方式与成功判定阈值，确保不同算法间结果的可比性。
8.1.1.1 抓取成功率
抓取成功率 (Grasp Success Rate) 定义为成功抓取次数与总尝试次数的比值，是抓取系统最核心的性能指标。成功判定需通过两阶段测试验证：抬升测试验证静态稳定性，抖动测试验证动态鲁棒性。
抬升测试 (Lift Test) 要求机械臂在抓取闭合后将物体抬升至目标高度 $h_{target}$ 并保持时间 $t_{hold}$ 。成功条件表述为：
$$p_{obj}(t) \cdot e_z \geq h_{target}, \quad \forall t \in [t_{lift}, t_{lift}+t_{hold}]$$
其中 $p_{obj}(t)$ 为物体质心位置，$e_z$ 为垂直方向单位向量。测试排除物体与支撑面接触导致的虚假支撑效应。
抖动测试 (Shake Test) 在物体抬升后施加外部扰动验证抓取稳定性。扰动形式包括正弦振动 $a(t) = A\sin(2\pi ft)n$ 与随机冲击 $J = \Delta p \cdot m_{obj}$ 。成功条件要求物体在扰动期间相对手爪保持位姿约束 $\|p_{rel}(t)\| < \epsilon_{pos}$ 与 $\|R_{rel}(t) - I\|_F < \epsilon_{rot}$ 。通过两阶段测试的抓取记为成功，否则记为失败。
8.1.1.2 碰撞率与穿透深度
碰撞率 (Collision Rate) 量化抓取过程中手爪与环境的非预期接触频率，定义为发生碰撞的抓取次数占总抓取次数的比例。碰撞检测采用连续碰撞检测 (CCD) 算法，识别手指连杆与物体、环境障碍物的穿透事件。
穿透深度 (Penetration Depth, PD) 量化碰撞的严重程度，定义为两个相交几何体间的最小平移距离使二者分离：
$$d_{PD}(A,B) = \min_{\|d\|} \{\|d\| : (A \oplus d) \cap B = \emptyset\}$$
其中 $\oplus$ 表示闵可夫斯基和。平均穿透深度 $\bar{d}_{PD}$ 反映抓取规划的保守性，最大穿透深度 $d_{PD}^{max}$ 识别灾难性碰撞。零碰撞率与零穿透深度是仿真到现实迁移的必要条件。
8.1.2 几何质量指标
几何质量指标在物理仿真前评估抓取姿态的理论品质，无需执行完整动力学仿真，适用于大规模抓取候选筛选。
8.1.2.1 抓取椭球体积
抓取椭球 (Grasp Ellipsoid) 建模接触点处的力传递能力。设接触点集 $\{c_i\}_{i=1}^n$ 与接触法向 $\{n_i\}_{i=1}^n$ ，抓取矩阵 $G$ 的列空间构成 wrench 空间中的可达范围。对 $GG^T$ 进行特征值分解：
$$GG^T = V\Lambda V^T$$
特征值 $\lambda_1 \geq \lambda_2 \geq \dots \geq \lambda_6$ 描述各方向的力/力矩传递能力。抓取椭球体积定义为特征值的几何平均：
$$V_{grasp} = \sqrt[6]{\prod_{i=1}^6 \lambda_i}$$
体积越大表明抓取在各方向上的力传递能力越均衡，各向异性抓取在某些方向上存在力传递缺陷。
8.1.2.2 力封闭性度量
力封闭 (Force Closure, FC) 判定抓取是否能够通过接触力平衡任意外部 wrench。摩擦锥约束下的力封闭条件要求抓取矩阵 $G$ 的列向量张成的锥包含原点在其内部。定量度量采用最小 wrench 残差 (Minimum Wrench Residual)：
$$\epsilon_{FC} = \min_f \|w_{ext} - Gf\|_2 \quad \text{s.t. } f_i \in FC_i$$
或基于凸包体积的封闭性指标：
$$\eta_{FC} = \frac{\text{Vol}(B)}{\text{Vol}(\text{conv}(\{g_i\}))}$$
其中 $g_i$ 为抓取矩阵列向量，$B$ 为单位球。$\eta_{FC} > 0$ 表明力封闭成立，值越大封闭性越强。
8.1.2.3 抓取稳定性指标
抓取稳定性衡量系统抵抗扰动的能力，与李雅普诺夫稳定性理论关联。质心偏移指标 (Center of Mass Offset) 计算接触力合力作用线与物体质心的距离：
$$d_{COM} = \|(c_{avg} - p_{COM}) \times n_{avg}\|$$
其中 $c_{avg} = \frac{1}{n}\sum_i c_i$ 。最小稳定性指标 (Minimum Stability Margin) 定义为摩擦系数的安全裕度：
$$\mu_{margin} = \min_i \frac{\|f_{n,i}\|}{\mu_i\|f_{n,i}\| - \|f_{t,i}\|}$$
正值表明当前摩擦系数足以维持抓取，负值预示滑移风险。
8.2 基准测试对比实验
基准测试通过标准化数据集与实验协议，对比不同算法在相同条件下的性能差异。对比实验设计需覆盖方法论的多样性，确保结论的普遍适用性。
8.2.1 对比算法选择
对比算法的选择体现抓取方法的发展谱系。传统分析方法基于几何约束与力学分析，提供可解释性强但适应性有限的基线；深度学习方法利用数据驱动特征提取，在复杂场景中表现优越；强化学习方法通过试错学习优化策略，展现端到端决策能力。
-
传统分析方法：包括基于形状原语 (Primitive-based) 的抓取规划、力封闭判定算法 (如 Ferrari-Canny 指标计算)、弹簧阻尼模型驱动的接触分析。这些方法依赖人工设计的启发式规则，计算效率高但泛化能力受限。
-
主流深度学习方法：涵盖卷积神经网络 (CNN) 抓取检测、点云网络 (PointNet/PointNet++) 姿态估计、生成模型 (VAE/GAN) 抓取采样。此类方法通过大规模数据集训练学习视觉-抓取映射，在已知物体类别上精度高但数据依赖性强。
-
强化学习方法：包括深度 Q 网络 (DQN)、策略梯度方法 (PPO/TRPO)、 actor-critic 架构。这些方法在仿真环境中通过交互学习最优控制策略，适应动态环境但样本效率低、训练不稳定。
8.2.2 实验结果深度分析
实验结果分析超越单一成功率指标，深入探讨算法在不同条件下的性能差异与失效模式。
8.2.2.1 可见物体vs未见物体性能对比
可见物体 (Seen Objects) 指训练数据中出现的物体实例，未见物体 (Unseen Objects) 指全新几何形状与类别。泛化差距 (Generalization Gap) 定义为：
$$\Delta_{gen} = SR_{seen} - SR_{unseen}$$
其中 $SR$ 为成功率。深度学习方法通常表现出 $\Delta_{gen} > 0$ 的显著差距，表明过拟合于训练集几何特征；基于几何分析的方法差距较小但绝对性能较低。分析形状复杂度 (如物体表面积与体积比) 与泛化性能的相关性，可识别算法的几何偏好。
8.2.2.2 不同物体类别的性能差异
物体类别按几何特性分为规则形状 (立方体、圆柱体)、曲面形状 (球体、椭球体)、复杂形状 (玩具、工具)。类别性能差异分析采用分层采样确保类别平衡。细长物体 (高长径比) 与扁平物体 (低高径比) 通常构成挑战性案例，要求算法处理退化接触几何。
8.2.2.3 计算效率对比
8.2.2.3.1 推理时间
推理时间 (Inference Time) $t_{inf}$ 测量从传感器输入到抓取动作输出的端到端延迟，包含前处理 (Preprocessing)、网络前向传播 (Forward Pass)、后处理 (Postprocessing)。实时性要求 $t_{inf} < 100\text{ms}$ 适用于动态抓取，离线规划可容忍秒级延迟。批处理吞吐量 (Throughput) $T = N/t_{total}$ 评估并行处理能力。
8.2.2.3.2 显存占用
显存占用 (GPU Memory Usage) $M_{GPU}$ 限制模型规模与批量大小。峰值显存需求由模型参数、激活值、优化器状态共同决定。内存-精度权衡曲线 (Memory-Accuracy Tradeoff) 分析模型剪枝与量化对性能的影响，指导边缘设备部署。
8.3 消融实验设计
消融实验 (Ablation Study) 通过系统性移除或修改算法组件，验证各模块对整体性能的贡献度，识别关键设计选择与冗余计算。
8.3.1 网络模块有效性验证
8.3.1.1 关键模块移除影响
关键模块移除 (Component Removal) 逐一删除网络中的特征提取器、注意力机制、接触预测头等组件，测量性能衰减。衰减量 $\Delta_{module} = SR_{full} - SR_{removed}$ 量化模块重要性。若 $\Delta_{module} \approx 0$ 表明模块冗余或设计低效；若 $\Delta_{module} > \sigma_{stat}$ 表明模块关键。
8.3.1.2 损失函数权重敏感性分析
多任务学习中的损失函数 $L_{total} = \sum_i \lambda_i L_i$ 包含权重系数 $\lambda_i$ 。敏感性分析通过网格搜索或随机采样探索权重空间 $\{\lambda_i\}$ ，绘制帕累托前沿 (Pareto Frontier) 展示任务间的权衡关系。权重稳定性区域 (Stable Region) 指示训练鲁棒的超参数配置。
8.3.2 数据规模影响分析
8.3.2.1 训练数据量与性能曲线
学习曲线 (Learning Curve) 绘制训练集规模 $N_{train}$ 与验证性能的关系，采用对数坐标 $N_{train} \in [10^2, 10^6]$ 。曲线形态 (对数线性、幂律衰减、饱和) 揭示算法的样本复杂度。数据效率 (Data Efficiency) 定义为达到目标性能所需的最小样本量，高效算法在 $N_{train} < 10^4$ 时即收敛。
8.3.2.2 数据增强策略效果
数据增强 (Data Augmentation) 在训练阶段引入随机变换扩充有效数据量。增强策略对比包括：几何增强 (旋转、缩放、镜像)、像素增强 (亮度、对比度、噪声)、域随机化 (材质、光照、物理参数)。增强有效性通过增强系数 $\alpha$ 与性能增益 $G(\alpha) = SR(\alpha) - SR(\alpha=0)$ 量化，识别过增强导致的域漂移风险。
script_grasp_success_metrics.py
该脚本实现抓取成功率的自动化评估，包含抬升测试高度验证、抖动测试扰动施加、两阶段成功判定逻辑。可视化展示成功率统计、测试阶段时序分析与失败案例分类。
Python
#!/usr/bin/env python3
"""
Grasp Success Metrics: Lift and Shake Test Implementation
Content: 8.1.1.1 Grasp Success Rate with Lift and Shake Standards
Usage: python script_grasp_success_metrics.py
Output: grasp_success_metrics.png, test_phases.png
"""
import numpy as np
import matplotlib.pyplot as plt
from dataclasses import dataclass
from typing import List, Tuple
import matplotlib.patches as mpatches
@dataclass
class GraspTestConfig:
"""Configuration for grasp success testing."""
lift_height: float = 0.3 # meters
hold_time: float = 2.0 # seconds
shake_amplitude: float = 0.05 # meters
shake_freq: float = 5.0 # Hz
position_threshold: float = 0.02 # meters
rotation_threshold: float = 0.1 # radians
dt: float = 0.01
class GraspSimulator:
"""
Simulates grasp execution with physics for testing.
"""
def __init__(self, config: GraspTestConfig):
self.config = config
def simulate_lift_test(self, grasp_quality: float,
object_mass: float = 0.5) -> Tuple[bool, dict]:
"""
Simulate lift test phase.
Args:
grasp_quality: Quality score [0, 1], affects slip probability
object_mass: Object mass in kg
Returns:
success: Whether lift test passed
metrics: Dictionary of test metrics
"""
# Simulate lifting trajectory
t_lift = 1.0 # Time to lift
t_hold = self.config.hold_time
total_steps = int((t_lift + t_hold) / self.config.dt)
# Success probability based on grasp quality and mass
base_success_prob = grasp_quality * (1 - object_mass / 2.0)
success_prob = max(0.1, min(0.95, base_success_prob))
# Simulate with noise
success = np.random.random() < success_prob
# Generate trajectory data
heights = []
for i in range(total_steps):
t = i * self.config.dt
if t < t_lift:
# Lifting phase
h = (t / t_lift) * self.config.lift_height
else:
# Holding phase
if success:
h = self.config.lift_height + np.random.randn() * 0.005
else:
# Slipping
slip_prob = 0.1 * (t - t_lift)
if np.random.random() < slip_prob:
h = max(0, h - 0.01)
heights.append(h)
# Check if maintained height
hold_start = int(t_lift / self.config.dt)
hold_heights = heights[hold_start:]
min_height = min(hold_heights) if hold_heights else 0
passed = min_height >= self.config.lift_height * 0.9
metrics = {
'max_height': max(heights),
'min_hold_height': min_height,
'final_height': heights[-1],
'trajectory': heights
}
return passed, metrics
def simulate_shake_test(self, grasp_quality: float,
previous_success: bool) -> Tuple[bool, dict]:
"""
Simulate shake test phase with vibrations.
Args:
grasp_quality: Quality score
previous_success: Whether lift test passed
Returns:
success: Whether shake test passed
metrics: Test metrics
"""
if not previous_success:
return False, {'reason': 'lift_test_failed'}
t_shake = 3.0 # 3 seconds of shaking
steps = int(t_shake / self.config.dt)
# Generate shaking trajectory
t_vals = np.arange(0, t_shake, self.config.dt)
shake_offset = self.config.shake_amplitude * np.sin(2 * np.pi * self.config.shake_freq * t_vals)
# Relative position (object vs gripper)
# Good grasp: small relative motion
# Bad grasp: large relative motion or drop
max_allowed_rel = self.config.position_threshold
rel_positions = []
drop_occurred = False
for i, t in enumerate(t_vals):
# Base relative position
if grasp_quality > 0.7:
rel_pos = np.random.randn() * 0.005
else:
rel_pos = np.random.randn() * 0.02 * (1 - grasp_quality)
# Add cumulative slip for bad grasps
if grasp_quality < 0.5:
slip = 0.001 * i * (0.5 - grasp_quality)
rel_pos += slip
rel_positions.append(rel_pos)
if abs(rel_pos) > max_allowed_rel * 2: # Drop threshold
drop_occurred = True
break
success = not drop_occurred and max(abs(np.array(rel_positions))) < max_allowed_rel
metrics = {
'max_relative_pos': max(abs(np.array(rel_positions))),
'drop_occurred': drop_occurred,
'shake_trajectory': shake_offset[:len(rel_positions)],
'relative_trajectory': rel_positions
}
return success, metrics
def evaluate_grasp(self, grasp_quality: float, object_mass: float = 0.5) -> dict:
"""
Full two-stage evaluation.
"""
lift_success, lift_metrics = self.simulate_lift_test(grasp_quality, object_mass)
shake_success, shake_metrics = self.simulate_shake_test(grasp_quality, lift_success)
overall_success = lift_success and shake_success
return {
'success': overall_success,
'lift_success': lift_success,
'shake_success': shake_success,
'lift_metrics': lift_metrics,
'shake_metrics': shake_metrics,
'grasp_quality': grasp_quality
}
class GraspEvaluator:
"""
Evaluates multiple grasps and aggregates statistics.
"""
def __init__(self, config: GraspTestConfig):
self.config = config
self.simulator = GraspSimulator(config)
self.results = []
def evaluate_batch(self, grasp_qualities: List[float],
object_masses: List[float]) -> dict:
"""
Evaluate batch of grasps.
"""
self.results = []
for quality, mass in zip(grasp_qualities, object_masses):
result = self.simulator.evaluate_grasp(quality, mass)
self.results.append(result)
return self._compute_statistics()
def _compute_statistics(self) -> dict:
"""Compute aggregate statistics."""
n_total = len(self.results)
n_success = sum(1 for r in self.results if r['success'])
n_lift_only = sum(1 for r in self.results if r['lift_success'] and not r['shake_success'])
n_lift_fail = sum(1 for r in self.results if not r['lift_success'])
success_rate = n_success / n_total if n_total > 0 else 0
# Quality correlation
qualities = [r['grasp_quality'] for r in self.results]
successes = [r['success'] for r in self.results]
return {
'success_rate': success_rate,
'lift_only_rate': n_lift_only / n_total,
'lift_fail_rate': n_lift_fail / n_total,
'total_tests': n_total,
'results': self.results
}
def visualize_success_metrics():
"""Visualize grasp success testing results."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
config = GraspTestConfig()
evaluator = GraspEvaluator(config)
# Generate test data with varying quality
np.random.seed(42)
n_tests = 100
qualities = np.random.beta(2, 5, n_tests) # Skewed distribution
masses = np.random.uniform(0.1, 1.0, n_tests)
stats = evaluator.evaluate_batch(qualities, masses)
# Plot 1: Success rate by quality bin
ax = axes[0, 0]
bins = np.linspace(0, 1, 11)
bin_centers = (bins[:-1] + bins[1:]) / 2
success_rates = []
for i in range(len(bins)-1):
mask = (qualities >= bins[i]) & (qualities < bins[i+1])
if mask.sum() > 0:
sr = sum(1 for j in range(n_tests) if mask[j] and stats['results'][j]['success']) / mask.sum()
success_rates.append(sr)
else:
success_rates.append(0)
ax.bar(bin_centers, success_rates, width=0.08, alpha=0.7, color='green', edgecolor='black')
ax.plot([0, 1], [0, 1], 'r--', label='Perfect correlation')
ax.set_xlabel('Grasp Quality Score')
ax.set_ylabel('Success Rate')
ax.set_title('Success Rate vs Grasp Quality')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 2: Test phase breakdown
ax = axes[0, 1]
categories = ['Full Success', 'Lift Only', 'Lift Failed']
values = [
stats['success_rate'],
stats['lift_only_rate'],
stats['lift_fail_rate']
]
colors = ['green', 'yellow', 'red']
bars = ax.bar(categories, values, color=colors, alpha=0.7, edgecolor='black')
ax.set_ylabel('Proportion')
ax.set_title('Test Phase Outcomes')
ax.set_ylim(0, 1)
# Add value labels
for bar, val in zip(bars, values):
height = bar.get_height()
ax.text(bar.get_x() + bar.get_width()/2., height,
f'{val:.2%}', ha='center', va='bottom')
# Plot 3: Sample trajectory (successful)
ax = axes[1, 0]
success_results = [r for r in stats['results'] if r['success']]
if success_results:
sample = success_results[0]
traj = sample['lift_metrics']['trajectory']
t_vals = np.arange(len(traj)) * config.dt
ax.plot(t_vals, traj, 'b-', linewidth=2, label='Height')
ax.axhline(y=config.lift_height, color='g', linestyle='--', alpha=0.5, label='Target')
ax.axvspan(1.0, 1.0 + config.hold_time, alpha=0.2, color='gray', label='Hold phase')
ax.set_xlabel('Time (s)')
ax.set_ylabel('Height (m)')
ax.set_title('Successful Lift Test Trajectory')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 4: Sample trajectory (failed shake)
ax = axes[1, 1]
shake_fails = [r for r in stats['results'] if r['lift_success'] and not r['shake_success']]
if shake_fails:
sample = shake_fails[0]
rel_traj = sample['shake_metrics']['relative_trajectory']
t_vals = np.arange(len(rel_traj)) * config.dt
ax.plot(t_vals, rel_traj, 'r-', linewidth=2, label='Relative Position')
ax.axhline(y=config.position_threshold, color='orange', linestyle='--',
alpha=0.5, label='Threshold')
ax.axhline(y=-config.position_threshold, color='orange', linestyle='--', alpha=0.5)
ax.set_xlabel('Time (s)')
ax.set_ylabel('Relative Position (m)')
ax.set_title('Failed Shake Test (Relative Motion)')
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('grasp_success_metrics.png', dpi=150, bbox_inches='tight')
print("Saved grasp_success_metrics.png")
def visualize_test_standards():
"""Visualize test standards and thresholds."""
fig, axes = plt.subplots(1, 2, figsize=(14, 6))
# Plot 1: Lift test decision boundary
ax = axes[0]
qualities = np.linspace(0, 1, 100)
masses = np.linspace(0, 2, 100)
Q, M = np.meshgrid(qualities, masses)
# Success probability model
success_prob = Q * (1 - M/2.5)
success_prob = np.clip(success_prob, 0, 1)
contour = ax.contourf(Q, M, success_prob, levels=20, cmap='RdYlGn')
plt.colorbar(contour, ax=ax, label='Success Probability')
ax.contour(Q, M, success_prob, levels=[0.5], colors='black', linewidths=2)
ax.set_xlabel('Grasp Quality')
ax.set_ylabel('Object Mass (kg)')
ax.set_title('Lift Test Success Probability')
# Plot 2: Shake test amplitude spectrum
ax = axes[1]
frequencies = np.linspace(1, 20, 100)
amplitudes = np.linspace(0, 0.1, 100)
F, A = np.meshgrid(frequencies, amplitudes)
# Danger zone: high freq + high amplitude
danger = (F > 5) & (A > 0.03)
ax.contourf(F, A, danger.astype(float), levels=[0, 0.5, 1],
colors=['green', 'red'], alpha=0.3)
ax.contour(F, A, danger.astype(float), levels=[0.5], colors='red', linewidths=2)
# Mark standard test point
ax.plot(5, 0.05, 'b*', markersize=15, label='Standard Test')
ax.set_xlabel('Frequency (Hz)')
ax.set_ylabel('Amplitude (m)')
ax.set_title('Shake Test Parameter Space')
ax.legend()
plt.tight_layout()
plt.savefig('test_standards.png', dpi=150, bbox_inches='tight')
print("Saved test_standards.png")
if __name__ == "__main__":
print("Generating grasp success metrics visualization...")
visualize_success_metrics()
print("\nGenerating test standards visualization...")
visualize_test_standards()
print("\nCompleted. Check output PNG files.")script_collision_penetration.py
该脚本实现碰撞检测统计与穿透深度计算，包含连续碰撞检测模拟、穿透深度分布分析、碰撞率时序统计。可视化展示碰撞热力图、穿透深度分布与碰撞-成功率权衡曲线。
Python
#!/usr/bin/env python3
"""
Collision Rate and Penetration Depth Analysis
Content: 8.1.1.2 Collision Rate and Penetration Depth
Usage: python script_collision_penetration.py
Output: collision_analysis.png, penetration_depth.png
"""
import numpy as np
import matplotlib.pyplot as plt
from dataclasses import dataclass
from typing import List, Tuple
from scipy.spatial import ConvexHull
import matplotlib.patches as patches
@dataclass
class CollisionConfig:
"""Configuration for collision analysis."""
max_penetration_threshold: float = 0.01 # meters (1cm)
collision_tolerance: float = 0.001 # meters (1mm)
class GeometryUtils:
"""Utility functions for geometry calculations."""
@staticmethod
def minkowski_sum(A: np.ndarray, B: np.ndarray) -> np.ndarray:
"""
Compute Minkowski sum of two point sets.
"""
# Simplified: return combined hull points
return np.vstack([A + b for b in B])
@staticmethod
def penetration_depth(poly1: np.ndarray, poly2: np.ndarray) -> Tuple[float, np.ndarray]:
"""
Estimate penetration depth using support mapping.
Simplified 2D implementation using GJK-like approach.
Returns:
depth: Minimum translation distance
direction: Translation direction
"""
# Simple implementation: distance between centroids minus radii
c1 = np.mean(poly1, axis=0)
c2 = np.mean(poly2, axis=0)
# Approximate "radius" as max distance from centroid
r1 = max(np.linalg.norm(p - c1) for p in poly1)
r2 = max(np.linalg.norm(p - c2) for p in poly2)
dist = np.linalg.norm(c1 - c2)
if dist > r1 + r2:
return 0.0, np.zeros(2)
depth = (r1 + r2) - dist
direction = (c1 - c2) / (dist + 1e-8)
return depth, direction
class CollisionDetector:
"""
Continuous collision detection and penetration analysis.
"""
def __init__(self, config: CollisionConfig):
self.config = config
self.collision_history = []
self.penetration_history = []
def check_collision_aabb(self, obj1_min: np.ndarray, obj1_max: np.ndarray,
obj2_min: np.ndarray, obj2_max: np.ndarray) -> bool:
"""
Broad phase: AABB collision check.
"""
return (obj1_min[0] <= obj2_max[0] and obj1_max[0] >= obj2_min[0] and
obj1_min[1] <= obj2_max[1] and obj1_max[1] >= obj2_min[1])
def check_collision_continuous(self, traj1: List[np.ndarray],
traj2: List[np.ndarray]) -> List[Tuple[int, float]]:
"""
Check collisions along trajectories.
Args:
traj1: List of object polygons at each timestep
traj2: List of hand/finger polygons at each timestep
Returns:
collisions: List of (timestep, penetration_depth)
"""
collisions = []
for t, (poly1, poly2) in enumerate(zip(traj1, traj2)):
# Quick AABB check first
min1, max1 = np.min(poly1, axis=0), np.max(poly1, axis=0)
min2, max2 = np.min(poly2, axis=0), np.max(poly2, axis=0)
if self.check_collision_aabb(min1, max1, min2, max2):
# Detailed check
depth, _ = GeometryUtils.penetration_depth(poly1, poly2)
if depth > self.config.collision_tolerance:
collisions.append((t, depth))
self.penetration_history.append(depth)
self.collision_history.extend([c[0] for c in collisions])
return collisions
def compute_statistics(self) -> dict:
"""Compute collision statistics."""
if not self.penetration_history:
return {
'collision_rate': 0.0,
'max_penetration': 0.0,
'mean_penetration': 0.0,
'severe_collisions': 0
}
return {
'collision_rate': len(self.collision_history) / max(1, len(self.penetration_history)),
'max_penetration': max(self.penetration_history),
'mean_penetration': np.mean(self.penetration_history),
'severe_collisions': sum(1 for p in self.penetration_history
if p > self.config.max_penetration_threshold)
}
class GraspCollisionAnalyzer:
"""
Analyze collisions in grasp executions.
"""
def __init__(self, config: CollisionConfig):
self.config = config
self.detector = CollisionDetector(config)
def simulate_grasp_trajectory(self, grasp_quality: float,
object_poly: np.ndarray,
n_steps: int = 100) -> dict:
"""
Simulate grasp approach and compute collisions.
"""
# Generate finger trajectory approaching object
finger_traj = []
obj_traj = []
# Approach from above
start_pos = np.array([0, 0.5])
end_pos = np.array([0, 0.1])
for i in range(n_steps):
t = i / n_steps
finger_pos = start_pos + (end_pos - start_pos) * t
# Add noise based on grasp quality
noise = (1 - grasp_quality) * 0.05 * np.random.randn(2)
finger_pos += noise
# Simple finger shape (rectangle)
finger_poly = np.array([
finger_pos + [-0.02, 0],
finger_pos + [0.02, 0],
finger_pos + [0.02, 0.05],
finger_pos + [-0.02, 0.05]
])
finger_traj.append(finger_poly)
obj_traj.append(object_poly) # Object stationary
# Detect collisions
collisions = self.detector.check_collision_continuous(obj_traj, finger_traj)
stats = self.detector.compute_statistics()
return {
'collisions': collisions,
'statistics': stats,
'trajectory': finger_traj,
'final_stats': stats
}
def visualize_collision_analysis():
"""Visualize collision rate analysis."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
config = CollisionConfig()
analyzer = GraspCollisionAnalyzer(config)
# Generate data for different grasp qualities
qualities = np.linspace(0.3, 1.0, 20)
collision_rates = []
max_penetrations = []
mean_penetrations = []
object_poly = np.array([[-0.05, 0], [0.05, 0], [0.05, 0.1], [-0.05, 0.1]])
np.random.seed(42)
for q in qualities:
results = []
for _ in range(50): # Multiple trials per quality
res = analyzer.simulate_grasp_trajectory(q, object_poly, n_steps=50)
results.append(res['final_stats'])
collision_rates.append(np.mean([r['collision_rate'] for r in results]))
max_penetrations.append(np.mean([r['max_penetration'] for r in results]))
mean_penetrations.append(np.mean([r['mean_penetration'] for r in results]))
# Plot 1: Collision rate vs grasp quality
ax = axes[0, 0]
ax.plot(qualities, collision_rates, 'b-o', linewidth=2, markersize=6)
ax.set_xlabel('Grasp Quality')
ax.set_ylabel('Collision Rate')
ax.set_title('Collision Rate vs Grasp Quality')
ax.grid(True, alpha=0.3)
ax.set_ylim(0, 1)
# Plot 2: Penetration depth distribution
ax = axes[0, 1]
# Simulate penetration depths for different qualities
low_q_depths = np.random.exponential(0.005, 100) # Bad grasps
high_q_depths = np.random.exponential(0.001, 100) # Good grasps
ax.hist(low_q_depths * 1000, bins=20, alpha=0.5, label='Low Quality', color='red')
ax.hist(high_q_depths * 1000, bins=20, alpha=0.5, label='High Quality', color='green')
ax.axvline(x=config.max_penetration_threshold * 1000, color='black',
linestyle='--', label='Threshold')
ax.set_xlabel('Penetration Depth (mm)')
ax.set_ylabel('Frequency')
ax.set_title('Penetration Depth Distribution')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 3: Max vs Mean penetration
ax = axes[1, 0]
ax.scatter(mean_penetrations, max_penetrations, c=qualities, cmap='viridis', s=50)
ax.plot([0, max(mean_penetrations)], [0, max(mean_penetrations)*3], 'r--', alpha=0.5)
ax.set_xlabel('Mean Penetration (m)')
ax.set_ylabel('Max Penetration (m)')
ax.set_title('Max vs Mean Penetration Depth')
cbar = plt.colorbar(ax.collections[0], ax=ax)
cbar.set_label('Grasp Quality')
ax.grid(True, alpha=0.3)
# Plot 4: Collision timeline heatmap
ax = axes[1, 1]
# Create heatmap of collisions over time for different qualities
time_bins = 20
quality_bins = len(qualities)
heatmap = np.zeros((quality_bins, time_bins))
for i, q in enumerate(qualities):
# Simulate collision distribution over time
if q < 0.5:
# Early collisions for bad grasps
heatmap[i, :5] = np.random.rand(5) * 0.8
else:
# Late collisions (if any) for good grasps
heatmap[i, -5:] = np.random.rand(5) * 0.2
im = ax.imshow(heatmap, aspect='auto', cmap='Reds', interpolation='bilinear')
ax.set_xlabel('Timestep')
ax.set_ylabel('Grasp Quality Index')
ax.set_title('Collision Timeline Heatmap')
plt.colorbar(im, ax=ax, label='Collision Intensity')
plt.tight_layout()
plt.savefig('collision_analysis.png', dpi=150, bbox_inches='tight')
print("Saved collision_analysis.png")
def visualize_penetration_depth():
"""Visualize penetration depth computation."""
fig, axes = plt.subplots(1, 2, figsize=(14, 6))
# Plot 1: 2D Penetration depth visualization
ax = axes[0]
# Two overlapping rectangles
rect1 = patches.Rectangle((-0.1, -0.05), 0.2, 0.1,
linewidth=2, edgecolor='blue', facecolor='blue', alpha=0.3)
rect2 = patches.Rectangle((-0.05, -0.08), 0.2, 0.1,
linewidth=2, edgecolor='red', facecolor='red', alpha=0.3)
ax.add_patch(rect1)
ax.add_patch(rect2)
# Show penetration depth arrow
ax.arrow(0.05, 0, 0.03, 0, head_width=0.02, head_length=0.01,
fc='green', ec='green', linewidth=2, label='PD direction')
ax.text(0.06, 0.02, 'Penetration\nDepth', fontsize=9, ha='center')
ax.set_xlim(-0.2, 0.2)
ax.set_ylim(-0.15, 0.15)
ax.set_aspect('equal')
ax.set_title('Penetration Depth Concept')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 2: Success-PD tradeoff curve
ax = axes[1]
# Simulate tradeoff: more aggressive grasps have higher success but more PD
aggressiveness = np.linspace(0, 1, 50)
success_rate = 1 - np.exp(-3 * aggressiveness)
avg_pd = 0.001 + 0.01 * aggressiveness**2
ax.plot(avg_pd * 1000, success_rate, 'b-', linewidth=2, label='Tradeoff curve')
# Mark optimal point
optimal_idx = np.argmax(success_rate - avg_pd * 10)
ax.plot(avg_pd[optimal_idx] * 1000, success_rate[optimal_idx],
'r*', markersize=15, label='Optimal')
ax.set_xlabel('Average Penetration Depth (mm)')
ax.set_ylabel('Success Rate')
ax.set_title('Success Rate vs Penetration Depth Tradeoff')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_xlim(0, 12)
ax.set_ylim(0, 1)
plt.tight_layout()
plt.savefig('penetration_depth.png', dpi=150, bbox_inches='tight')
print("Saved penetration_depth.png")
if __name__ == "__main__":
print("Generating collision analysis visualization...")
visualize_collision_analysis()
print("\nGenerating penetration depth visualization...")
visualize_penetration_depth()
print("\nCompleted. Check output PNG files.")script_grasp_quality_metrics.py
该脚本实现几何质量指标计算，包含抓取椭球体积、力封闭性判定、稳定性裕度分析。可视化展示椭球形状、力封闭锥、稳定性边界与质量指标分布。
Python
#!/usr/bin/env python3
"""
Geometric Grasp Quality Metrics: Ellipsoid, Force Closure, Stability
Content: 8.1.2 Geometric Quality Metrics (8.1.2.1-8.1.2.3)
Usage: python script_grasp_quality_metrics.py
Output: grasp_ellipsoid.png, force_closure.png, stability_metrics.png
"""
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
import matplotlib.patches as mpatches
from scipy.linalg import eigh, inv
from scipy.spatial import ConvexHull
class GraspQualityAnalyzer:
"""
Compute geometric grasp quality metrics.
"""
def __init__(self, n_contacts=4):
self.n_contacts = n_contacts
def compute_grasp_matrix(self, contact_points, contact_normals,
friction_coeffs):
"""
Compute grasp matrix G.
G = [n1 n2 ... nn; c1xn1 c2xn2 ... cnxnn]
"""
n = len(contact_points)
G = np.zeros((6, 3*n))
for i, (cp, cn, mu) in enumerate(zip(contact_points, contact_normals, friction_coeffs)):
# Force component (normals)
G[0:3, 3*i:3*i+3] = np.eye(3) # Simplified: should be friction cone basis
# Torque component
G[3:6, 3*i:3*i+3] = self._skew_matrix(cp) @ np.eye(3)
return G
def _skew_matrix(self, v):
"""Compute skew-symmetric matrix."""
return np.array([
[0, -v[2], v[1]],
[v[2], 0, -v[0]],
[-v[1], v[0], 0]
])
def grasp_ellipsoid_volume(self, G):
"""
Compute grasp ellipsoid volume from singular values.
"""
# Grasp matrix G (6 x 3n)
# GG^T is 6x6
GG_T = G @ G.T
# Eigenvalues
eigvals = eigh(GG_T, eigvals_only=True)
eigvals = np.maximum(eigvals, 1e-8) # Ensure positive
# Ellipsoid volume (product of sqrt eigenvalues)
volume = np.prod(np.sqrt(eigvals))
# Normalized volume
volume_norm = np.exp(np.mean(np.log(eigvals))) # Geometric mean
return volume, eigvals
def force_closure_check(self, contact_points, contact_normals,
friction_coeffs, external_wrench=None):
"""
Check force closure using convex hull method.
"""
# Build wrench space representation
wrenches = []
for cp, cn, mu in zip(contact_points, contact_normals, friction_coeffs):
# Wrench from contact: [f; c x f]
# Sample friction cone
n_samples = 8
angles = np.linspace(0, 2*np.pi, n_samples)
for theta in angles:
# Tangential direction
if abs(cn[2]) < 0.9:
tangent = np.cross(cn, [0, 0, 1])
else:
tangent = np.cross(cn, [0, 1, 0])
tangent = tangent / (np.linalg.norm(tangent) + 1e-8)
binormal = np.cross(cn, tangent)
# Friction cone force
f_tan = mu * np.cos(theta)
f_bin = mu * np.sin(theta)
force = cn + f_tan * tangent + f_bin * binormal
force = force / (np.linalg.norm(force) + 1e-8)
# Wrench
torque = np.cross(cp, force)
wrench = np.concatenate([force, torque])
wrenches.append(wrench)
wrenches = np.array(wrenches)
# Check if origin is inside convex hull
if len(wrenches) < 7:
return False, 0.0, wrenches
try:
hull = ConvexHull(wrenches)
# Check if origin is inside
# Simplified: check if origin can be expressed as convex combination
# True force closure check requires LP solver
centroid = np.mean(wrenches, axis=0)
is_fc = np.linalg.norm(centroid) < 0.1 # Heuristic
# Quality: volume of convex hull
quality = hull.volume if hasattr(hull, 'volume') else 1.0
except:
is_fc = False
quality = 0.0
return is_fc, quality, wrenches
def stability_metrics(self, contact_points, contact_normals,
forces, object_com):
"""
Compute stability metrics.
"""
# Center of pressure offset
avg_contact = np.mean(contact_points, axis=0)
com_offset = np.linalg.norm(avg_contact - object_com)
# Force alignment
total_force = np.sum(forces, axis=0)
force_alignment = np.dot(total_force, [0, 0, 1]) / (np.linalg.norm(total_force) + 1e-8)
# Minimum distance to friction cone boundary
min_margin = float('inf')
for cp, cn, f in zip(contact_points, contact_normals, forces):
f_n = np.dot(f, cn) * cn
f_t = f - f_n
ratio = np.linalg.norm(f_t) / (np.linalg.norm(f_n) + 1e-8)
margin = 0.5 - ratio # Assuming mu=0.5
min_margin = min(min_margin, margin)
return {
'com_offset': com_offset,
'force_alignment': force_alignment,
'friction_margin': min_margin,
'stability_score': max(0, min_margin) * force_alignment
}
def visualize_grasp_ellipsoid():
"""Visualize grasp ellipsoid from singular values."""
fig = plt.figure(figsize=(14, 6))
# Create grasp matrix for different configurations
ax1 = fig.add_subplot(121, projection='3d')
# Good grasp: power grasp
contact_points_good = np.array([
[0.05, 0, 0], [-0.05, 0, 0],
[0, 0.05, 0], [0, -0.05, 0]
])
normals_good = np.array([
[-1, 0, 0], [1, 0, 0],
[0, -1, 0], [0, 1, 0]
])
analyzer = GraspQualityAnalyzer()
G_good = analyzer.compute_grasp_matrix(contact_points_good, normals_good, [0.5]*4)
vol_good, eigs_good = analyzer.grasp_ellipsoid_volume(G_good)
# Draw ellipsoid
u = np.linspace(0, 2 * np.pi, 20)
v = np.linspace(0, np.pi, 20)
# Scale by eigenvalues
scales = np.sqrt(eigs_good[:3])
x = scales[0] * np.outer(np.cos(u), np.sin(v))
y = scales[1] * np.outer(np.sin(u), np.sin(v))
z = scales[2] * np.outer(np.ones(np.size(u)), np.cos(v))
ax1.plot_surface(x, y, z, alpha=0.3, color='green')
ax1.set_title(f'Good Grasp Ellipsoid\nVolume: {vol_good:.4f}')
ax1.set_xlabel('Fx')
ax1.set_ylabel('Fy')
ax1.set_zlabel('Fz')
# Bad grasp: contacts aligned
ax2 = fig.add_subplot(122, projection='3d')
contact_points_bad = np.array([
[0.05, 0.05, 0], [0.05, -0.05, 0],
[-0.05, 0.05, 0], [-0.05, -0.05, 0]
])
normals_bad = np.array([
[1, 0, 0], [1, 0, 0],
[-1, 0, 0], [-1, 0, 0]
])
G_bad = analyzer.compute_grasp_matrix(contact_points_bad, normals_bad, [0.5]*4)
vol_bad, eigs_bad = analyzer.grasp_ellipsoid_volume(G_bad)
scales_bad = np.sqrt(eigs_bad[:3])
x_bad = scales_bad[0] * np.outer(np.cos(u), np.sin(v))
y_bad = scales_bad[1] * np.outer(np.sin(u), np.sin(v))
z_bad = scales_bad[2] * np.outer(np.ones(np.size(u)), np.cos(v))
ax2.plot_surface(x_bad, y_bad, z_bad, alpha=0.3, color='red')
ax2.set_title(f'Bad Grasp Ellipsoid\nVolume: {vol_bad:.4f}')
ax2.set_xlabel('Fx')
ax2.set_ylabel('Fy')
ax2.set_zlabel('Fz')
plt.tight_layout()
plt.savefig('grasp_ellipsoid.png', dpi=150, bbox_inches='tight')
print("Saved grasp_ellipsoid.png")
def visualize_force_closure():
"""Visualize force closure analysis."""
fig, axes = plt.subplots(1, 2, figsize=(14, 6))
analyzer = GraspQualityAnalyzer()
# Scenario 1: Force closure grasp
ax = axes[0]
# Antipodal grasp on cylinder
contacts = np.array([[0.05, 0], [-0.05, 0]])
normals = np.array([[-1, 0], [1, 0]])
mus = [0.5, 0.5]
# Draw object
circle = plt.Circle((0, 0), 0.05, fill=False, color='blue', linewidth=2)
ax.add_patch(circle)
# Draw contacts and friction cones
colors = ['red', 'green']
for i, (cp, cn, color) in enumerate(zip(contacts, normals, colors)):
ax.plot(cp[0], cp[1], 'o', markersize=10, color=color)
# Friction cone
cone_angle = np.arctan(0.5) # mu=0.5
direction = np.arctan2(cn[1], cn[0])
angles = np.linspace(direction - cone_angle, direction + cone_angle, 20)
cone_x = cp[0] + 0.03 * np.cos(angles)
cone_y = cp[1] + 0.03 * np.sin(angles)
ax.fill(cone_x, cone_y, alpha=0.3, color=color)
# Check force closure
is_fc, quality, _ = analyzer.force_closure_check(
np.array([[0.05, 0, 0], [-0.05, 0, 0]]),
np.array([[-1, 0, 0], [1, 0, 0]]),
[0.5, 0.5]
)
ax.set_xlim(-0.1, 0.1)
ax.set_ylim(-0.1, 0.1)
ax.set_aspect('equal')
ax.set_title(f'Force Closure: {is_fc}, Quality: {quality:.3f}')
ax.grid(True, alpha=0.3)
# Scenario 2: No force closure
ax = axes[1]
# All contacts on same side
contacts_bad = np.array([[0.04, 0.04], [0.04, -0.04]])
normals_bad = np.array([[-1, 0], [-1, 0]])
circle2 = plt.Circle((0, 0), 0.05, fill=False, color='blue', linewidth=2)
ax.add_patch(circle2)
for cp, cn in zip(contacts_bad, normals_bad):
ax.plot(cp[0], cp[1], 'o', markersize=10, color='red')
cone_angle = np.arctan(0.5)
direction = np.arctan2(cn[1], cn[0])
angles = np.linspace(direction - cone_angle, direction + cone_angle, 20)
cone_x = cp[0] + 0.03 * np.cos(angles)
cone_y = cp[1] + 0.03 * np.sin(angles)
ax.fill(cone_x, cone_y, alpha=0.3, color='red')
is_fc2, quality2, _ = analyzer.force_closure_check(
np.array([[0.04, 0.04, 0], [0.04, -0.04, 0]]),
np.array([[-1, 0, 0], [-1, 0, 0]]),
[0.5, 0.5]
)
ax.set_xlim(-0.1, 0.1)
ax.set_ylim(-0.1, 0.1)
ax.set_aspect('equal')
ax.set_title(f'Force Closure: {is_fc2}, Quality: {quality2:.3f}')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('force_closure.png', dpi=150, bbox_inches='tight')
print("Saved force_closure.png")
def visualize_stability_metrics():
"""Visualize stability analysis."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
analyzer = GraspQualityAnalyzer()
# Generate different grasp configurations
com = np.array([0, 0, 0])
configs = []
for offset in np.linspace(0, 0.05, 10):
contacts = np.array([
[0.05, offset, 0],
[-0.05, -offset, 0],
[0, 0.05, 0],
[0, -0.05, 0]
])
normals = np.array([
[-1, 0, 0],
[1, 0, 0],
[0, -1, 0],
[0, 1, 0]
])
forces = normals * 2 # Equal forces along normals
metrics = analyzer.stability_metrics(contacts, normals, forces, com)
configs.append({
'offset': offset,
'metrics': metrics
})
# Plot 1: COM offset vs stability
ax = axes[0, 0]
offsets = [c['offset'] for c in configs]
stability = [c['metrics']['stability_score'] for c in configs]
com_offsets = [c['metrics']['com_offset'] for c in configs]
ax.plot(offsets, stability, 'b-o', label='Stability Score')
ax.set_xlabel('Contact Asymmetry (m)')
ax.set_ylabel('Stability Score')
ax.set_title('Stability vs Contact Asymmetry')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 2: Friction margin distribution
ax = axes[0, 1]
# Simulate many grasps
np.random.seed(42)
n_samples = 1000
margins = np.random.beta(2, 5, n_samples) * 0.5 # Skewed towards low margin
ax.hist(margins, bins=30, alpha=0.7, color='green', edgecolor='black')
ax.axvline(x=0, color='red', linestyle='--', linewidth=2, label='Slip threshold')
ax.set_xlabel('Friction Margin')
ax.set_ylabel('Count')
ax.set_title('Friction Margin Distribution')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 3: Force alignment radar
ax = axes[1, 0]
categories = ['Force\nAlignment', 'Friction\nMargin', 'COM\nOffset', 'Contact\nSpread']
values = [0.9, 0.6, 0.8, 0.7] # Example good grasp
values_bad = [0.4, 0.2, 0.3, 0.5] # Example bad grasp
angles = np.linspace(0, 2*np.pi, len(categories), endpoint=False).tolist()
values += values[:1]
values_bad += values_bad[:1]
angles += angles[:1]
ax.plot(angles, values, 'o-', linewidth=2, label='Good Grasp', color='green')
ax.fill(angles, values, alpha=0.25, color='green')
ax.plot(angles, values_bad, 'o-', linewidth=2, label='Bad Grasp', color='red')
ax.fill(angles, values_bad, alpha=0.25, color='red')
ax.set_xticks(angles[:-1])
ax.set_xticklabels(categories)
ax.set_ylim(0, 1)
ax.set_title('Stability Metrics Radar')
ax.legend()
ax.grid(True)
# Plot 4: Stability classes
ax = axes[1, 1]
# 2D scatter of stability vs success (simulated)
stability_vals = np.random.beta(3, 2, 500)
success = (stability_vals + np.random.randn(500) * 0.1) > 0.5
ax.scatter(stability_vals, success + np.random.randn(500)*0.05,
c=success, cmap='RdYlGn', alpha=0.5)
ax.set_xlabel('Stability Score')
ax.set_ylabel('Success (jittered)')
ax.set_title('Stability vs Actual Success')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('stability_metrics.png', dpi=150, bbox_inches='tight')
print("Saved stability_metrics.png")
if __name__ == "__main__":
print("Generating grasp ellipsoid visualization...")
visualize_grasp_ellipsoid()
print("\nGenerating force closure visualization...")
visualize_force_closure()
print("\nGenerating stability metrics visualization...")
visualize_stability_metrics()
print("\nCompleted. Check output PNG files.")script_baseline_comparison.py
该脚本实现基准测试对比实验，包含传统分析、深度学习、强化学习方法对比，可见vs未见物体性能分析，类别差异统计，推理时间与显存占用评估。可视化展示对比雷达图、泛化差距、类别性能矩阵与计算效率权衡。
Python
#!/usr/bin/env python3
"""
Baseline Comparison Experiments: Methods, Generalization, and Efficiency
Content: 8.2 Baseline Comparison (8.2.1-8.2.2)
Usage: python script_baseline_comparison.py
Output: baseline_comparison.png, generalization_analysis.png, efficiency_tradeoff.png
"""
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from typing import Dict, List
import matplotlib.patches as mpatches
class BaselineExperiment:
"""
Simulate baseline comparison experiments.
"""
def __init__(self):
self.methods = {
'GraspIt!': {'type': 'analytical', 'params': 0},
'GPD': {'type': 'analytical', 'params': 0},
'PointNetGPD': {'type': 'deep_learning', 'params': 3.5e6},
'GraspPointNet': {'type': 'deep_learning', 'params': 4.2e6},
'DexNet 4.0': {'type': 'deep_learning', 'params': 1.8e6},
'DDPG-Grasp': {'type': 'rl', 'params': 1.2e6},
'PPO-Grasp': {'type': 'rl', 'params': 1.5e6},
'SAC-Grasp': {'type': 'rl', 'params': 1.4e6}
}
def simulate_performance(self, method: str, seen: bool = True) -> Dict:
"""
Simulate performance metrics for a method.
"""
np.random.seed(hash(method) % 2**32)
base_perf = {
'GraspIt!': 0.65,
'GPD': 0.70,
'PointNetGPD': 0.82,
'GraspPointNet': 0.85,
'DexNet 4.0': 0.88,
'DDPG-Grasp': 0.75,
'PPO-Grasp': 0.78,
'SAC-Grasp': 0.80
}[method]
# Generalization gap
if not seen:
if self.methods[method]['type'] == 'analytical':
gap = 0.05
elif self.methods[method]['type'] == 'deep_learning':
gap = 0.15
else:
gap = 0.10
base_perf -= gap
# Add noise
success_rate = base_perf + np.random.randn() * 0.02
inference_time = {
'analytical': np.random.uniform(0.5, 2.0),
'deep_learning': np.random.uniform(0.02, 0.1),
'rl': np.random.uniform(0.01, 0.05)
}[self.methods[method]['type']]
memory = {
'analytical': np.random.uniform(0.1, 0.5),
'deep_learning': np.random.uniform(2, 8),
'rl': np.random.uniform(1, 4)
}[self.methods[method]['type']]
return {
'success_rate': max(0, min(1, success_rate)),
'inference_time': inference_time,
'gpu_memory': memory,
'type': self.methods[method]['type']
}
def category_performance(self, method: str) -> Dict[str, float]:
"""
Performance across object categories.
"""
categories = ['Box', 'Cylinder', 'Sphere', 'Bottle', 'Mug', 'Tool']
base = self.simulate_performance(method, seen=True)['success_rate']
# Category specific biases
biases = {
'Box': 0.05,
'Cylinder': 0.03,
'Sphere': -0.10,
'Bottle': -0.05,
'Mug': -0.08,
'Tool': -0.15
}
return {cat: max(0, min(1, base + biases[cat] + np.random.randn()*0.03))
for cat in categories}
def visualize_baseline_comparison():
"""Visualize baseline method comparison."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
exp = BaselineExperiment()
# Collect data
methods = list(exp.methods.keys())
seen_perf = [exp.simulate_performance(m, seen=True) for m in methods]
unseen_perf = [exp.simulate_performance(m, seen=False) for m in methods]
# Plot 1: Overall performance comparison
ax = axes[0, 0]
x = np.arange(len(methods))
width = 0.35
seen_rates = [p['success_rate'] for p in seen_perf]
unseen_rates = [p['success_rate'] for p in unseen_perf]
bars1 = ax.bar(x - width/2, seen_rates, width, label='Seen Objects', alpha=0.8)
bars2 = ax.bar(x + width/2, unseen_rates, width, label='Unseen Objects', alpha=0.8)
# Color by type
colors = {'analytical': 'blue', 'deep_learning': 'green', 'rl': 'red'}
for i, (bar1, bar2, m) in enumerate(zip(bars1, bars2, methods)):
c = colors[exp.methods[m]['type']]
bar1.set_color(c)
bar2.set_color(c)
bar2.set_hatch('//')
ax.set_ylabel('Success Rate')
ax.set_title('Method Comparison: Seen vs Unseen')
ax.set_xticks(x)
ax.set_xticklabels(methods, rotation=45, ha='right')
ax.legend()
ax.grid(True, alpha=0.3, axis='y')
# Plot 2: Generalization gap
ax = axes[0, 1]
gaps = [s - u for s, u in zip(seen_rates, unseen_rates)]
colors_list = [colors[exp.methods[m]['type']] for m in methods]
bars = ax.bar(methods, gaps, color=colors_list, alpha=0.7, edgecolor='black')
ax.axhline(y=0, color='black', linestyle='-', linewidth=0.5)
ax.set_ylabel('Generalization Gap')
ax.set_title('Seen - Unseen Performance Gap')
plt.setp(ax.xaxis.get_majorticklabels(), rotation=45, ha='right')
ax.grid(True, alpha=0.3, axis='y')
# Add value labels
for bar, gap in zip(bars, gaps):
height = bar.get_height()
ax.text(bar.get_x() + bar.get_width()/2., height,
f'{gap:.2f}', ha='center', va='bottom' if height > 0 else 'top')
# Plot 3: Category performance heatmap
ax = axes[1, 0]
categories = ['Box', 'Cylinder', 'Sphere', 'Bottle', 'Mug', 'Tool']
selected_methods = ['GraspIt!', 'PointNetGPD', 'SAC-Grasp']
data = []
for m in selected_methods:
cat_perf = exp.category_performance(m)
data.append([cat_perf[cat] for cat in categories])
data = np.array(data)
im = ax.imshow(data, cmap='RdYlGn', aspect='auto', vmin=0, vmax=1)
ax.set_xticks(np.arange(len(categories)))
ax.set_yticks(np.arange(len(selected_methods)))
ax.set_xticklabels(categories)
ax.set_yticklabels(selected_methods)
ax.set_title('Performance by Object Category')
# Add text annotations
for i in range(len(selected_methods)):
for j in range(len(categories)):
text = ax.text(j, i, f'{data[i, j]:.2f}',
ha="center", va="center", color="black", fontsize=9)
plt.colorbar(im, ax=ax, label='Success Rate')
# Plot 4: Radar chart for multi-dimensional comparison
ax = axes[1, 1]
categories_radar = ['Success\nRate', 'Speed', 'Memory\nEfficiency', 'Generalization', 'Robustness']
# Normalize metrics
def normalize_metric(method):
perf = exp.simulate_performance(method, seen=True)
# Speed: inverse of time (higher is better)
speed = 1 / (perf['inference_time'] + 0.01)
# Memory: inverse (higher is better)
mem_eff = 1 / (perf['gpu_memory'] + 0.1)
# Success rate
succ = perf['success_rate']
# Gen gap (inverse, smaller gap is better)
gen = 1 - (exp.simulate_performance(method, seen=True)['success_rate'] -
exp.simulate_performance(method, seen=False)['success_rate'])
# Robustness (simulated)
robust = np.random.uniform(0.6, 0.9)
vals = [succ, speed/10, mem_eff, gen, robust]
return [min(1, max(0, v)) for v in vals]
angles = np.linspace(0, 2*np.pi, len(categories_radar), endpoint=False).tolist()
angles += angles[:1]
for method in ['GraspIt!', 'PointNetGPD', 'SAC-Grasp']:
values = normalize_metric(method)
values += values[:1]
ax.plot(angles, values, 'o-', linewidth=2, label=method)
ax.fill(angles, values, alpha=0.15)
ax.set_xticks(angles[:-1])
ax.set_xticklabels(categories_radar)
ax.set_ylim(0, 1)
ax.set_title('Multi-dimensional Performance')
ax.legend()
ax.grid(True)
plt.tight_layout()
plt.savefig('baseline_comparison.png', dpi=150, bbox_inches='tight')
print("Saved baseline_comparison.png")
def visualize_generalization_analysis():
"""Visualize generalization gap analysis."""
fig, axes = plt.subplots(1, 2, figsize=(14, 6))
# Plot 1: Performance vs shape complexity
ax = axes[0]
# Shape complexity metric (surface area / volume^(2/3))
complexities = np.linspace(1, 5, 50)
# Different method types
for method_type, color in [('analytical', 'blue'),
('deep_learning', 'red'),
('rl', 'green')]:
if method_type == 'analytical':
base = 0.65
decay = 0.02
elif method_type == 'deep_learning':
base = 0.85
decay = 0.08
else:
base = 0.78
decay = 0.05
perf_seen = base - decay * (complexities - 1) * 0.3
perf_unseen = perf_seen - (0.05 if method_type == 'analytical' else 0.12)
ax.plot(complexities, perf_seen, color=color, linestyle='-',
linewidth=2, label=f'{method_type} (seen)')
ax.plot(complexities, perf_unseen, color=color, linestyle='--',
linewidth=2, alpha=0.7, label=f'{method_type} (unseen)')
ax.set_xlabel('Shape Complexity (SA/Vol^{2/3})')
ax.set_ylabel('Success Rate')
ax.set_title('Performance vs Object Complexity')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(0, 1)
# Plot 2: Learning curve comparison
ax = axes[1]
# Training samples vs performance
n_samples = np.logspace(2, 5, 20)
# Analytical (flat)
ax.plot(n_samples, [0.65]*20, 'b-', linewidth=2, label='Analytical')
# Deep learning (logarithmic improvement)
dl_perf = 0.5 + 0.35 * (1 - np.exp(-n_samples/10000))
ax.plot(n_samples, dl_perf, 'r-', linewidth=2, label='Deep Learning')
# RL (needs more data)
rl_perf = 0.4 + 0.4 * (1 - np.exp(-n_samples/50000))
ax.plot(n_samples, rl_perf, 'g-', linewidth=2, label='Reinforcement Learning')
ax.set_xscale('log')
ax.set_xlabel('Training Samples')
ax.set_ylabel('Success Rate')
ax.set_title('Data Efficiency Comparison')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(0, 1)
plt.tight_layout()
plt.savefig('generalization_analysis.png', dpi=150, bbox_inches='tight')
print("Saved generalization_analysis.png")
def visualize_efficiency_tradeoff():
"""Visualize computational efficiency analysis."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
exp = BaselineExperiment()
methods = list(exp.methods.keys())
# Collect efficiency data
data = []
for m in methods:
perf = exp.simulate_performance(m, seen=True)
data.append({
'method': m,
'type': perf['type'],
'time': perf['inference_time'],
'memory': perf['gpu_memory'],
'success': perf['success_rate']
})
# Plot 1: Inference time distribution
ax = axes[0, 0]
times_by_type = {}
for d in data:
t = d['type']
if t not in times_by_type:
times_by_type[t] = []
times_by_type[t].append(d['time'])
positions = [1, 2, 3]
bp = ax.boxplot([times_by_type['analytical'],
times_by_type['deep_learning'],
times_by_type['rl']],
positions=positions,
labels=['Analytical', 'Deep Learning', 'RL'])
colors = ['blue', 'green', 'red']
for patch, color in zip(bp['boxes'], colors):
patch.set_facecolor(color)
patch.set_alpha(0.7)
ax.set_ylabel('Inference Time (s)')
ax.set_title('Inference Time by Method Type')
ax.grid(True, alpha=0.3, axis='y')
# Plot 2: Memory usage vs Performance
ax = axes[0, 1]
for d in data:
color = {'analytical': 'blue', 'deep_learning': 'green', 'rl': 'red'}[d['type']]
ax.scatter(d['memory'], d['success'],
c=color, s=100, alpha=0.7, edgecolors='black')
ax.annotate(d['method'], (d['memory'], d['success']),
fontsize=8, alpha=0.7)
ax.set_xlabel('GPU Memory (GB)')
ax.set_ylabel('Success Rate')
ax.set_title('Memory-Performance Tradeoff')
ax.grid(True, alpha=0.3)
# Plot 3: Throughput analysis (batch processing)
ax = axes[1, 0]
batch_sizes = [1, 4, 8, 16, 32]
for method_type, color in [('analytical', 'blue'),
('deep_learning', 'green'),
('rl', 'red')]:
# Simulate batch processing efficiency
base_time = {'analytical': 1.0, 'deep_learning': 0.05, 'rl': 0.02}[method_type]
# Batch efficiency (not perfectly linear due to overhead)
times = [base_time * (1 + 0.1 * np.log2(b)) for b in batch_sizes]
throughput = [b/t for b, t in zip(batch_sizes, times)]
ax.plot(batch_sizes, throughput, 'o-', color=color,
linewidth=2, label=method_type)
ax.set_xlabel('Batch Size')
ax.set_ylabel('Throughput (grasps/s)')
ax.set_title('Batch Processing Throughput')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 4: Real-time feasibility
ax = axes[1, 1]
# Pareto frontier: success rate vs 1/latency
success = [d['success'] for d in data]
latency_inv = [1/d['time'] for d in data]
ax.scatter(latency_inv, success, s=100, c='blue', alpha=0.6, edgecolors='black')
for d in data:
ax.annotate(d['method'], (1/d['time'], d['success']),
fontsize=8, alpha=0.7)
# Mark real-time threshold (10Hz = 0.1s)
ax.axvline(x=10, color='red', linestyle='--', label='Real-time threshold (10Hz)')
ax.set_xlabel('1 / Latency (Hz)')
ax.set_ylabel('Success Rate')
ax.set_title('Success-Latency Pareto Frontier')
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('efficiency_tradeoff.png', dpi=150, bbox_inches='tight')
print("Saved efficiency_tradeoff.png")
if __name__ == "__main__":
print("Generating baseline comparison visualization...")
visualize_baseline_comparison()
print("\nGenerating generalization analysis visualization...")
visualize_generalization_analysis()
print("\nGenerating efficiency tradeoff visualization...")
visualize_efficiency_tradeoff()
print("\nCompleted. Check output PNG files.")script_ablation_study.py
该脚本实现消融实验分析，包含网络模块移除实验、敏感性分析、组件贡献度计算。可视化展示模块重要性排序、损失权重热力图与特征消融影响。
Python
#!/usr/bin/env python3
"""
Ablation Study: Module Effectiveness and Loss Sensitivity Analysis
Content: 8.3.1 Ablation Study Design (8.3.1.1-8.3.1.2)
Usage: python script_ablation_study.py
Output: ablation_modules.png, loss_sensitivity.png, component_analysis.png
"""
import numpy as as np
import matplotlib.pyplot as plt
from typing import Dict, List, Tuple
import pandas as pd
import matplotlib.patches as mpatches
class AblationExperiment:
"""
Simulate ablation study experiments.
"""
def __init__(self):
self.modules = {
'full_model': {'components': ['backbone', 'attention', 'contact_head', 'collision_head', 'refinement']},
'wo_attention': {'components': ['backbone', 'contact_head', 'collision_head', 'refinement']},
'wo_contact': {'components': ['backbone', 'attention', 'collision_head', 'refinement']},
'wo_collision': {'components': ['backbone', 'attention', 'contact_head', 'refinement']},
'wo_refinement': {'components': ['backbone', 'attention', 'contact_head', 'collision_head']},
'backbone_only': {'components': ['backbone']}
}
# Simulate performance drops
self.baseline_performance = 0.88
self.component_impacts = {
'backbone': 0.30, # Critical
'attention': 0.08, # Moderate
'contact_head': 0.12, # Important
'collision_head': 0.15, # Important
'refinement': 0.05 # Minor
}
def simulate_ablation(self, model_config: str) -> Dict:
"""
Simulate performance of ablated model.
"""
components = self.modules[model_config]['components']
# Calculate performance
performance = 0.40 # Base without any components
for comp in components:
performance += self.component_impacts[comp]
# Add noise
performance += np.random.randn() * 0.02
# Training time impact (more components = longer)
train_time = len(components) * 2 + np.random.randn() * 0.5
# Convergence speed
convergence = 50 - len(components) * 5 + np.random.randn() * 3
return {
'performance': min(0.95, performance),
'components': components,
'n_components': len(components),
'train_time': max(10, train_time),
'convergence_epoch': max(10, convergence)
}
def component_importance(self) -> Dict[str, float]:
"""
Calculate individual component importance by single removal.
"""
importances = {}
for comp in self.component_impacts.keys():
# Performance with vs without
with_perf = self.baseline_performance
without_perf = self.baseline_performance - self.component_impacts[comp]
importances[comp] = with_perf - without_perf
return importances
class LossSensitivityAnalyzer:
"""
Analyze sensitivity to loss function weights.
"""
def __init__(self):
self.loss_terms = ['grasp_quality', 'collision', 'smoothness', 'alignment']
self.weight_ranges = {
'grasp_quality': (0.1, 2.0),
'collision': (0.1, 2.0),
'smoothness': (0.01, 0.5),
'alignment': (0.1, 1.0)
}
def grid_search(self, resolution: int = 10) -> pd.DataFrame:
"""
Perform grid search over loss weights.
"""
results = []
# Simplified: vary two main weights, keep others fixed
w1_range = np.linspace(0.1, 2.0, resolution)
w2_range = np.linspace(0.1, 2.0, resolution)
for w1 in w1_range:
for w2 in w2_range:
# Simulate performance surface
# Optimal around (1.0, 1.0)
perf = 0.85 - 0.05 * ((w1 - 1.0)**2 + (w2 - 1.0)**2) + np.random.randn() * 0.01
perf = max(0, min(1, perf))
results.append({
'grasp_quality_weight': w1,
'collision_weight': w2,
'performance': perf,
'stability': perf > 0.80
})
return pd.DataFrame(results)
def sensitivity_gradient(self, base_weights: Dict[str, float]) -> Dict[str, float]:
"""
Compute sensitivity gradient at base point.
"""
gradients = {}
for term in self.loss_terms:
base = base_weights.get(term, 1.0)
# Perturb
delta = 0.1
perf_plus = 0.85 - 0.05 * ((base + delta - 1.0)**2)
perf_minus = 0.85 - 0.05 * ((base - delta - 1.0)**2)
gradient = (perf_plus - perf_minus) / (2 * delta)
gradients[term] = abs(gradient)
return gradients
def visualize_ablation_modules():
"""Visualize module ablation results."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
exp = AblationExperiment()
# Run ablations
results = {}
for model in exp.modules.keys():
results[model] = exp.simulate_ablation(model)
# Plot 1: Performance by ablation
ax = axes[0, 0]
models = list(results.keys())
perfs = [results[m]['performance'] for m in models]
colors = plt.cm.RdYlGn(np.array(perfs))
bars = ax.barh(models, perfs, color=colors, edgecolor='black')
ax.axvline(x=exp.baseline_performance, color='red', linestyle='--',
label='Full Model')
ax.set_xlabel('Success Rate')
ax.set_title('Performance by Model Configuration')
ax.set_xlim(0.4, 0.9)
ax.grid(True, alpha=0.3, axis='x')
# Add value labels
for bar, perf in zip(bars, perfs):
width = bar.get_width()
ax.text(width, bar.get_y() + bar.get_height()/2.,
f'{perf:.3f}', ha='left', va='center', fontsize=9)
# Plot 2: Component importance
ax = axes[0, 1]
importances = exp.component_importance()
comps = list(importances.keys())
values = list(importances.values())
# Sort by importance
sorted_idx = np.argsort(values)[::-1]
comps = [comps[i] for i in sorted_idx]
values = [values[i] for i in sorted_idx]
colors_imp = plt.cm.Reds(np.linspace(0.3, 0.9, len(values)))
bars = ax.barh(comps, values, color=colors_imp, edgecolor='black')
ax.set_xlabel('Performance Drop (Importance)')
ax.set_title('Component Importance Ranking')
ax.grid(True, alpha=0.3, axis='x')
# Plot 3: Training efficiency
ax = axes[1, 0]
train_times = [results[m]['train_time'] for m in models]
perfs_list = [results[m]['performance'] for m in models]
scatter = ax.scatter(train_times, perfs_list,
s=[results[m]['n_components']*50 for m in models],
c=perfs_list, cmap='RdYlGn',
alpha=0.7, edgecolors='black')
for i, m in enumerate(models):
ax.annotate(m, (train_times[i], perfs_list[i]),
fontsize=8, alpha=0.7)
ax.set_xlabel('Training Time (hours)')
ax.set_ylabel('Performance')
ax.set_title('Training Cost vs Performance')
ax.grid(True, alpha=0.3)
# Plot 4: Convergence analysis
ax = axes[1, 1]
conv_epochs = [results[m]['convergence_epoch'] for m in models]
ax.scatter(conv_epochs, perfs_list, s=100, c='blue', alpha=0.6, edgecolors='black')
for i, m in enumerate(models):
ax.annotate(m, (conv_epochs[i], perfs_list[i]),
fontsize=8, alpha=0.7)
ax.set_xlabel('Convergence Epoch')
ax.set_ylabel('Final Performance')
ax.set_title('Convergence Speed vs Final Performance')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('ablation_modules.png', dpi=150, bbox_inches='tight')
print("Saved ablation_modules.png")
def visualize_loss_sensitivity():
"""Visualize loss function sensitivity analysis."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
analyzer = LossSensitivityAnalyzer()
# Plot 1: Weight space heatmap
ax = axes[0, 0]
df = analyzer.grid_search(resolution=20)
pivot = df.pivot(index='grasp_quality_weight',
columns='collision_weight',
values='performance')
im = ax.imshow(pivot.values, aspect='auto', cmap='RdYlGn',
extent=[0.1, 2.0, 0.1, 2.0], origin='lower')
ax.set_xlabel('Collision Weight')
ax.set_ylabel('Grasp Quality Weight')
ax.set_title('Performance Heatmap (2D Weight Space)')
plt.colorbar(im, ax=ax, label='Success Rate')
# Mark optimal
opt_idx = df['performance'].idxmax()
opt = df.iloc[opt_idx]
ax.plot(opt['collision_weight'], opt['grasp_quality_weight'],
'r*', markersize=15, label='Optimal')
ax.legend()
# Plot 2: Sensitivity gradients
ax = axes[0, 1]
base_weights = {term: 1.0 for term in analyzer.loss_terms}
grads = analyzer.sensitivity_gradient(base_weights)
terms = list(grads.keys())
sensitivities = list(grads.values())
bars = ax.bar(terms, sensitivities, color='purple', alpha=0.7, edgecolor='black')
ax.set_ylabel('Sensitivity (|dPerf/dw|)')
ax.set_title('Loss Term Sensitivity at Baseline')
plt.setp(ax.xaxis.get_majorticklabels(), rotation=45, ha='right')
ax.grid(True, alpha=0.3, axis='y')
# Plot 3: Stability region
ax = axes[1, 0]
# Identify stable region (perf > 0.80)
stable = df[df['stability'] == True]
unstable = df[df['stability'] == False]
ax.scatter(stable['grasp_quality_weight'], stable['collision_weight'],
c='green', alpha=0.5, label='Stable (acc>0.8)', s=20)
ax.scatter(unstable['grasp_quality_weight'], unstable['collision_weight'],
c='red', alpha=0.5, label='Unstable', s=20)
ax.set_xlabel('Grasp Quality Weight')
ax.set_ylabel('Collision Weight')
ax.set_title('Training Stability Regions')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 4: Pareto frontier for multi-objective
ax = axes[1, 1]
# Simulate multiple runs with different weight combinations
np.random.seed(42)
n_runs = 50
grasp_qualities = []
collision_avoidance = []
for _ in range(n_runs):
w_q = np.random.uniform(0.1, 2.0)
w_c = np.random.uniform(0.1, 2.0)
# Tradeoff: higher w_q improves grasp but may increase collision
q_score = 0.7 + 0.2 * (w_q / (w_q + 0.5)) + np.random.randn() * 0.05
c_score = 0.9 - 0.3 * (w_q / (w_q + 1.0)) + 0.1 * (w_c / (w_c + 0.5)) + np.random.randn() * 0.05
grasp_qualities.append(q_score)
collision_avoidance.append(c_score)
ax.scatter(grasp_qualities, collision_avoidance, c='blue', alpha=0.5, s=50)
# Compute Pareto frontier
points = np.array(list(zip(grasp_qualities, collision_avoidance)))
pareto = []
for i, p in enumerate(points):
dominated = False
for j, q in enumerate(points):
if i != j and q[0] >= p[0] and q[1] >= p[1] and (q[0] > p[0] or q[1] > p[1]):
dominated = True
break
if not dominated:
pareto.append(p)
pareto = sorted(pareto, key=lambda x: x[0])
pareto = np.array(pareto)
ax.plot(pareto[:, 0], pareto[:, 1], 'r-', linewidth=2, label='Pareto Frontier')
ax.set_xlabel('Grasp Quality Score')
ax.set_ylabel('Collision Avoidance Score')
ax.set_title('Multi-objective Tradeoff')
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('loss_sensitivity.png', dpi=150, bbox_inches='tight')
print("Saved loss_sensitivity.png")
def visualize_component_analysis():
"""Visualize detailed component analysis."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
# Plot 1: Feature importance waterfall
ax = axes[0, 0]
components = ['Backbone', 'Attention', 'Contact\nHead', 'Collision\nHead', 'Refinement']
impacts = [0.30, 0.08, 0.12, 0.15, 0.05]
cumulative = np.cumsum([0.4] + impacts)
# Waterfall chart
x = np.arange(len(components))
bottom = cumulative[:-1]
heights = impacts
colors = ['green' if h > 0.1 else 'yellow' if h > 0.05 else 'red' for h in heights]
bars = ax.bar(x, heights, bottom=bottom[:-1], color=colors, edgecolor='black')
ax.plot([-0.5, len(components)-0.5], [0.88, 0.88], 'r--', label='Full Model')
ax.set_xticks(x)
ax.set_xticklabels(components)
ax.set_ylabel('Cumulative Performance')
ax.set_title('Component Contribution (Waterfall)')
ax.legend()
ax.grid(True, alpha=0.3, axis='y')
# Plot 2: Ablation impact matrix
ax = axes[0, 1]
# Matrix showing pairwise interactions
n = len(components)
interaction_matrix = np.random.rand(n, n) * 0.05
np.fill_diagonal(interaction_matrix, 0)
# Make symmetric
interaction_matrix = (interaction_matrix + interaction_matrix.T) / 2
im = ax.imshow(interaction_matrix, cmap='Reds')
ax.set_xticks(range(n))
ax.set_yticks(range(n))
ax.set_xticklabels(components, rotation=45, ha='right')
ax.set_yticklabels(components)
ax.set_title('Component Interaction Effects')
plt.colorbar(im, ax=ax, label='Interaction Strength')
# Plot 3: Robustness to ablation
ax = axes[1, 0]
# Test robustness: how much can we remove before catastrophic failure
removal_order = ['Refinement', 'Attention', 'Contact Head', 'Collision Head', 'Backbone']
remaining_perf = [0.88, 0.83, 0.75, 0.63, 0.40]
ax.plot(range(len(removal_order)), remaining_perf, 'o-', linewidth=2, markersize=8)
ax.axhline(y=0.6, color='red', linestyle='--', label='Minimum viable')
ax.fill_between(range(len(removal_order)), 0.6, remaining_perf,
where=[p > 0.6 for p in remaining_perf],
alpha=0.3, color='green', label='Operational')
ax.set_xticks(range(len(removal_order)))
ax.set_xticklabels(removal_order, rotation=45, ha='right')
ax.set_ylabel('Remaining Performance')
ax.set_title('Cumulative Ablation Robustness')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 4: Efficiency vs importance scatter
ax = axes[1, 1]
# Compute cost and benefit for each component
costs = [100, 50, 30, 40, 20] # FLOPs or parameters
benefits = [30, 8, 12, 15, 5] # Performance gain
scatter = ax.scatter(costs, benefits, s=200, c=benefits, cmap='RdYlGn',
edgecolors='black', alpha=0.7)
for i, comp in enumerate(components):
ax.annotate(comp, (costs[i], benefits[i]),
fontsize=9, ha='center', va='bottom')
# Efficiency frontier (benefit/cost)
efficiency = np.array(benefits) / np.array(costs)
for i, eff in enumerate(efficiency):
if eff > 0.15: # High efficiency threshold
ax.plot(costs[i], benefits[i], 'g*', markersize=15)
ax.set_xlabel('Computational Cost (relative)')
ax.set_ylabel('Performance Benefit (%)')
ax.set_title('Cost-Benefit Analysis')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('component_analysis.png', dpi=150, bbox_inches='tight')
print("Saved component_analysis.png")
if __name__ == "__main__":
print("Generating ablation study visualizations...")
visualize_ablation_modules()
visualize_loss_sensitivity()
visualize_component_analysis()
print("\nCompleted. Check output PNG files.")script_data_scaling.py
该脚本实现数据规模影响分析，包含训练数据量学习曲线、数据增强策略对比、样本效率评估。可视化展示数据量-性能曲线、增强策略效果对比与数据效率边界。
Python
#!/usr/bin/env python3
"""
Data Scaling Analysis: Dataset Size and Augmentation Effects
Content: 8.3.2 Data Scale Impact Analysis (8.3.2.1-8.3.2.2)
Usage: python script_data_scaling.py
Output: data_scaling_curves.png, augmentation_effects.png, data_efficiency.png
"""
import numpy as np
import matplotlib.pyplot as plt
from typing import Dict, List, Tuple
import matplotlib.patches as mpatches
class DataScalingExperiment:
"""
Simulate learning with varying dataset sizes.
"""
def __init__(self):
self.max_performance = 0.92
self.noise_level = 0.015
def learning_curve(self, n_samples: int, method: str = 'deep_learning') -> float:
"""
Simulate performance vs dataset size.
"""
# Power law learning curve: perf = a * n^b + c
if method == 'analytical':
# Saturates quickly, needs less data
a, b, c = 0.3, 0.1, 0.60
saturation = 1000
elif method == 'deep_learning':
# Needs more data, slower saturation
a, b, c = 0.25, 0.15, 0.40
saturation = 50000
else: # RL
# Needs most data
a, b, c = 0.30, 0.12, 0.30
saturation = 100000
# Saturation effect
effective_n = min(n_samples, saturation)
perf = a * (effective_n ** b) / (saturation ** b) * (self.max_performance - c) + c
# Add noise
perf += np.random.randn() * self.noise_level
return min(self.max_performance, max(0, perf))
def sample_efficiency(self, target_perf: float = 0.80) -> Dict[str, int]:
"""
Compute samples needed to reach target performance.
"""
methods = ['analytical', 'deep_learning', 'rl']
efficiency = {}
for method in methods:
# Binary search for crossing point
n_low, n_high = 100, 200000
while n_high - n_low > 100:
n_mid = (n_low + n_high) // 2
perf = self.learning_curve(n_mid, method)
if perf < target_perf:
n_low = n_mid
else:
n_high = n_mid
efficiency[method] = n_mid
return efficiency
class AugmentationAnalyzer:
"""
Analyze data augmentation strategies.
"""
def __init__(self):
self.strategies = {
'none': {'gain': 0.0, 'robustness': 0.0},
'geometric': {'gain': 0.05, 'robustness': 0.08},
'photometric': {'gain': 0.03, 'robustness': 0.12},
'domain_randomization': {'gain': 0.08, 'robustness': 0.15},
'combined': {'gain': 0.12, 'robustness': 0.20}
}
def apply_augmentation(self, base_perf: float, strategy: str,
alpha: float = 1.0) -> float:
"""
Apply augmentation effect with coefficient.
"""
gain = self.strategies[strategy]['gain'] * alpha
robustness = self.strategies[strategy]['robustness'] * alpha
# Diminishing returns at high alpha
if alpha > 1.5:
gain *= 0.8
robustness *= 0.8
perf = base_perf + gain + np.random.randn() * 0.01
return min(0.95, perf), robustness
def compute_augmentation_coefficient(self, n_real_samples: int,
target_equiv: int = 10000) -> float:
"""
Compute effective augmentation coefficient based on real data scarcity.
"""
# More augmentation needed when real data is scarce
ratio = target_equiv / (n_real_samples + 100)
return min(3.0, max(0.5, ratio ** 0.5))
def visualize_data_scaling():
"""Visualize data scaling curves."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
exp = DataScalingExperiment()
# Plot 1: Learning curves by method
ax = axes[0, 0]
sample_sizes = np.logspace(2, 5.5, 30).astype(int)
methods = ['analytical', 'deep_learning', 'rl']
colors = ['blue', 'green', 'red']
for method, color in zip(methods, colors):
perfs = [exp.learning_curve(n, method) for n in sample_sizes]
ax.plot(sample_sizes, perfs, 'o-', color=color, label=method, alpha=0.7)
ax.set_xscale('log')
ax.set_xlabel('Training Samples')
ax.set_ylabel('Success Rate')
ax.set_title('Learning Curves by Method')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(0.5, 0.95)
# Plot 2: Sample efficiency comparison
ax = axes[0, 1]
targets = [0.70, 0.75, 0.80, 0.85]
methods = ['analytical', 'deep_learning', 'rl']
x = np.arange(len(targets))
width = 0.25
for i, method in enumerate(methods):
samples_needed = []
for target in targets:
eff = exp.sample_efficiency(target)
samples_needed.append(eff[method])
# Log scale for visualization
log_samples = np.log10(samples_needed)
ax.bar(x + i*width, log_samples, width, label=method, alpha=0.8)
ax.set_ylabel('Log10(Samples Needed)')
ax.set_title('Sample Efficiency by Target Performance')
ax.set_xticks(x + width)
ax.set_xticklabels([f'{t:.0%}' for t in targets])
ax.legend()
ax.grid(True, alpha=0.3, axis='y')
# Plot 3: Data efficiency frontier
ax = axes[1, 0]
# Performance vs computational cost (proportional to data)
for method, color in zip(methods, colors):
samples = np.logspace(2, 5, 20)
perfs = [exp.learning_curve(n, method) for n in samples]
cost = samples / 1000 # Normalized cost
ax.plot(cost, perfs, color=color, linewidth=2, label=method)
# Mark efficiency points
eff_80_idx = next(i for i, p in enumerate(perfs) if p > 0.80)
ax.scatter(cost[eff_80_idx], perfs[eff_80_idx],
s=100, color=color, marker='*', zorder=5)
ax.set_xlabel('Computational Cost (relative)')
ax.set_ylabel('Performance')
ax.set_title('Cost-Performance Frontier')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_xscale('log')
# Plot 4: Saturation analysis
ax = axes[1, 1]
# Show diminishing returns
n_large = np.logspace(4, 6, 20)
for method, color in zip(['deep_learning', 'rl'], ['green', 'red']):
perfs = [exp.learning_curve(int(n), method) for n in n_large]
# Compute marginal gain
marginal = np.diff(perfs) / np.diff(np.log10(n_large))
ax.plot(n_large[1:], marginal, color=color, label=f'{method} marginal gain')
ax.axhline(y=0.01, color='black', linestyle='--', label='Diminishing threshold')
ax.set_xscale('log')
ax.set_xlabel('Training Samples')
ax.set_ylabel('Marginal Gain per Decade')
ax.set_title('Diminishing Returns Analysis')
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('data_scaling_curves.png', dpi=150, bbox_inches='tight')
print("Saved data_scaling_curves.png")
def visualize_augmentation_effects():
"""Visualize data augmentation effects."""
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
analyzer = AugmentationAnalyzer()
# Plot 1: Augmentation strategy comparison
ax = axes[0, 0]
strategies = list(analyzer.strategies.keys())
base_perf = 0.75
gains = []
robustness = []
for strategy in strategies:
perf, rob = analyzer.apply_augmentation(base_perf, strategy)
gains.append(perf - base_perf)
robustness.append(rob)
x = np.arange(len(strategies))
width = 0.35
bars1 = ax.bar(x - width/2, gains, width, label='Performance Gain', alpha=0.8)
bars2 = ax.bar(x + width/2, robustness, width, label='Robustness', alpha=0.8)
ax.set_ylabel('Improvement')
ax.set_title('Augmentation Strategy Comparison')
ax.set_xticks(x)
ax.set_xticklabels(strategies, rotation=45, ha='right')
ax.legend()
ax.grid(True, alpha=0.3, axis='y')
# Plot 2: Augmentation coefficient vs data scarcity
ax = axes[0, 1]
n_real = np.logspace(2, 4, 20)
coeffs = [analyzer.compute_augmentation_coefficient(int(n)) for n in n_real]
ax.plot(n_real, coeffs, 'b-', linewidth=2)
ax.axhline(y=1.0, color='red', linestyle='--', label='Baseline')
ax.fill_between(n_real, 1.0, coeffs, where=[c > 1.0 for c in coeffs],
alpha=0.3, color='green', label='Augmentation beneficial')
ax.set_xscale('log')
ax.set_xlabel('Real Samples Available')
ax.set_ylabel('Optimal Augmentation Coefficient')
ax.set_title('Augmentation Strength vs Data Scarcity')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 3: Effective dataset size
ax = axes[1, 0]
real_sizes = np.array([100, 500, 1000, 5000, 10000])
multipliers = {
'none': 1,
'geometric': 5,
'photometric': 3,
'domain_randomization': 10,
'combined': 20
}
for aug_type, mult in multipliers.items():
effective = real_sizes * mult
ax.plot(real_sizes, effective, 'o-', label=aug_type, linewidth=2)
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel('Real Samples')
ax.set_ylabel('Effective Samples')
ax.set_title('Effective Dataset Size by Augmentation')
ax.legend()
ax.grid(True, alpha=0.3)
# Plot 4: Over-augmentation analysis
ax = axes[1, 1]
alphas = np.linspace(0, 3, 30)
perfs = []
for alpha in alphas:
perf, _ = analyzer.apply_augmentation(0.75, 'combined', alpha)
perfs.append(perf)
ax.plot(alphas, perfs, 'b-', linewidth=2)
ax.axvline(x=1.0, color='green', linestyle='--', label='Standard')
# Find peak
peak_idx = np.argmax(perfs)
ax.axvline(x=alphas[peak_idx], color='red', linestyle='--',
label=f'Optimal (α={alphas[peak_idx]:.1f})')
ax.set_xlabel('Augmentation Coefficient α')
ax.set_ylabel('Performance')
ax.set_title('Over-Augmentation Effect')
ax.legend()
ax.grid(True, alpha=0.3)
# Mark over-augmentation region
ax.axvspan(2.0, 3.0, alpha=0.2, color='red', label='Over-augmentation')
plt.tight_layout()
plt.savefig('augmentation_effects.png', dpi=150, bbox_inches='tight')
print("Saved augmentation_effects.png")
def visualize_data_efficiency():
"""Visualize data efficiency metrics."""
fig, axes = plt.subplots(1, 2, figsize=(14, 6))
# Plot 1: Data efficiency frontier
ax = axes[0]
# Methods with different data efficiencies
methods_data = {
'GraspIt!': {'samples_80': 500, 'samples_90': 2000, 'color': 'blue'},
'PointNetGPD': {'samples_80': 5000, 'samples_90': 50000, 'color': 'green'},
'SAC-Grasp': {'samples_80': 20000, 'samples_90': 100000, 'color': 'red'}
}
for name, data in methods_data.items():
# Efficiency curve
samples = [data['samples_80'], data['samples_90']]
perfs = [0.80, 0.90]
ax.loglog(samples, perfs, 'o-', color=data['color'],
linewidth=2, label=name, markersize=8)
# Area under curve as efficiency metric
ax.fill_between(samples, 0.7, perfs, alpha=0.1, color=data['color'])
ax.set_xlabel('Training Samples')
ax.set_ylabel('Performance')
ax.set_title('Data Efficiency Comparison')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(0.75, 0.95)
# Plot 2: Efficiency metrics bar chart
ax = axes[1]
# Compute efficiency score (inverse of samples needed for 80%)
efficiency_scores = {}
for name, data in methods_data.items():
efficiency_scores[name] = 1.0 / (data['samples_80'] / 1000)
names = list(efficiency_scores.keys())
scores = list(efficiency_scores.values())
colors = [methods_data[n]['color'] for n in names]
bars = ax.barh(names, scores, color=colors, alpha=0.7, edgecolor='black')
ax.set_xlabel('Efficiency Score (higher is better)')
ax.set_title('Data Efficiency Ranking')
ax.grid(True, alpha=0.3, axis='x')
# Add value labels
for bar, score in zip(bars, scores):
width = bar.get_width()
ax.text(width, bar.get_y() + bar.get_height()/2.,
f'{score:.2f}', ha='left', va='center', fontsize=10)
plt.tight_layout()
plt.savefig('data_efficiency.png', dpi=150, bbox_inches='tight')
print("Saved data_efficiency.png")
if __name__ == "__main__":
print("Generating data scaling visualizations...")
visualize_data_scaling()
visualize_augmentation_effects()
visualize_data_efficiency()
print("\nCompleted. Check output PNG files.")
1628
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
