机器学习常用特征相似度或距离度量【Pytorch实现】

最新推荐文章于 2025-03-21 11:44:18 发布

liuz_notes

最新推荐文章于 2025-03-21 11:44:18 发布

阅读量1.8k

点赞数 1

分类专栏：知识经验分享文章标签： pytorch 深度学习机器学习度量学习特征度量

本文链接：https://blog.youkuaiyun.com/liuz_notes/article/details/120069436

版权

知识经验分享专栏收录该内容

8 篇文章

订阅专栏

本文介绍了如何在PyTorch中计算特征向量的余弦相似度、不同范数（F范数、L1和L∞）以及KL散度，包括针对三维矩阵的扩展方法。通过实例演示了这些技术在深度学习中的应用和计算步骤。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

余弦相似度、特征距离、KL散度计算

import torch

feat1 = torch.randn((3, 4))
feat2 = torch.randn((3, 4))

# ==========cosine相似度===============
a_norm = torch.linalg.norm(feat1, dim=1)
b_norm = torch.linalg.norm(feat2, dim=1)
cos = ((feat1 * feat2).sum(dim=(-2, -1)) / (a_norm * b_norm)).mean()
cos = 0.5 * cos + 0.5   # [-1, 1]  --> [0, 1]

# ==========矩阵范数===============
lfro = torch.linalg.norm(feat1 - feat2, dim=-1).mean()	# F范数
l1 = torch.linalg.norm(feat1 - feat2, dim=-1, ord=1).mean()
linf_max = torch.linalg.norm(feat1 - feat2, dim=-1, ord=float('inf')).mean()
linf_min = torch.linalg.norm(feat1 - feat2, dim=-1, ord=-float('inf')).mean()

# ==========KL散度===============
kl = torch.nn.functional.kl_div(feat1.softmax(dim=-1).log(), feat2.softmax(dim=-1), reduction='batchmean')	# 'mean'
kl2 = torch.nn.functional.kl_div(feat1.softmax(dim=-1).log(), feat2.softmax(dim=-1), reduction='sum')

若为三维矩阵：batch * channel * feature，则

import torch

feat1 = torch.randn((2, 3, 4))
feat2 = torch.randn((2, 3, 4))

# ==========cosine相似度===============
a_norm = torch.linalg.norm(feat1, dim=(-2, -1))
b_norm = torch.linalg.norm(feat2, dim=(-2, -1))
cos = ((feat1 * feat2).sum(dim=(-2, -1)) / (a_norm * b_norm)).mean()
cos = 0.5 * cos + 0.5   # [-1, 1]  --> [0, 1]

# ==========矩阵范数===============
lfro = torch.linalg.norm(feat1 - feat2, dim=(-2, -1)).mean()	# F范数
l1 = torch.linalg.norm(feat1 - feat2, dim=(-2, -1), ord=1).mean()
linf_max = torch.linalg.norm(feat1 - feat2, dim=(-2, -1), ord=float('inf')).mean()
linf_min = torch.linalg.norm(feat1 - feat2, dim=(-2, -1), ord=-float('inf')).mean()

# ==========KL散度===============
kl = torch.nn.functional.kl_div(feat1.softmax(dim=-1).log(), feat2.softmax(dim=-1), reduction='batchmean')	# 'mean'
kl2 = torch.nn.functional.kl_div(feat1.softmax(dim=-1).log(), feat2.softmax(dim=-1), reduction='sum')