torch.nn.functional.cosine_similarity使用详解

本文解析了如何在PyTorch中使用`F.cosine_similarity`函数计算矩阵间的余弦相似度,特别关注了dim参数的选择及其对结果的影响,通过实例演示了dim=0和dim=1的不同效果,并用Scipy库进行了验证。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

更多、更及时内容欢迎留意微信公众号小窗幽记机器学习

概述

根据官网文档的描述,其中 dim表示沿着对应的维度计算余弦相似。那么怎么理解呢?
首先,先介绍下所谓的dim:

a = torch.tensor([[ [1, 2], [3, 4] ], [ [5, 6], [7, 8] ] ], dtype=torch.float)
print(a.shape)
"""
[
    [
        [1, 2],
        [3, 4]
    ],
    [
        [5, 6],
        [7, 8]
    ]
]
"""

在这里插入图片描述

假设有2个矩阵:[[1, 2], [3, 4]] 和 [[5, 6], [7, 8]], 求2者的余弦相似。

按照dim=0求余弦相似:

import torch.nn.functional as F
input1 = torch.tensor([[1, 2], [3, 4]], dtype=torch.float)
input2 = torch.tensor([[5, 6], [7, 8]], dtype=torch.float)
output = F.cosine_similarity(input1, input2, dim=0)
print(output)

结果如下:

tensor([0.9558, 0.9839])

那么,这个数值是怎么得来的?是按照

在这里插入图片描述

具体求解如下:

print(F.cosine_similarity(torch.tensor([1,3], dtype=torch.float) , torch.tensor([5,7], dtype=torch.float), dim=0))
print(F.cosine_similarity(torch.tensor([2,4], dtype=torch.float) , torch.tensor([6,8], dtype=torch.float), dim=0))

运行结果如下:

tensor(0.9558)
tensor(0.9839)

可以用scipy.spatial进一步佐证:

from scipy import spatial

dataSetI = [1,3]
dataSetII = [5,7]
result = 1 - spatial.distance.cosine(dataSetI, dataSetII)
print(result)

运行结果如下:

0.95577900872195

同理:

dataSetI = [2,4]
dataSetII = [6,8]
result = 1 - spatial.distance.cosine(dataSetI, dataSetII)
print(result)

运行结果如下:

0.9838699100999074

按照dim=1求余弦相似:

output = F.cosine_similarity(input1, input2, dim=1)
print(output)

运行结果如下:

tensor([0.9734, 0.9972])

同理,用用scipy.spatial进一步佐证:

dataSetI = [1,2]
dataSetII = [5,6]
result = 1 - spatial.distance.cosine(dataSetI, dataSetII)
print(result)

运行结果:0.973417168333576

dataSetI = [3,4]
dataSetII = [7,8]
result = 1 - spatial.distance.cosine(dataSetI, dataSetII)
print(result)

运行结果:

0.9971641204866132

结果与F.cosine_similarity相符合。

【更多、更及时内容欢迎留意微信公众号小窗幽记机器学习

(style_tune) C:\Users\28996\Desktop\AI\persona_contrastive_finetuning>python Contrastive_Training_LM.py INFO:accelerate.utils.modeling:We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk). trainable params: 1,572,864 || all params: 1,838,401,536 || trainable%: 0.0856 训练集样本示例: {'anchor_input_ids': [56568, 118919, 116122, 11319], 'positive_input_ids': [116122, 20412, 107340, 9370, 100357, 102323, 3837, 109202, 104078, 103975, 100675, 101940, 100912, 105054, 6313], 'negative_input_ids': [100323, 104307, 99245, 9370, 106059, 104060, 3837, 104530, 115604, 99329, 11319]} 验证集样本示例: {'anchor_input_ids': [56568, 118919, 116122, 11319], 'positive_input_ids': [116122, 20412, 107340, 9370, 100357, 102323, 3837, 109202, 104078, 103975, 100675, 101940, 100912, 105054, 6313], 'negative_input_ids': [100323, 104307, 99245, 9370, 106059, 104060, 3837, 104530, 115604, 99329, 11319]} Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. INFO:__main__:GPU内存使用: 已分配 1.77GB, 保留 1.81GB 0%| | 0/3 [00:00<?, ?it/s]You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. Traceback (most recent call last): File "C:\Users\28996\Desktop\AI\persona_contrastive_finetuning\Contrastive_Training_LM.py", line 349, in <module> trainer.train() File "C:\Users\28996\miniconda3\envs\style_tune\lib\site-packages\transformers\trainer.py", line 2171, in train return inner_training_loop( File "C:\Users\28996\miniconda3\envs\style_tune\lib\site-packages\transformers\trainer.py", line 2531, in _inner_training_loop tr_loss_step = self.training_step(model, inputs, num_items_in_batch) File "C:\Users\28996\miniconda3\envs\style_tune\lib\site-packages\transformers\trainer.py", line 3676, in training_step loss = self.compute_loss(model, inputs) File "C:\Users\28996\Desktop\AI\persona_contrastive_finetuning\Contrastive_Training_LM.py", line 191, in compute_loss anchor_ids = anchor_ids.requires_grad_() RuntimeError: only Tensors of floating point dtype can require gradients 0%| | 0/3 [00:00<?, ?it/s]
07-21
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值