sentence_transformers模型无法直接下载的解决方案

最新推荐文章于 2025-09-19 04:14:29 发布

原创最新推荐文章于 2025-09-19 04:14:29 发布 · 1.8w 阅读

30 ·

CC 4.0 BY-SA版权

人工智能学习笔记专栏收录该内容

277 篇文章

订阅专栏

诸神缄默不语-个人优快云博文目录

本文介绍使用sentence_transformers包（官网：https://huggingface.co/sentence-transformers）时，直接调用模型名称无法下载模型的解决方案。
其实跟transformers包差不多，都是把文件下载到本地然后直接使用路径加载模型。

一开始我用的代码是：

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')

好几次都在下载了一小部分之后失败了。

所以改为提前将模型下载到本地（wget稳定性更强，可以无限retry，我下pytorch_model.bin重试了8次）：

这个模型的网址很容易找到：https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
挨个下载文件到本地：

mkdir /data/pretrained_model/all-MiniLM-L6-v2
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/config.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/pytorch_model.bin
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/data_config.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/config_sentence_transformers.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/modules.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/sentence_bert_config.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/special_tokens_map.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer_config.json
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/train_script.py
wget -P /data/pretrained_model/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/vocab.txt
mkdir /data/pretrained_model/all-MiniLM-L6-v2/1_Pooling
wget -P /data/pretrained_model/all-MiniLM-L6-v2/1_Pooling https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/1_Pooling/config.json

然后代码直接改成：

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('/data/pretrained_model/all-MiniLM-L6-v2')

其他一切不变即可。