huggingface网络连接问题:
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like BAAI/bge-large-en-v1.5 is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
例如:BAAI/bge-large-en-v1.5
model_name = "BAAI/bge-large-en-v1.5"
model_kwargs = {"device": "cpu"}
encode_kwargs = {"normalize_embeddings": True}
embedding_model = HuggingFaceBgeEmbeddings(
model_name=model_name, model_kwargs=model_kwargs, encode_kwargs=encode_kwargs
)
问题描述
OSError: We couldn‘t connect to ‘https://huggingface.co‘ to load this file, couldn‘t find it(亲测有效)
解决方案:
提示:https://hf-mirror.com/
点进去这个镜像网站,里面有四种解决办法
方案一:
去hugging face 上手动下载
方案二:
-
安装依赖
pip install -U huggingface_hub
-
设置环境变量
Linux
export HF_ENDPOINT=https://hf-mirror.com
Windows Powershell
$env:HF_ENDPOINT = "https://hf-mirror.com"
建议将上面这一行写入 ~/.bashrc。
3.1 下载模型
huggingface-cli download --resume-download gpt2 --local-dir gpt2
3.2 下载数据集
huggingface-cli download --repo-type dataset --resume-download wikitext --local-dir wikitext
可以添加 --local-dir-use-symlinks False 参数禁用文件软链接,这样下载路径下所见即所得,详细解释请见上面提到的教程。
方案三:
-
下载hfd
wget https://hf-mirror.com/hfd/hfd.sh chmod a+x hfd.sh
-
设置环境变量
Linux
export HF_ENDPOINT=https://hf-mirror.com
Windows Powershell
$env:HF_ENDPOINT = "https://hf-mirror.com"
-
下载模型
./hfd.sh gpt2
-
下载数据集
./hfd.sh wikitext --dataset