open_clip.create_model_and_transforms 加载本地模型

最新推荐文章于 2025-09-06 01:34:27 发布

原创最新推荐文章于 2025-09-06 01:34:27 发布 · 1w 阅读

16 ·

CC 4.0 BY-SA版权

文章标签：

#open_clip #多模态

多模态同时被 2 个专栏收录

2 篇文章

订阅专栏

open_clip

1 篇文章

订阅专栏

文章讲述了在使用open_clip创建模型时遇到的错误，由于缺少关键文件（如positional_embedding等），提示从HuggingFace下载正确的open_clip_pytorch_model.bin进行替换，解决了模型加载问题。

open_clip.create_model_and_transforms 加载本地模型

open_clip.create_model_and_transforms 加载本地模型
参考

open_clip.create_model_and_transforms 加载本地模型

代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin'
    )

pretrained 指定路径 laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin

报错

    model, _, preprocess = open_clip.create_model_and_transforms(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 382, in create_model_and_transforms
    model = create_model(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 288, in create_model
    load_checkpoint(model, checkpoint_path)
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 159, in load_checkpoint
    incompatible_keys = model.load_state_dict(state_dict, strict=strict)
  File "/data/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIP:
        Missing key(s) in state_dict: "positional_embedding", "text_projection", "visual.class_embedding", "visual.positional_embedding", "visual.proj"

解决方法

到 https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K/tree/main 下载 open_clip_pytorch_model.bin
修改代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/open_clip_pytorch_model.bin'
    )

再次执行，正常运行！！