在pytorch中使用用npz文件保存的预训练模型

最新推荐文章于 2025-04-24 16:41:59 发布

沙琪玛sqm

最新推荐文章于 2025-04-24 16:41:59 发布

阅读量1.3w

点赞数 27

分类专栏：深度学习跳坑日志

本文链接：https://blog.youkuaiyun.com/weixin_44020747/article/details/115208672

版权

开始

最近在看transformer相关的文章，在AN IMAGE IS WORTH 16X16 WORDS:
TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE这篇文章中，作者给出了在超大数据集上训练的模型。但是采用的是npz文件保存。这里将介绍npz文件和如何将其应用到pytorch中进行加载参数和训练。

1、什么是npz文件？

npz文件是python中numpy的文件存储。
使用方法如下：

from numpy import load
data = load('D:/CsdnProject/TransUNet-main/ViT-B_16.npz')
lst = data.files
for item in lst:
    print(item)
    print(data[item])

采用load的方式进行加载。然后打印其内容（其中加载路径为这次的预训练模型的路径）

2、如何将预训练模型加载到pytorch中进行使用？

2.1预训练模型npz文件中的内容

首先，npz文件中存放的内容有两个内容，一个是模型中每个模块的名称，第二个是这个模块保存的参数。
如1所示代码

from numpy import load
from os.path import join as join
data = load('D:/CsdnProject/TransUNet-main/ViT-B_16.npz')
lst = data.files
for item in lst:
    print(item)
    print(data[item])

将其打印出来的结果是（未显示参数）

C:\Users\sg\Anaconda3\envs\pytorch\python.exe D:/CsdnProject/TransUNet-main/TestNpz.py
Transformer/encoder_norm/bias
Transformer/encoder_norm/scale
Transformer/encoderblock_0/LayerNorm_0/bias
Transformer/encoderblock_0/LayerNorm_0/scale
Transformer/encoderblock_0/LayerNorm_2/bias
Transformer/encoderblock_0/LayerNorm_2/scale
Transformer/encoderblock_0/MlpBlock_3/Dense_0/bias
Transformer/encoderblock_0/MlpBlock_3/Dense_0/kernel
Transformer/encoderblock_0/MlpBlock_3/Dense_1/bias
Transformer/encoderblock_0/MlpBlock_3/Dense_1/kernel
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/key/bias
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/key/kernel
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/out/bias
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/out/kernel
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/query/bias
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/query/kernel
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/value/bias
Transformer/encoderblock_0/MultiHeadDotProductAttention_1/value/kernel
Transformer/encoderblock_1/LayerNorm_0/bias
Transformer/encoderblock_1/LayerNorm_0/scale
Transformer/encoderblock_1/LayerNorm_2/bias
Transformer/encoderblock_1/LayerNorm_2/scale
Transformer/encoderblock_1/MlpBlock_3/Dense_0/bias
Transformer/encoderblock_1/MlpBlock_3/Dense_0/kernel
Transformer/encoderblock_1/MlpBlock_3/Dense_1/bias
Transformer/encoderblock_1/MlpBlock_3/Dense_1/kernel
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/key/bias
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/key/kernel
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/out/bias
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/out/kernel
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/query/bias
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/query/kernel
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/value/bias
Transformer/encoderblock_1/MultiHeadDotProductAttention_1/value/kernel
Transformer/encoderblock_10/LayerNorm_0/bias
Transformer/encoderblock_10/LayerNorm_0/scale
Transformer/encoderblock_10/LayerNorm_2/bias
Transformer/encoderblock_10/LayerNorm_2/scale
Transformer/encoderblock_10/MlpBlock_3/Dense_0/bias
Transformer/encoderblock_10/MlpBlock_3/Dense_0/kernel
Transformer/encoderblock_10/MlpBlock_3/Dense_1/bias
Transformer/encoderblock_10/MlpBlock_3/Dense_1/kernel
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/key/bias
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/key/kernel
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/out/bias
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/out/kernel
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/query/bias
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/query/kernel
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/value/bias
Transformer/encoderblock_10/MultiHeadDotProductAttention_1/value/kernel
Transformer/encoderblock_11/LayerNorm_0/bias
Transformer/encoderblock_11/LayerNorm_0/scale
Transformer/encoderblock_11/LayerNorm_2/bias
Transformer/encoderblock_11/LayerNorm_2/scale
Transformer/encoderblock_11/MlpBlock_3/Dense_0/bias
Transformer/encoderblock_11/MlpBlock_3/Dense_0/kernel
Transformer/encoderblock_11/MlpBlock_3/Dense_1/bias
Transformer/encoderblock_11/MlpBlock_3/Dense_1/kernel
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/key/bias
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/key/kernel
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/out/bias
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/out/kernel
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/query/bias
Transformer/encoderblock_11/MultiHeadDotProductAttention_1/que

最低0.47元/天解锁文章