【Transformer】llama2.c探秘

Once upon a time, there was a big red barn on a farm. Inside the barn, there were many animals. There was a cow, a pig, and a chicken. The cow was very reliable. She always did her best to help her friends.
One day, the cow saw a small boy. The boy was playing with a ball. The cow wanted to help the boy. She wanted to make the boy happy. So, the cow went to the boy and gave him the ball. The boy smiled and said, “Thank you, cow!”
From that day on, the boy and the cow became best friends. They played with the ball every day. The boy would hold the ball and pull the string. The cow would help him throw the ball, and the boy would laugh and play. They were very happy together on the farm.

从前,在一个农场上有一座大红色的谷仓。谷仓里住着许多动物,有一头牛、一只猪和一只鸡。这头牛非常可靠,总是尽力帮助她的朋友们。

一天,这头牛看到一个小男孩。男孩正在玩球。牛想帮助男孩,让他开心。于是,牛走向男孩,把球给了他。男孩笑了起来,说:“谢谢你,牛!”

从那天起,男孩和牛成了最好的朋友。他们每天一起玩球。男孩抓住球的绳子,牛帮助他把球扔出去,男孩笑着玩耍。他们在农场里非常开心地生活在一起。

https://github.com/karpathy/llama2.c

非常容易运行,现在,让我们一起探秘吧

C:\Users\15251\PycharmProjects\3.12.7\.venv\Scripts\python.exe C:\Users\15251\PycharmProjects\3.12.7\1.py Traceback (most recent call last): File "C:\Users\15251\PycharmProjects\3.12.7\1.py", line 24, in <module> model.load_state_dict(torch.load(r'D:\esmc_600m_2024_12_v0.pth')) File "C:\Users\15251\PycharmProjects\3.12.7\.venv\Lib\site-packages\torch\nn\modules\module.py", line 2624, in load_state_dict raise RuntimeError( RuntimeError: Error(s) in loading state_dict for ESMC600M: Missing key(s) in state_dict: "layer1.weight", "layer1.bias", "layer2.weight", "layer2.bias", "layer3.weight", "layer3.bias". Unexpected key(s) in state_dict: "transformer.blocks.0.attn.layernorm_qkv.0.weight", "transformer.blocks.0.attn.layernorm_qkv.0.bias", "transformer.blocks.0.attn.layernorm_qkv.1.weight", "transformer.blocks.0.attn.q_ln.weight", "transformer.blocks.0.attn.k_ln.weight", "transformer.blocks.0.attn.out_proj.weight", "transformer.blocks.0.ffn.0.weight", "transformer.blocks.0.ffn.0.bias", "transformer.blocks.0.ffn.1.weight", "transformer.blocks.0.ffn.3.weight", "transformer.blocks.1.attn.layernorm_qkv.0.weight", "transformer.blocks.1.attn.layernorm_qkv.0.bias", "transformer.blocks.1.attn.layernorm_qkv.1.weight", "transformer.blocks.1.attn.q_ln.weight", "transformer.blocks.1.attn.k_ln.weight", "transformer.blocks.1.attn.out_proj.weight", "transformer.blocks.1.ffn.0.weight", "transformer.blocks.1.ffn.0.bias", "transformer.blocks.1.ffn.1.weight", "transformer.blocks.1.ffn.3.weight", "transformer.blocks.2.attn.layernorm_qkv.0.weight", "transformer.blocks.2.attn.layernorm_qkv.0.bias", "transformer.blocks.2.attn.layernorm_qkv.1.weight", "transformer.blocks.2.attn.q_ln.weight", "transformer.blocks.2.attn.k_ln.weight", "transformer.blocks.2.attn.out_proj.weight", "transformer.blocks.2.ffn.0.weight", "transformer.blocks.2.ffn.0.bias", "transformer.blocks.2.ffn.1.weight", "transformer.blocks.2.ffn.3.weight", "transformer.blocks.3.attn.layernorm_qkv.0.weight", "transformer.blocks.3.attn.layernorm_qkv.0.bias", "transformer.blocks.3.attn.layernorm_qkv.1.weight", "transformer.blocks.3.attn.q_ln.weight", "transformer.blocks.3.attn.k_ln.weight", "transformer.blocks.3.attn.out_proj.weight", "transformer.blocks.3.ffn.0.weight", "transformer.blocks.3.ffn.0.bias", "transformer.blocks.3.ffn.1.weight", "transformer.blocks.3.ffn.3.weight", "transformer.blocks.4.attn.layernorm_qkv.0.weight", "transformer.blocks.4.attn.layernorm_qkv.0.bias", "transformer.blocks.4.attn.layernorm_qkv.1.weight", "transformer.blocks.4.attn.q_ln.weight", "transformer.blocks.4.attn.k_ln.weight", "transformer.blocks.4.attn.out_proj.weight", "transformer.blocks.4.ffn.0.weight", "transformer.blocks.4.ffn.0.bias", "transformer.blocks.4.ffn.1.weight", "transformer.blocks.4.ffn.3.weight", "transformer.blocks.5.attn.layernorm_qkv.0.weight", "transformer.blocks.5.attn.layernorm_qkv.0.bias", "transformer.blocks.5.attn.layernorm_qkv.1.weight", "transformer.blocks.5.attn.q_ln.weight", "transformer.blocks.5.attn.k_ln.weight", "transformer.blocks.5.attn.out_proj.weight", "transformer.blocks.5.ffn.0.weight", "transformer.blocks.5.ffn.0.bias", "transformer.blocks.5.ffn.1.weight", "transformer.blocks.5.ffn.3.weight", "transformer.blocks.6.attn.layernorm_qkv.0.weight", "transformer.blocks.6.attn.layernorm_qkv.0.bias", "transformer.blocks.6.attn.layernorm_qkv.1.weight", "transformer.blocks.6.attn.q_ln.weight", "transformer.blocks.6.attn.k_ln.weight", "transformer.blocks.6.attn.out_proj.weight", "transformer.blocks.6.ffn.0.weight", "transformer.blocks.6.ffn.0.bias", "transformer.blocks.6.ffn.1.weight", "transformer.blocks.6.ffn.3.weight", "transformer.blocks.7.attn.layernorm_qkv.0.weight", "transformer.blocks.7.attn.layernorm_qkv.0.bias", "transformer.blocks.7.attn.layernorm_qkv.1.weight", "transformer.blocks.7.attn.q_ln.weight", "transformer.blocks.7.attn.k_ln.weight", "transformer.blocks.7.attn.out_proj.weight", "transformer.blocks.7.ffn.0.weight", "transformer.blocks.7.ffn.0.bias", "transformer.blocks.7.ffn.1.weight", "transformer.blocks.7.ffn.3.weight", "transformer.blocks.8.attn.layernorm_qkv.0.weight", "transformer.blocks.8.attn.layernorm_qkv.0.bias", "transformer.blocks.8.attn.layernorm_qkv.1.weight", "transformer.blocks.8.attn.q_ln.weight", "transformer.blocks.8.attn.k_ln.weight", "transformer.blocks.8.attn.out_proj.weight", "transformer.blocks.8.ffn.0.weight", "transformer.blocks.8.ffn.0.bias", "transformer.blocks.8.ffn.1.weight", "transformer.blocks.8.ffn.3.weight", "transformer.blocks.9.attn.layernorm_qkv.0.weight", "transformer.blocks.9.attn.layernorm_qkv.0.bias", "transformer.blocks.9.attn.layernorm_qkv.1.weight", "transformer.blocks.9.attn.q_ln.weight", "transformer.blocks.9.attn.k_ln.weight", "transformer.blocks.9.attn.out_proj.weight", "transformer.blocks.9.ffn.0.weight", "transformer.blocks.9.ffn.0.bias", "transformer.blocks.9.ffn.1.weight", "transformer.blocks.9.ffn.3.weight", "transformer.blocks.10.attn.layernorm_qkv.0.weight", "transformer.blocks.10.attn.layernorm_qkv.0.bias", "transformer.blocks.10.attn.layernorm_qkv.1.weight", "transformer.blocks.10.attn.q_ln.weight", "transformer.blocks.10.attn.k_ln.weight", "transformer.blocks.10.attn.out_proj.weight", "transformer.blocks.10.ffn.0.weight", "transformer.blocks.10.ffn.0.bias", "transformer.blocks.10.ffn.1.weight", "transformer.blocks.10.ffn.3.weight", "transformer.blocks.11.attn.layernorm_qkv.0.weight", "transformer.blocks.11.attn.layernorm_qkv.0.bias", "transformer.blocks.11.attn.layernorm_qkv.1.weight", "transformer.blocks.11.attn.q_ln.weight", "transformer.blocks.11.attn.k_ln.weight", "transformer.blocks.11.attn.out_proj.weight", "transformer.blocks.11.ffn.0.weight", "transformer.blocks.11.ffn.0.bias", "transformer.blocks.11.ffn.1.weight", "transformer.blocks.11.ffn.3.weight", "transformer.blocks.12.attn.layernorm_qkv.0.weight", "transformer.blocks.12.attn.layernorm_qkv.0.bias", "transformer.blocks.12.attn.layernorm_qkv.1.weight", "transformer.blocks.12.attn.q_ln.weight", "transformer.blocks.12.attn.k_ln.weight", "transformer.blocks.12.attn.out_proj.weight", "transformer.blocks.12.ffn.0.weight", "transformer.blocks.12.ffn.0.bias", "transformer.blocks.12.ffn.1.weight", "transformer.blocks.12.ffn.3.weight", "transformer.blocks.13.attn.layernorm_qkv.0.weight", "transformer.blocks.13.attn.layernorm_qkv.0.bias", "transformer.blocks.13.attn.layernorm_qkv.1.weight", "transformer.blocks.13.attn.q_ln.weight", "transformer.blocks.13.attn.k_ln.weight", "transformer.blocks.13.attn.out_proj.weight", "transformer.blocks.13.ffn.0.weight", "transformer.blocks.13.ffn.0.bias", "transformer.blocks.13.ffn.1.weight", "transformer.blocks.13.ffn.3.weight", "transformer.blocks.14.attn.layernorm_qkv.0.weight", "transformer.blocks.14.attn.layernorm_qkv.0.bias", "transformer.blocks.14.attn.layernorm_qkv.1.weight", "transformer.blocks.14.attn.q_ln.weight", "transformer.blocks.14.attn.k_ln.weight", "transformer.blocks.14.attn.out_proj.weight", "transformer.blocks.14.ffn.0.weight", "transformer.blocks.14.ffn.0.bias", "transformer.blocks.14.ffn.1.weight", "transformer.blocks.14.ffn.3.weight", "transformer.blocks.15.attn.layernorm_qkv.0.weight", "transformer.blocks.15.attn.layernorm_qkv.0.bias", "transformer.blocks.15.attn.layernorm_qkv.1.weight", "transformer.blocks.15.attn.q_ln.weight", "transformer.blocks.15.attn.k_ln.weight", "transformer.blocks.15.attn.out_proj.weight", "transformer.blocks.15.ffn.0.weight", "transformer.blocks.15.ffn.0.bias", "transformer.blocks.15.ffn.1.weight", "transformer.blocks.15.ffn.3.weight", "transformer.blocks.16.attn.layernorm_qkv.0.weight", "transformer.blocks.16.attn.layernorm_qkv.0.bias", "transformer.blocks.16.attn.layernorm_qkv.1.weight", "transformer.blocks.16.attn.q_ln.weight", "transformer.blocks.16.attn.k_ln.weight", "transformer.blocks.16.attn.out_proj.weight", "transformer.blocks.16.ffn.0.weight", "transformer.blocks.16.ffn.0.bias", "transformer.blocks.16.ffn.1.weight", "transformer.blocks.16.ffn.3.weight", "transformer.blocks.17.attn.layernorm_qkv.0.weight", "transformer.blocks.17.attn.layernorm_qkv.0.bias", "transformer.blocks.17.attn.layernorm_qkv.1.weight", "transformer.blocks.17.attn.q_ln.weight", "transformer.blocks.17.attn.k_ln.weight", "transformer.blocks.17.attn.out_proj.weight", "transformer.blocks.17.ffn.0.weight", "transformer.blocks.17.ffn.0.bias", "transformer.blocks.17.ffn.1.weight", "transformer.blocks.17.ffn.3.weight", "transformer.blocks.18.attn.layernorm_qkv.0.weight", "transformer.blocks.18.attn.layernorm_qkv.0.bias", "transformer.blocks.18.attn.layernorm_qkv.1.weight", "transformer.blocks.18.attn.q_ln.weight", "transformer.blocks.18.attn.k_ln.weight", "transformer.blocks.18.attn.out_proj.weight", "transformer.blocks.18.ffn.0.weight", "transformer.blocks.18.ffn.0.bias", "transformer.blocks.18.ffn.1.weight", "transformer.blocks.18.ffn.3.weight", "transformer.blocks.19.attn.layernorm_qkv.0.weight", "transformer.blocks.19.attn.layernorm_qkv.0.bias", "transformer.blocks.19.attn.layernorm_qkv.1.weight", "transformer.blocks.19.attn.q_ln.weight", "transformer.blocks.19.attn.k_ln.weight", "transformer.blocks.19.attn.out_proj.weight", "transformer.blocks.19.ffn.0.weight", "transformer.blocks.19.ffn.0.bias", "transformer.blocks.19.ffn.1.weight", "transformer.blocks.19.ffn.3.weight", "transformer.blocks.20.attn.layernorm_qkv.0.weight", "transformer.blocks.20.attn.layernorm_qkv.0.bias", "transformer.blocks.20.attn.layernorm_qkv.1.weight", "transformer.blocks.20.attn.q_ln.weight", "transformer.blocks.20.attn.k_ln.weight", "transformer.blocks.20.attn.out_proj.weight", "transformer.blocks.20.ffn.0.weight", "transformer.blocks.20.ffn.0.bias", "transformer.blocks.20.ffn.1.weight", "transformer.blocks.20.ffn.3.weight", "transformer.blocks.21.attn.layernorm_qkv.0.weight", "transformer.blocks.21.attn.layernorm_qkv.0.bias", "transformer.blocks.21.attn.layernorm_qkv.1.weight", "transformer.blocks.21.attn.q_ln.weight", "transformer.blocks.21.attn.k_ln.weight", "transformer.blocks.21.attn.out_proj.weight", "transformer.blocks.21.ffn.0.weight", "transformer.blocks.21.ffn.0.bias", "transformer.blocks.21.ffn.1.weight", "transformer.blocks.21.ffn.3.weight", "transformer.blocks.22.attn.layernorm_qkv.0.weight", "transformer.blocks.22.attn.layernorm_qkv.0.bias", "transformer.blocks.22.attn.layernorm_qkv.1.weight", "transformer.blocks.22.attn.q_ln.weight", "transformer.blocks.22.attn.k_ln.weight", "transformer.blocks.22.attn.out_proj.weight", "transformer.blocks.22.ffn.0.weight", "transformer.blocks.22.ffn.0.bias", "transformer.blocks.22.ffn.1.weight", "transformer.blocks.22.ffn.3.weight", "transformer.blocks.23.attn.layernorm_qkv.0.weight", "transformer.blocks.23.attn.layernorm_qkv.0.bias", "transformer.blocks.23.attn.layernorm_qkv.1.weight", "transformer.blocks.23.attn.q_ln.weight", "transformer.blocks.23.attn.k_ln.weight", "transformer.blocks.23.attn.out_proj.weight", "transformer.blocks.23.ffn.0.weight", "transformer.blocks.23.ffn.0.bias", "transformer.blocks.23.ffn.1.weight", "transformer.blocks.23.ffn.3.weight", "transformer.blocks.24.attn.layernorm_qkv.0.weight", "transformer.blocks.24.attn.layernorm_qkv.0.bias", "transformer.blocks.24.attn.layernorm_qkv.1.weight", "transformer.blocks.24.attn.q_ln.weight", "transformer.blocks.24.attn.k_ln.weight", "transformer.blocks.24.attn.out_proj.weight", "transformer.blocks.24.ffn.0.weight", "transformer.blocks.24.ffn.0.bias", "transformer.blocks.24.ffn.1.weight", "transformer.blocks.24.ffn.3.weight", "transformer.blocks.25.attn.layernorm_qkv.0.weight", "transformer.blocks.25.attn.layernorm_qkv.0.bias", "transformer.blocks.25.attn.layernorm_qkv.1.weight", "transformer.blocks.25.attn.q_ln.weight", "transformer.blocks.25.attn.k_ln.weight", "transformer.blocks.25.attn.out_proj.weight", "transformer.blocks.25.ffn.0.weight", "transformer.blocks.25.ffn.0.bias", "transformer.blocks.25.ffn.1.weight", "transformer.blocks.25.ffn.3.weight", "transformer.blocks.26.attn.layernorm_qkv.0.weight", "transformer.blocks.26.attn.layernorm_qkv.0.bias", "transformer.blocks.26.attn.layernorm_qkv.1.weight", "transformer.blocks.26.attn.q_ln.weight", "transformer.blocks.26.attn.k_ln.weight", "transformer.blocks.26.attn.out_proj.weight", "transformer.blocks.26.ffn.0.weight", "transformer.blocks.26.ffn.0.bias", "transformer.blocks.26.ffn.1.weight", "transformer.blocks.26.ffn.3.weight", "transformer.blocks.27.attn.layernorm_qkv.0.weight", "transformer.blocks.27.attn.layernorm_qkv.0.bias", "transformer.blocks.27.attn.layernorm_qkv.1.weight", "transformer.blocks.27.attn.q_ln.weight", "transformer.blocks.27.attn.k_ln.weight", "transformer.blocks.27.attn.out_proj.weight", "transformer.blocks.27.ffn.0.weight", "transformer.blocks.27.ffn.0.bias", "transformer.blocks.27.ffn.1.weight", "transformer.blocks.27.ffn.3.weight", "transformer.blocks.28.attn.layernorm_qkv.0.weight", "transformer.blocks.28.attn.layernorm_qkv.0.bias", "transformer.blocks.28.attn.layernorm_qkv.1.weight", "transformer.blocks.28.attn.q_ln.weight", "transformer.blocks.28.attn.k_ln.weight", "transformer.blocks.28.attn.out_proj.weight", "transformer.blocks.28.ffn.0.weight", "transformer.blocks.28.ffn.0.bias", "transformer.blocks.28.ffn.1.weight", "transformer.blocks.28.ffn.3.weight", "transformer.blocks.29.attn.layernorm_qkv.0.weight", "transformer.blocks.29.attn.layernorm_qkv.0.bias", "transformer.blocks.29.attn.layernorm_qkv.1.weight", "transformer.blocks.29.attn.q_ln.weight", "transformer.blocks.29.attn.k_ln.weight", "transformer.blocks.29.attn.out_proj.weight", "transformer.blocks.29.ffn.0.weight", "transformer.blocks.29.ffn.0.bias", "transformer.blocks.29.ffn.1.weight", "transformer.blocks.29.ffn.3.weight", "transformer.blocks.30.attn.layernorm_qkv.0.weight", "transformer.blocks.30.attn.layernorm_qkv.0.bias", "transformer.blocks.30.attn.layernorm_qkv.1.weight", "transformer.blocks.30.attn.q_ln.weight", "transformer.blocks.30.attn.k_ln.weight", "transformer.blocks.30.attn.out_proj.weight", "transformer.blocks.30.ffn.0.weight", "transformer.blocks.30.ffn.0.bias", "transformer.blocks.30.ffn.1.weight", "transformer.blocks.30.ffn.3.weight", "transformer.blocks.31.attn.layernorm_qkv.0.weight", "transformer.blocks.31.attn.layernorm_qkv.0.bias", "transformer.blocks.31.attn.layernorm_qkv.1.weight", "transformer.blocks.31.attn.q_ln.weight", "transformer.blocks.31.attn.k_ln.weight", "transformer.blocks.31.attn.out_proj.weight", "transformer.blocks.31.ffn.0.weight", "transformer.blocks.31.ffn.0.bias", "transformer.blocks.31.ffn.1.weight", "transformer.blocks.31.ffn.3.weight", "transformer.blocks.32.attn.layernorm_qkv.0.weight", "transformer.blocks.32.attn.layernorm_qkv.0.bias", "transformer.blocks.32.attn.layernorm_qkv.1.weight", "transformer.blocks.32.attn.q_ln.weight", "transformer.blocks.32.attn.k_ln.weight", "transformer.blocks.32.attn.out_proj.weight", "transformer.blocks.32.ffn.0.weight", "transformer.blocks.32.ffn.0.bias", "transformer.blocks.32.ffn.1.weight", "transformer.blocks.32.ffn.3.weight", "transformer.blocks.33.attn.layernorm_qkv.0.weight", "transformer.blocks.33.attn.layernorm_qkv.0.bias", "transformer.blocks.33.attn.layernorm_qkv.1.weight", "transformer.blocks.33.attn.q_ln.weight", "transformer.blocks.33.attn.k_ln.weight", "transformer.blocks.33.attn.out_proj.weight", "transformer.blocks.33.ffn.0.weight", "transformer.blocks.33.ffn.0.bias", "transformer.blocks.33.ffn.1.weight", "transformer.blocks.33.ffn.3.weight", "transformer.blocks.34.attn.layernorm_qkv.0.weight", "transformer.blocks.34.attn.layernorm_qkv.0.bias", "transformer.blocks.34.attn.layernorm_qkv.1.weight", "transformer.blocks.34.attn.q_ln.weight", "transformer.blocks.34.attn.k_ln.weight", "transformer.blocks.34.attn.out_proj.weight", "transformer.blocks.34.ffn.0.weight", "transformer.blocks.34.ffn.0.bias", "transformer.blocks.34.ffn.1.weight", "transformer.blocks.34.ffn.3.weight", "transformer.blocks.35.attn.layernorm_qkv.0.weight", "transformer.blocks.35.attn.layernorm_qkv.0.bias", "transformer.blocks.35.attn.layernorm_qkv.1.weight", "transformer.blocks.35.attn.q_ln.weight", "transformer.blocks.35.attn.k_ln.weight", "transformer.blocks.35.attn.out_proj.weight", "transformer.blocks.35.ffn.0.weight", "transformer.blocks.35.ffn.0.bias", "transformer.blocks.35.ffn.1.weight", "transformer.blocks.35.ffn.3.weight", "transformer.norm.weight", "embed.weight", "sequence_head.0.weight", "sequence_head.0.bias", "sequence_head.2.weight", "sequence_head.2.bias", "sequence_head.3.weight", "sequence_head.3.bias". 进程已结束,退出代码为 1 报错如上,可以用“YKVNEYQGPVAPSGGFFLFRPRN”蛋白质序列简单尝试
最新发布
09-30
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值