【Llama3源码】llama3-implemented-from-scratch源码运行踩坑Incorrect padding

Error: Incorrect padding

问题描述

学习naklecha/llama3-from-scratch代码时，首先要下载Meta-Llama-3-8B模型文件

in this file, i implemented llama3 from scratch, one tensor and matrix multiplication at a time. also, im going to load tensors directly from the model file that meta provided for llama3, you need to download the weights before running this file. here is the offical link to download the weights: https://llama.meta.com/llama-downloads/

结合上述说明以及Llama 3 使用方法以及模型下载教程链接方式下载Meta-Llama-3-8B模型文件后，运行下述代码：

from pathlib import Path
import tiktoken
from tiktoken.load import load_tiktoken_bpe
import torch
import json
import matplotlib.pyplot as plt

tokenizer_path =