题意:
Size mismatch for embed_out.weight: copying a param with shape torch.Size([0]) from checkpoint - Huggingface PyTorch
这个错误信息 "Size mismatch for embed_out.weight: copying a param with shape torch.Size([0]) from checkpoint - Huggingface PyTorch" 通常出现在使用 Hugging Face 的 Transformers 库加载预训练模型时,模型的某些参数与预训练模型检查点(checkpoint)中的参数形状不匹配。
问题背景:
I want to finetune an LLM. I am able to successfully finetune LLM. But when reload the model after save, gets error. Below is the code
想要微调(finetune)一个大型语言模型(LLM)并在保存后重新加载模型时遇到错误,以下是代码:
import argparse
import numpy as np
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM
from trl import DPOTr