SFT训练时,损失LOSS一直为0的问题解决方案

UserWarning: Could not find response key `[/INST]` in the following instance: [INST] ...


在微调模型的时候遇到该问题,log文件如下:
This instance will be ignored in loss calculation. Note, if this happens often, consider increasing the max_seq_length.
warnings.warn(/home/wanglinjie/anaconda3/envs/chatkbqa/lib/python3.9/site-packages/trl/trainer/utils.py:110: UserWarning: Could not find response key [/INST] in the following instance: [INST] << SYS>><< SYS>>

找到的解决方案:

  1. https://github.com/huggingface/trl/issues/980
  2. https://discuss.huggingface.co/t/zero-loss-while-finetuning-llama2-usin-sft-trainer-and-the-use-of-collator/63831

这两个都提出,将collator进行修改:
变成response_template = “Answer: [/INST]”
collator = DataCollatorForCompletionOnlyLM(tokenizer.encode(response_template, add_special_tokens = False)[2:], tokenizer=tokenizer)

原因:
在NLP

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值