UserWarning: Could not find response key `[/INST]` in the following instance: [INST] ...
在微调模型的时候遇到该问题,log文件如下:
This instance will be ignored in loss calculation. Note, if this happens often, consider increasing the
max_seq_length
.
warnings.warn(/home/wanglinjie/anaconda3/envs/chatkbqa/lib/python3.9/site-packages/trl/trainer/utils.py:110: UserWarning: Could not find response key
[/INST]
in the following instance: [INST] << SYS>><< SYS>>
找到的解决方案:
- https://github.com/huggingface/trl/issues/980
- https://discuss.huggingface.co/t/zero-loss-while-finetuning-llama2-usin-sft-trainer-and-the-use-of-collator/63831
这两个都提出,将collator进行修改:
变成response_template = “Answer: [/INST]”
collator = DataCollatorForCompletionOnlyLM(tokenizer.encode(response_template, add_special_tokens = False)[2:], tokenizer=tokenizer)
原因:
在NLP