qwen2.5-7b-instruct生成内容截断

最近在尝试微调qwen2.5-7b-instruct完成kaggle的比赛：Drawing With LLMs。简单讲就是根据一段描述生成对应的svg代码。

但是在本地部署时发现，模型生成的回答会有截断，基本上生成不到一行就结束了，小白不明白是什么原因。代码如下：

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
base_prompt = """Generate SVG code to visually represent the following text description, while respecting the given constraints.
<constraints>
* **Allowed Elements:** `svg`, `path`, `circle`, `rect`, `ellipse`, `line`, `polyline`, `polygon`, `g`, `linearGradient`, `radialGradient`, `stop`, `defs`
* **Allowed Attributes:** `viewBox`, `width`, `height`, `fill`, `stroke`, `stroke-width`, `d`, `cx`, `cy`, `r`, `x`, `y`, `rx`, `ry`, `x1`, `y1`, `x2`, `y2`, `points`, `transform`, `opacity`
</constraints>

<example>
<description>"A red circle with a blue square inside"</description>
```svg
<svg viewBox="0 0 256 256" width="256" height="256">
  <circle cx="50" cy="50" r="40" fill="red"/>
  <rect x="30" y="30" width="40" height="40" fill="blue"/>
</svg>
```
</example>


Please ensure that the generated SVG code is well-formed, valid, and strictly adheres to these constraints. Focus on a clear and concise representation of the input description within the given limitations. Always give the complete SVG code with nothing omitted. Never use an ellipsis.

<description>{description}</description>
"""
# 检查CUDA是否可用
if torch.cuda.is_available():
    print("CUDA is available. Using GPU.")
    device = torch.device("cuda")
else:
    print("CUDA is not available. Using CPU.")
    device = torch.device("cpu")
base_model = "Qwen/Qwen2.5-7B-Instruct"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16)
# tokenizer = AutoTokenizer.from_pretrained(finetuned_model_path)
# base_model = AutoModelForCausalLM.from_pretrained(finetuned_model_path, device_map={"":0}, quantization_config=bnb_config)
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, device_map={"":0}, quantization_config=bnb_config)

def generate_svg(description, model, tokenizer):
    prompt = f"{base_prompt.format(description=description)}"
    
    messages = [
    {
   "role": "user", "content": prompt}
  ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt")
      # 将输入数据移动到GPU上（如果可用）
    inputs = {k: v.to(device) for k, v in inputs.items()}
    output = model.generate(**inputs)
    svg_code = tokenizer.decode(output[0], skip_special_tokens=True)
    print(svg_code)
    return svg_code

description = "a purple forest at dusk"
generate_svg(description, model, tokenizer)

生成结果如下：

system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.
user
Generate SVG code to visually represent the following text description, while respecting the given constraints.
<constraints>
* **Allowed Elements:** `svg`, `path`, `circle`, `rect`, `ellipse`, `line`, `polyline`, `polygon`, `g`, `linearGradient`, `radialGradient`, `stop`, `defs`
* **Allowed Attributes:** `viewBox`, `width`, `height`, `fill`, `stroke`, `stroke-width`, `d`, `cx`, `cy`, `r`, `x`, `y`, `rx`, `ry`, `x1`, `y1`, `x2`, `y2`, `points`, `transform`, `opacity`
</constraints>

<example>
<description>"A red circle with a blue square inside"</description>
```svg
<svg viewBox="0 0 256 256" width="256" height="256">
  <circle cx="50" cy="50" r="40" fill="red"/>
  <rect x="30" y="30" width="40" height="40" fill="blue"/>
</svg>
```
</example>


Please ensure that the generated SVG code is well-formed, valid, and strictly adheres to these constraints. Focus on a clear and concise representation of the input description within the given limitations. Always give the complete SVG code with nothing omitted. Never use an ellipsis.

<description>a purple forest at dusk</description>

assistant
```svg
<svg viewBox="0 0 512 512" width

assistant后面就是模型根据输入生成的，可以看出来有明显截断。这也导致我的微调效果不好。求大神解答