使用 Llama Factory 对 Qwen3 SFT 微调和部署

Qwen3 模型基础

Qwen3 作为推理模型，如果开启了推理模式，输出形式为 CotOutput

源码

https://github.com/hiyouga/LLaMA-Factory/blob/main/README_zh.md

官方的文档已经很详细了，可以参考

git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git

环境配置

除了torch、metrics之外，可选项还包括，可以自行添加

torch、torch-npu、metrics、deepspeed、liger-kernel、bitsandbytes、hqq、eetq、gptq、aqlm、vllm、sglang、galore、apollo、badam、adam-mini、qwen、minicpm_v、modelscope、openmind、swanlab、dev

cd LLaMA-Factory
pip install -e ".[torch,metrics]" --no-build-isolation

数据集导入

数据形式优先考虑 Alpaca 格式，如下：

[
  {
    "instruction": "用户指令（必填）",
    "input": "用户输入（选填）",
    "output": "模型回答（必填）",
    "system": "系统提示词（选填）",
    "history": [
      ["第一轮指令（选填）", "第一轮回答（选填）"],
      ["第二轮指令（选填）", "第二轮回答（选填）"]
    ]
  }
]

以一条 Text2Cypher 数据为示例，output 当中有 标签，train-00000-of-00001-5000-cot-en-alpaca.json

{
        "instruction": "You are a Cypher expert. Given the database schema, help convert the user's question to Cypher.\n\n",
        "input": "\nDatabase Schema:\n\nNode properties:\nPatient {name: STRING, dob: DATETIME, gender: STRING, patient_id: STRING}\nDoctor {name: STRING, dob: DATETIME, gender: STRING, specialization: STRING, doctor_id: STRING}\nNurse {name: STRING, dob: DATETIME, gender: STRING, specialization: STRING, nurse_id: STRING}\nPharmacist {name: STRING, dob: DATETIME, gender: STRING, pharmacist_id: STRING}\nHospital {name: STRING, location: POINT, hospital_id: STRING}\nPharmacy {name: STRING, location: POINT, pharmacy_id: STRING}\nDisease {name: STRING, icd_code: STRING}\nSymptom {name: STRING, description: STRING}\nMedication {name: STRING, medication_id: STRING, manufacturer: STRING}\nClinical_Trial {name: STRING, trial_id: STRING, start_date: DATETIME, end_date: DATETIME}\nMedical_Device {name: STRING, device_id: STRING, manufacturer: STRING}\nInsurance_Provider {name: STRING, provider_id: STRING, coverage_type: STRING}\nPayer {name: STRING, payer_id: STRING}\nResearcher {name: STRING, researcher_id: STRING, affiliation: STRING}\nMedical_Journal {name: STRING, journal_id: STRING, publisher: STRING}\nPatient_Record {patient_id: STRING, date: DATETIME, record: STRING}\nDoctor_Record {doctor_id: STRING, date: DATETIME, record: STRING}\nHospital_Record {hospital_id: STRING, date: DATETIME, record: STRING}\nPharmacy_Record {pharmacy_id: STRING, date: DATETIME, record: STRING}\nResearch_Record {researcher_id: STRING, date: DATETIME, record: STRING}\nMedical_Journal_Record {journal_id: STRING, date: DATETIME, record: STRING}\nInsurance_Provider_Record {provider_id: STRING, date: DATETIME, record: STRING}\nPayer_Record {payer_id: STRING, date: DATETIME, record: STRING}\nResearcher_Record {researcher_id: STRING, date: DATETIME, record: STRING}\nMedication_Record {medication_id: STRING,