【qwen】DeepSpeed Zero-3 is not compatible with `low_cpu_mem_usage=True` or with passing a `device_map

报错

DeepSpeed Zero-3 is not compatible with low_cpu_mem_usage=True or with passing a `device_map

原因

DeepSpeed Zero-3的核心功能就是在显存不足的情况下，使用CPU内存

解决

如下：注释device_map=device_map行

    model = transformers.AutoModelForCausalLM.from_pretrained(
        model_args.model_name_or_path,
        config=config,
        cache_dir=training_args.cache_dir,
        # device_map=device_map,# 注释
        trust_remote_code=True,
        quantization_config=GPTQConfig(
            bits=4, disable_exllama=True
        )
        if training_args.use_lora and lora_args.q_lora
        else None,
    )