VLLM: Failed to infer device type 简单解决方案

最新推荐文章于 2025-10-23 16:52:22 发布

原创

最新推荐文章于 2025-10-23 16:52:22 发布 · 2k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#python #开发语言 #vllm

该文章已生成可运行项目，

    def __init__(self, device: str = "auto") -> None:
        if device == "auto":
            # Automated device type detection
            from vllm.platforms import current_platform
            self.device_type = current_platform.device_type
            if not self.device_type:
                raise RuntimeError(
                    "Failed to infer device type, please set "
                    "the environment variable `VLLM_LOGGING_LEVEL=DEBUG` "
                    "to turn on verbose logging to help debug the issue.")
        else:
            # Device type is assigned explicitly
            self.device_type = device

        # Some device types require processing inputs on CPU
        if self.device_type in ["neuron"]:
            self.device = torch.device("cpu")
        elif self.device_type in ["tpu"]:
            self.device = None
        else:
            # Set device with device type
            self.device = torch.device(self.device

本文章已经生成可运行项目