DeepSeek 搭建

一、搭建vllm

sudo apt-get update  -y
sudo apt-get install -y gcc-12 g++-12 libnuma-dev
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12

clone vllm 代码

git clone https://github.com/vllm-project/vllm.git vllm_source
cd vllm_source

pip 安装

pip install --upgrade pip
pip install "cmake>=3.26" wheel packaging ninja "setuptools-scm>=8" numpy
pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu

构建安装：

VLLM_TARGET_DEVICE=cpu python setup.py install

二、下载Deepseek蒸馏模型

sudo apt get install git-lfs

安装模型

git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

三、运行deepseek

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B --tensor-parallel-size 2 --max-model-len 32768 --enforce-eager