在Ubuntu系统下使用Docker部署ollama并使用英伟达gpu加速

longze_7

已于 2025-07-10 09:59:08 修改

阅读量152

点赞数 1

CC 4.0 BY-SA版权

文章标签： docker 容器运维

于 2025-07-09 17:26:56 首次发布

本文链接：https://blog.youkuaiyun.com/kentturing/article/details/149228745

下载安装英伟达Nvidia container toolkit：

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
  sudo apt-get install -y \
      nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

安装完成后检查是否安装成功

nvidia-container-runtime --version

安装完Nvidia container toolkit 后重启docker ：

sudo systemctl restart docker

运行docker容器：

sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama