TensorRT-LLM的AutoDL部署

最新推荐文章于 2025-08-04 21:05:10 发布

wwwsctvcom

最新推荐文章于 2025-08-04 21:05:10 发布

阅读量767

点赞数 6

CC 4.0 BY-SA版权

文章标签：人工智能

本文链接：https://blog.youkuaiyun.com/wwwsctvcom/article/details/134747464

安装cuda

wget https://developer.download.nvidia.com/compute/cuda/12.2.1/local_installers/cuda_12.2.1_535.86.10_linux.run

chmod +x chmod +x cuda_12.2.1_535.86.10_linux.run

./cuda_12.2.1_535.86.10_linux.run

安装cudnn

必须安装好cuda之后，才能成功安装

dpkg -i cudnn-local-repo-ubuntu2204-8.9.6.50_1.0-1_amd64.deb
cd /var/cudnn-local-repo-ubuntu2204-8.9.6.50/ && dpkg -i *.deb

ldconfig -v | grep cudnn

安装tensorrt

wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/9.1.0/tars/tensorrt-9.1.0.4.linux.x86_64-gnu.cuda-12.2.tar.gz

tar -xvzf tensorrt-9.1.0.4.linux.x86_64-gnu.cuda-12.2.tar.gz

mv TensorRT-9.1.0.4/ /usr/local/tensorrt
rm tensorrt-9.1.0.4.linux.x86_64-gnu.cuda-12.2.tar.gz

# 配置
echo "/usr/local/tensorrt/lib" > /etc/ld.so.conf.d/tensorrt.conf

# 查看
ldconfig -v | grep nvinfer

# 设置环境
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/tensorrt/lib