Langchain-chatchat+ChatGlm3-6b部署

recover517

已于 2024-03-20 13:46:09 修改

阅读量2.2k

点赞数 10

文章标签： langchain

于 2024-03-17 19:49:56 首次发布

本文链接：https://blog.youkuaiyun.com/qq_31343581/article/details/136788143

版权

我的环境

升级了下配置，加载知识库成功

内存：~~16GB~~ 32B

显卡：~~GTX1060-6G~~ RTX4080 Laptop-12G

1. 基础环境准备

1.1. 安装anaconda，创建环境python版本3.11

conda create -n chatglm3 python=3.11
conda activate chatglm3

1.2. 搭建cuda环境

# 查看cuda版本，版本是CUDA Version: 12.4
$ nvidia-smi
Sat Mar 16 19:22:23 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 551.76                 Driver Version: 551.76         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1060 6GB  WDDM  |   00000000:08:00.0  On |                  N/A |
|  0%   34C    P8              2W /  130W |     688MiB /   6144MiB |     21%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+----------------