项目实训——后端篇5：实现SpringBoot后端访问部署在AutoDL上的大模型

AltairParadx

已于 2024-06-24 02:38:23 修改

阅读量585

点赞数 5

于 2024-06-24 02:32:07 首次发布

本文链接：https://blog.youkuaiyun.com/qq_43723962/article/details/139910019

版权

1、AutoDL上的操作

autoDL每个实例都留了端口6006暴露在外以提供服务，这就给了我们通过这个端口访问部署在AutoDL上的MING的可能性

首先简单编写脚本启动在部署在AutoDL上的model_worker、controller和ming_api_server

import subprocess
import threading


def run_controller():
    subprocess.run(["python", "-m", "fastchat.serve.controller", "--host", "0.0.0.0", "--port", "21001"])


def run_model_worker():
    subprocess.run(["python", "-m", "fastchat.serve.model_worker", "--host", "0.0.0.0", "--controller-address",
                    "http://0.0.0.0:21001", "--port", "21002","--model-path", "/root/autodl-tmp/MING-MOE-4B","--model-name", "ming-moe-4B"])


def run_api_server():
    subprocess.run(["python", "-m", "ming_api_serve", "--host", "0.0.0.0", "--controller-address",