基于Baichuan2的新冠流感中医自我诊断治疗(大模型微调+Gradio)

军哥说AI

于 2024-06-19 13:42:13 发布

阅读量1.2k

点赞数 29

CC 4.0 BY-SA版权

分类专栏： AI写算法文章标签：人工智能算法

本文链接：https://blog.youkuaiyun.com/class4715/article/details/139800770

一、项目说明

项目使用paddleNLP提供的大模型套件对Baichuan2-7b/13b进行微调，使用《中医治疗新冠流感支原体感染等有效病历集》进行Lora训练，使大模型具备使用中医方案诊断和治疗新冠、流感等上呼吸道感染的能力。

二、PaddleNLP

PaddleNLP提供的飞桨大模型套件秉承了一站式体验、性能极致、生态兼容的设计理念，旨在提供业界主流大模型预训练、精调（含SFT、PEFT）、量化、推理等统一流程，帮助开发者低成本、低门槛、快速实现大语言模型定制化。PaddleNLP支持多个主流大模型的SFT、LoRA、Prefix Tuning等精调策略，提供统一、高效精调方案：

 1. 统一训练入口。飞桨大模型套件精调方案可适配业界主流大模型，用户只需修改配置文件，即能在单卡或多卡（支持4D并行分布式策略）进行多种大模型精调。

 1. 高效数据和分布式策略。Zero Padding零填充优化策略有效减少了pad token的占比，提高模型训练效率高达100%。独创PEFT结合低比特和分布式并行策略，大幅降低大模型精调硬件门槛，支持单卡   （A100 80G）百亿模型微调、单机（A100 80G * 8）千亿模型微调。

 1. 支持多轮对话。支持统一对话模板，支持多轮对话高效训练，详参多轮对话文档。

三、Baichuan2-7b/13b-chat

Baichuan2系列产品是百川智能在深度学习领域的最新成果，经过微调后的模型在多个任务上取得了优异的性能。开源这些模型将为开发者提供一个强大的工具，帮助他们在各种应用场景中实现更高效、更准确的人工智能应用.

Baichuan 2系列产品完全开源，并且在在「免费商用」这条路上，Baichuan 2 践行得非常彻底，极大弥补了中国开源生态的短板，让中国开发者用上了对中文场景更友好的开源大模型。

Baichuan2系列模型效率也很高，130亿参数的Baichuan2-13b量化版，在消费级显卡的笔记本电脑上也可以实现快速推理。因此，我们选用Baichuan2系统模型做为本项目的基座

四、训练数据说明

《中医治疗新冠流感支原体感染等有效病历集》是云中医整理的近期高发上呼吸道感染中医诊断治疗的有效病历，包含新冠，甲流，支原体，腺病毒，合胞病毒等各种病毒引发的感冒、咳嗽等病历。经处理弱化了原病历的处方及处方药，增加了OTC中成药及家庭食疗的治疗方案，避免医疗的资质问题及可能的纠纷，更适合于一般轻症的自我诊所治疗。数据分两部分:case为病历记录，diagnosis为从病历提取的诊断结果及处方。数据示例如下：

    {"case":"患者，男性，45岁，因新冠感染前来就诊。患者近日出现恶寒、无汗、后背痛的症状，并有发热、身痛、头痛。
背部疼痛严重，影响日常生活。患者还表现出清涕、鼻塞、神疲乏力、声哑、无食欲等症状。舌淡苔白，脉紧。根据患者的主症
和症状关联，考虑为葛根汤证。葛根汤为中医经典方剂，主要用于治疗风寒感冒，尤其对于恶寒、无汗、后背痛等症状有显著疗
效。综上所述，患者新冠感染后出现恶寒、无汗、后背痛、发热、身痛、头痛等症状，考虑为葛根汤证。建议采用葛根汤进行治
疗。",
    "diagnosis":"诊断：太阳阳明伤寒 。建议处方：葛根汤。建议中成药：葛根汤颗粒或风寒感冒颗粒或感冒软胶
    囊 建议食疗：葱白姜汤"}

PaddleNLP训练数据支持的数据格式是每行包含一个字典，每个字典包含以下字段：

src : str, List(str), 模型的输入指令（instruction）、提示（prompt），模型应该执行的任务。

tgt : str, List(str), 模型的输出。

因此，在训练前，需要将训练数据转换为要求的格式数据。

五、环境准备

1. 获取并安装最新版PaddleNLP

In [1]


#直接克隆github上的最新版本，考虑网络问题，也可以从gitee上克隆(gitee可能版本不是最新，最好是从github上取)
#!git clone https://gitee.com/PaddlePaddle/PaddleNLP
!git clone https://github.com/PaddlePaddle/PaddleNLP.git

Cloning into 'PaddleNLP'...
remote: Enumerating objects: 60471, done.
remote: Counting objects: 100% (578/578), done.
remote: Compressing objects: 100% (423/423), done.
remote: Total 60471 (delta 271), reused 382 (delta 144), pack-reused 59893
Receiving objects: 100% (60471/60471), 97.72 MiB | 15.36 MiB/s, done.
Resolving deltas: 100% (41419/41419), done.

In [2]

# 安装本地下载的版本.
!pip install -r PaddleNLP/requirements.txt
!pip install -e ./PaddleNLP

Looking in indexes: https://mirror.baidu.com/pypi/simple/, https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.tsinghua.edu.cn/simple/
Ignoring protobuf: markers 'platform_system == "Windows"' don't match your environment
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 1)) (0.42.1)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 2)) (6.8.0)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 3)) (0.4.6)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 4)) (1.2.2)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 5)) (0.3.4)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 6)) (0.70.12.2)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 7)) (2.16.0)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 8)) (4.66.1)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 9)) (1.1.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 10)) (0.1.99)
Requirement already satisfied: huggingface_hub>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 11)) (0.20.1)
Requirement already satisfied: onnx>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 12)) (1.15.0)
Requirement already satisfied: protobuf>=3.20.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 13)) (3.20.3)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 15)) (1.1.0)
Requirement already satisfied: Flask-Babel in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 16)) (4.0.0)
Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 17)) (2.5.3)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 18)) (0.105.0)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 19)) (0.25.0)
Requirement already satisfied: typer in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 20)) (0.9.0)
Requirement already satisfied: rich in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 21)) (13.7.0)
Requirement already satisfied: safetensors in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 22)) (0.4.1)
Requirement already satisfied: tool_helpers in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 23)) (0.1.1)
Requirement already satisfied: aistudio-sdk>=0.1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 24)) (0.1.5)
Requirement already satisfied: jinja2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from -r PaddleNLP/requirements.txt (line 25)) (3.1.2)
Requirement already satisfied: numpy>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from seqeval->-r PaddleNLP/requirements.txt (line 4)) (1.26.2)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from seqeval->-r PaddleNLP/requirements.txt (line 4)) (1.3.2)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (3.13.1)
Requirement already satisfied: pyarrow>=8.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (14.0.2)
Requirement already satisfied: pyarrow-hotfix in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (0.6)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2.1.4)
Requirement already satisfied: requests>=2.19.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2.31.0)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (3.4.1)
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2023.10.0)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (3.9.1)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (6.0.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from huggingface_hub>=0.11.1->-r PaddleNLP/requirements.txt (line 11)) (4.9.0)
Requirement already satisfied: Babel>=2.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (2.14.0)
Requirement already satisfied: Flask>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (3.0.0)
Requirement already satisfied: pytz>=2022.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (2023.3.post1)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (0.8.98)
Requirement already satisfied: Pillow>=7.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (10.1.0)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (1.16.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (3.8.2)
Requirement already satisfied: rarfile in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (4.1)
Requirement already satisfied: psutil in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->-r PaddleNLP/requirements.txt (line 17)) (5.9.7)
Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->-r PaddleNLP/requirements.txt (line 18)) (3.7.1)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->-r PaddleNLP/requirements.txt (line 18)) (2.5.3)
Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->-r PaddleNLP/requirements.txt (line 18)) (0.27.0)
Requirement already satisfied: click>=7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from uvicorn->-r PaddleNLP/requirements.txt (line 19)) (8.1.7)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from uvicorn->-r PaddleNLP/requirements.txt (line 19)) (0.14.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rich->-r PaddleNLP/requirements.txt (line 21)) (2.2.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rich->-r PaddleNLP/requirements.txt (line 21)) (2.17.2)
Requirement already satisfied: pybind11 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from tool_helpers->-r PaddleNLP/requirements.txt (line 23)) (2.11.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from jinja2->-r PaddleNLP/requirements.txt (line 25)) (2.1.3)
Requirement already satisfied: idna>=2.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->-r PaddleNLP/requirements.txt (line 18)) (3.6)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->-r PaddleNLP/requirements.txt (line 18)) (1.3.0)
Requirement already satisfied: exceptiongroup in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->-r PaddleNLP/requirements.txt (line 18)) (1.2.0)
Requirement already satisfied: Werkzeug>=3.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (3.0.1)
Requirement already satisfied: itsdangerous>=2.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (2.1.2)
Requirement already satisfied: blinker>=1.6.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->-r PaddleNLP/requirements.txt (line 16)) (1.7.0)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (4.0.3)
Requirement already satisfied: mdurl~=0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->-r PaddleNLP/requirements.txt (line 21)) (0.1.1)
Requirement already satisfied: annotated-types>=0.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi->-r PaddleNLP/requirements.txt (line 18)) (0.6.0)
Requirement already satisfied: pydantic-core==2.14.6 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi->-r PaddleNLP/requirements.txt (line 18)) (2.14.6)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2023.11.17)
Requirement already satisfied: scipy>=1.5.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from scikit-learn>=0.21.3->seqeval->-r PaddleNLP/requirements.txt (line 4)) (1.11.4)
Requirement already satisfied: joblib>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from scikit-learn>=0.21.3->seqeval->-r PaddleNLP/requirements.txt (line 4)) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from scikit-learn>=0.21.3->seqeval->-r PaddleNLP/requirements.txt (line 4)) (3.2.0)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from bce-python-sdk->visualdl->-r PaddleNLP/requirements.txt (line 17)) (3.19.0)
Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from bce-python-sdk->visualdl->-r PaddleNLP/requirements.txt (line 17)) (0.18.3)
Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (4.47.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from matplotlib->visualdl->-r PaddleNLP/requirements.txt (line 17)) (2.8.2)
Requirement already satisfied: tzdata>=2022.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from pandas->datasets>=2.0.0->-r PaddleNLP/requirements.txt (line 7)) (2023.3)
Looking in indexes: https://mirror.baidu.com/pypi/simple/, https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.tsinghua.edu.cn/simple/
Obtaining file:///home/aistudio/PaddleNLP
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.42.1)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (6.8.0)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.4.6)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (1.2.2)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.3.4)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.70.12.2)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (2.16.0)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (4.66.1)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (1.1.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.1.99)
Requirement already satisfied: huggingface-hub>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.20.1)
Requirement already satisfied: onnx>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (1.15.0)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (1.1.0)
Requirement already satisfied: Flask-Babel in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (4.0.0)
Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (2.5.3)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.105.0)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.25.0)
Requirement already satisfied: typer in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.9.0)
Requirement already satisfied: rich in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (13.7.0)
Requirement already satisfied: safetensors in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.4.1)
Requirement already satisfied: tool-helpers in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.1.1)
Requirement already satisfied: aistudio-sdk>=0.1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (0.1.5)
Requirement already satisfied: jinja2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (3.1.2)
Requirement already satisfied: protobuf>=3.20.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from paddlenlp==2.6.1.post0) (3.20.3)
Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aistudio-sdk>=0.1.3->paddlenlp==2.6.1.post0) (2.31.0)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (3.13.1)
Requirement already satisfied: numpy>=1.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (1.26.2)
Requirement already satisfied: pyarrow>=8.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (14.0.2)
Requirement already satisfied: pyarrow-hotfix in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (0.6)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (2.1.4)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (3.4.1)
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets>=2.0.0->paddlenlp==2.6.1.post0) (2023.10.0)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (3.9.1)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from datasets>=2.0.0->paddlenlp==2.6.1.post0) (6.0.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from huggingface-hub>=0.11.1->paddlenlp==2.6.1.post0) (4.9.0)
Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->paddlenlp==2.6.1.post0) (3.7.1)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->paddlenlp==2.6.1.post0) (2.5.3)
Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from fastapi->paddlenlp==2.6.1.post0) (0.27.0)
Requirement already satisfied: Babel>=2.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->paddlenlp==2.6.1.post0) (2.14.0)
Requirement already satisfied: Flask>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->paddlenlp==2.6.1.post0) (3.0.0)
Requirement already satisfied: pytz>=2022.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask-Babel->paddlenlp==2.6.1.post0) (2023.3.post1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from jinja2->paddlenlp==2.6.1.post0) (2.1.3)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rich->paddlenlp==2.6.1.post0) (2.2.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rich->paddlenlp==2.6.1.post0) (2.17.2)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from seqeval->paddlenlp==2.6.1.post0) (1.3.2)
Requirement already satisfied: pybind11 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from tool-helpers->paddlenlp==2.6.1.post0) (2.11.1)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from typer->paddlenlp==2.6.1.post0) (8.1.7)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from uvicorn->paddlenlp==2.6.1.post0) (0.14.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (0.8.98)
Requirement already satisfied: Pillow>=7.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (10.1.0)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (1.16.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (3.8.2)
Requirement already satisfied: rarfile in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (4.1)
Requirement already satisfied: psutil in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from visualdl->paddlenlp==2.6.1.post0) (5.9.7)
Requirement already satisfied: idna>=2.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->paddlenlp==2.6.1.post0) (3.6)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->paddlenlp==2.6.1.post0) (1.3.0)
Requirement already satisfied: exceptiongroup in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->paddlenlp==2.6.1.post0) (1.2.0)
Requirement already satisfied: Werkzeug>=3.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->paddlenlp==2.6.1.post0) (3.0.1)
Requirement already satisfied: itsdangerous>=2.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->paddlenlp==2.6.1.post0) (2.1.2)
Requirement already satisfied: blinker>=1.6.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from Flask>=2.0->Flask-Babel->paddlenlp==2.6.1.post0) (1.7.0)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp==2.6.1.post0) (4.0.3)
Requirement already satisfied: mdurl~=0.1 in /opt/conda/envs/python35-paddle120-e