基于milvus数据库的RAG-Demo

苍墨穹天

已于 2024-09-19 15:28:47 修改

阅读量697

点赞数 11

分类专栏：大模型文章标签： milvus langchain

于 2024-09-19 15:28:27 首次发布

本文链接：https://blog.youkuaiyun.com/Mooczx/article/details/142361689

版权

1.上传文本并将文本向量化

import os
from django.conf import settings
from langchain.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter, CharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.embeddings import DashScopeEmbeddings
from langchain_community.document_loaders import PyPDFLoader, Docx2txtLoader

from pymilvus import MilvusClient, DataType


def get_embedding(text):
    file_path = os.path.join(settings.BASE_DIR, 'media', text)
    loader = ''
    # 判断文件类型
    if file_path.endswith('.txt'):
        loader = TextLoader(file_path, encoding='utf-8')

    elif file_path.endswith('.pdf'):
        loader = PyPDFLoader(file_path)

    elif file_path.endswith('.docx'):
        loader = Do