从ConversationalRetrievalChain到LCEL：掌握更清晰的检索增强生成-优快云博客

本文链接：https://blog.youkuaiyun.com/afgGRWESDAW/article/details/144439076

从 ConversationalRetrievalChain 到 LCEL：掌握更清晰的检索增强生成

在当今快速发展的人工智能和自然语言处理领域，检索增强生成（Retrieval-Augmented Generation, RAG）已成为一项核心技术，能够显著提高模型的回答质量。过去，我们依赖 ConversationalRetrievalChain 来结合检索增强生成和聊天历史，实现文档互动。如今，LCEL（LangChain Execution Layer）提供了更精细的控制和额外特性，如异步操作与流式方法支持。本文将深入探讨迁移到 LCEL 的优势，并通过代码示例展示如何实现这一切。

为什么选择 LCEL？

迁移到 LCEL 有几个明显的优势：

更清晰的内部结构：LCEL 具有更透明的内部架构，无需隐藏如问题重构等复杂步骤。
更灵活的返回格式：能够更加灵活地返回源文档，提升实用性。
支持更多操作：如流式处理和异步操作，这在大规模应用中尤为重要。

代码示例

在以下示例中，我们将展示如何使用 LCEL 实现检索增强生成。示例假设我们已经加载了文档并构建了一个向量存储。

%pip install --upgrade --quiet langchain-community langchain langchain-openai faiss-cpu

import os
from getpass import getpass

os.environ["OPENAI_API_KEY"] = getpass()  # 请勿在代码中明文存储API密钥

# 加载文档
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain_openai.chat_models import ChatOpenAI
from langchain_openai.embeddings import OpenAIEmbeddings

loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
data = loader.load()

# 文本拆分
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

# 向量存储
vectorstore = FAISS.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

# 使用 OpenAI 的聊天模型
llm = ChatOpenAI()

# 创建用于重构问题的系统提示
from langchain.chains import create_history_aware_retriever, create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

condense_question_system_template = (
    "Given a chat history and the latest user question "
    "which might reference context in the chat history, "
    "formulate a standalone question which can be understood "
    "without the chat history. Do NOT answer the question, "
    "just reformulate it if needed and otherwise return it as is."
)

condense_question_prompt = ChatPromptTemplate.from_messages(
    [("system", condense_question_system_template), ("placeholder", "{chat_history}"), ("human", "{input}")]
)

history_aware_retriever = create_history_aware_retriever(
    llm, vectorstore.as_retriever(), condense_question_prompt
)

system_prompt = (
    "You are an assistant for question-answering tasks. "
    "Use the following pieces of retrieved context to answer "
    "the question. If you don't know the answer, say that you "
    "don't know. Use three sentences maximum and keep the "
    "answer concise."
    "\n\n"
    "{context}"
)

qa_prompt = ChatPromptTemplate.from_messages(
    [("system", system_prompt), ("placeholder", "{chat_history}"), ("human", "{input}")]
)

qa_chain = create_stuff_documents_chain(llm, qa_prompt)

convo_qa_chain = create_retrieval_chain(history_aware_retriever, qa_chain)

response = convo_qa_chain.invoke(
    {
        "input": "What are autonomous agents?",
        "chat_history": [],
    }
)

print(response)