Weaviate实战:从零构建高精度RAG问答系统的完整指南
如果你正在寻找一种能够快速将企业内部文档、知识库转化为智能问答系统的方法,那么基于向量数据库的RAG(检索增强生成)架构可能是你需要的解决方案。在众多向量数据库中,Weaviate以其开箱即用的特性、强大的语义搜索能力和简洁的API设计,成为了许多开发者的首选。
我最近在一个客户项目中使用了Weaviate来构建企业知识库问答系统,从最初的原型验证到最终的生产部署,整个过程让我对这款向量数据库有了深刻的理解。今天,我将分享如何用Weaviate在短时间内搭建一个功能完整的RAG应用,并深入探讨一些实际开发中会遇到的关键问题。
1. 为什么选择Weaviate构建RAG应用?
在开始技术细节之前,让我们先理解为什么Weaviate在RAG场景中表现突出。传统的全文搜索在面对语义相似但词汇不同的查询时往往力不从心,而纯粹的生成式AI又容易产生“幻觉”——即编造看似合理但实际错误的信息。RAG架构通过结合两者的优势,既保证了回答的准确性,又具备了理解语义的能力。
Weaviate在这个架构中扮演着核心角色:它负责将非结构化文本转换为向量表示,并基于语义相似度快速检索相关信息。与其他向量数据库相比,Weaviate有几个显著优势:
- 内置向量化模块:无需额外部署嵌入模型服务,Weaviate可以直接集成OpenAI、Cohere、HuggingFace等主流嵌入模型
- 混合搜索能力:同时支持向量相似度搜索和传统关键词搜索,可以灵活调整两者的权重
- 生产就绪的特性:包括数据持久化、高可用集群、权限控制等企业级功能
- 活跃的社区和丰富的SDK:Python、JavaScript、Go、Java等多种语言支持
下面是一个简单的对比表格,展示了Weaviate与其他常见向量存储方案的差异:
| 特性 | Weaviate | FAISS | Pinecone | pgvector |
|---|---|---|---|---|
| 部署方式 | 独立服务/云服务 | 库/嵌入式 | 云服务 | PostgreSQL扩展 |
| 向量化集成 | 内置 | 需要外部服务 | 内置 | 需要外部服务 |
| 混合搜索 | 支持 | 不支持 | 部分支持 | 不支持 |
| 数据持久化 | 自动 | 手动处理 | 自动 | 依赖PostgreSQL |
| 生产就绪 | 是 | 有限 | 是 | 依赖PostgreSQL配置 |
2. 环境准备与Weaviate部署
2.1 选择适合的部署方式
Weaviate提供了多种部署选项,从本地开发到生产环境都有相应的方案。对于快速原型开发,我推荐使用Docker Compose方式,它能在几分钟内启动一个功能完整的Weaviate实例。
首先,创建一个docker-compose.yml文件:
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: semitechnologies/weaviate:1.33.1
ports:
- 8080:8080
- 50051:50051
volumes:
- ./weaviate_data:/var/lib/weaviate
restart: unless-stopped
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
CLUSTER_HOSTNAME: 'node1'
DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
ENABLE_MODULES: 'text2vec-openai,generative-openai'
OPENAI_APIKEY: ${OPENAI_API_KEY}
注意:这里我使用了OpenAI的文本向量化模块,你需要设置
OPENAI_API_KEY环境变量。如果你希望使用本地模型,可以替换为text2vec-transformers模块并配置相应的模型服务。
启动服务非常简单:
export OPENAI_API_KEY="your-openai-api-key"
docker-compose up -d
等待几秒钟后,访问http://localhost:8080/v1/meta,如果看到JSON格式的元数据响应,说明Weaviate已经成功启动。
2.2 安装Python客户端
Weaviate提供了完善的Python SDK,让开发者能够以编程方式操作数据库:
pip install weaviate-client openai
对于生产环境,我建议固定版本以避免兼容性问题:
pip install weaviate-client==4.4.0 openai==1.12.0
2.3 连接测试
让我们写一个简单的连接测试脚本:
import weaviate
import os
def test_connection():
"""测试Weaviate连接"""
client = weaviate.connect_to_local(
host="localhost",
port=8080,
grpc_port=50051
)
try:
# 检查服务是否就绪
is_ready = client.is_ready()
print(f"连接状态: {'成功' if is_ready else '失败'}")
# 获取元数据
meta = client.get_meta()
print(f"版本: {meta.get('version', '未知')}")
print(f"模块: {', '.join(meta.get('modules', []))}")
return True
except Exception as e:
print(f"连接失败: {e}")
return False
finally:
client.close()
if __name__ == "__main__":
test_connection()
运行这个脚本,你应该能看到类似下面的输出:
连接状态: 成功
版本: 1.33.1
模块: text2vec-openai, generative-openai
3. 数据准备与向量化策略
3.1 理解文档分块的重要性
在RAG系统中,文档分块(chunking)是影响检索质量的关键因素。分块太大可能导致检索到不相关的信息,分块太小则可能丢失上下文。根据我的经验,以下分块策略在实践中效果较好:
- 按段落分块:适合结构清晰的文档,每个段落200-500字
- 滑动窗口分块:适合连续文本,设置50-100字的重叠区域
- 语义分块:基于句子嵌入的相似度进行分块,更智能但计算成本高
下面是一个基于Markdown文档结构的智能分块实现:
import re
from typing import List, Dict, Any
from dataclasses import dataclass
@dataclass
class DocumentChunk:
"""文档分块数据结构"""
content: str
metadata: Dict[str, Any]
chunk_id: str
start_index: int
end_index: int
class SmartChunker:
"""智能文档分块器"""
def __init__(self, chunk_size: int = 500, overlap: int = 50):
self.chunk_size = chunk_size
self.overlap = overlap
def chunk_markdown(self, content: str, metadata: Dict[str, Any] = None) -> List[DocumentChunk]:
"""分块Markdown文档"""
if metadata is None:
metadata = {}
chunks = []
# 按标题分割文档
sections = self._split_by_headers(content)
for section_idx, (header, section_content) in enumerate(sections):
# 如果章节内容较短,直接作为一个块
if len(section_content) <= self.chunk_size:
chunk_metadata = metadata.copy()
chunk_metadata.update({
"section_header": header,
"section_index": section_idx
})
chunks.append(DocumentChunk(
content=section_content,
metadata=chunk_metadata,
chunk_id=f"{metadata.get('doc_id', '')}_sec{section_idx}",
start_index=0,
end_index=len(section_content)
))
else:
# 使用滑动窗口分块
section_chunks = self._sliding_window_chunk(
section_content,
metadata,
header,
section_idx
)
chunks.extend(section_chunks)
return chunks
def _split_by_headers(self, content: str) -> List[tuple]:
"""按Markdown标题分割内容"""
# 匹配各级标题
header_pattern = r'(^#{1,6}\s+.+?$)(?=\n#{1,6}|\Z)'
sections = []
# 添加文档开始作为第一个章节
current_header = "文档开始"
current_content = []
lines = content.split('\n')
for line in lines:
if line.startswith('#'):
# 保存当前章节
if current_content:
sections.append((current_header, '\n'.join(current_content)))
current_content = []
# 开始新章节
current_header = line.strip('#').strip()
else:
current_content.append(line)
# 添加最后一个章节
if current_content:
sections.append((current_header, '\n'.join(current_content)))
return sections
def _sliding_window_chunk(self, content: str, base_metadata: Dict,
header: str, section_idx: int) -> List[DocumentChunk]:
"""滑动窗口分块"""
chunks = []
words = content.split()
start = 0
while start < len(words):
end = min(start + self.chunk_size, len(words))
# 确保不在单词中间截断
chunk_text = ' '.join(words[start:end])
chunk_metadata = base_metadata.copy()
chunk_metadata.update({
"section_header": header,
"section_index": section_idx,
"chunk_in_section": len(chunks) + 1
})
chunks.append(DocumentChunk(
content=chunk_text,
metadata=chunk_metadata,
chunk_id=f"{base_metadata.get('doc_id', '')}_sec{section_idx}_ch{len(chunks)+1}",
start_index=start,
end_index=end
))
# 移动窗口,考虑重叠
start += self.chunk_size - self.overlap
return chunks
3.2 准备示例数据集
为了演示,我准备了一个关于机器学习基础概念的小型数据集。在实际项目中,这可能是你的产品文档、技术手册或客户支持问答:
sample_documents = [
{
"id": "ml_basics_001",
"title": "机器学习基础概念",
"content": """
# 机器学习概述
机器学习是人工智能的一个分支,它使计算机系统能够从数据中学习并改进,而无需明确编程。
## 主要类型
### 监督学习
监督学习使用带有标签的训练数据来训练模型。每个训练样本都包含输入特征和对应的输出标签。常见的监督学习算法包括线性回归、逻辑回归、支持向量机和神经网络。
### 无监督学习
无监督学习处理没有标签的数据。算法尝试发现数据中的内在结构和模式。聚类和降维是无监督学习的典型任务。
### 强化学习
强化学习通过与环境交互来学习最优策略。智能体根据行动获得的奖励或惩罚来调整行为。
## 关键术语
**特征工程**:将原始数据转换为更能代表预测模型的特征的过程。
**过拟合**:模型在训练数据上表现很好,但在新数据上表现不佳的现象。
**交叉验证**:评估模型泛化能力的技术,将数据分成多个子集进行训练和测试。
""",
"category": "机器学习",
"tags": ["基础", "概念", "入门"],
"author": "AI教育团队",
"created_at": "2024-01-15"
},
{
"id": "neural_networks_001",
"title": "神经网络基本原理",
"content": """
# 神经网络基础
神经网络是受生物神经系统启发的计算模型,由相互连接的节点(神经元)组成。
## 网络结构
### 前馈神经网络
信息单向流动,从输入层经过隐藏层到输出层。这是最简单的神经网络结构。
### 卷积神经网络(CNN)
专门用于处理网格状数据(如图像)的神经网络。通过卷积层提取局部特征。
### 循环神经网络(RNN)
适合处理序列数据,具有记忆先前信息的能力。
## 训练过程
1. **前向传播**:输入数据通过网络计算输出
2. **损失计算**:比较预测输出和真实标签
3. **反向传播**:计算梯度并更新权重
4. **优化**:使用优化算法(如SGD、Adam)调整参数
## 激活函数
**ReLU**:最常用的激活函数,计算简单且缓解梯度消失问题。
**Sigmoid**:将输入映射到(0,1)区间,适合二分类输出层。
**Tanh**:将输入映射到(-1,1)区间,输出以零为中心。
""",
"category": "深度学习",
"tags": ["神经网络", "基础", "CNN", "RNN"],
"author": "深度学习研究组",
"created_at": "2024-02-20"
}
]
4. 构建Weaviate集合与数据导入
4.1 设计集合模式
在Weaviate中,集合(Collection)类似于传统数据库中的表。我们需要仔细设计集合的模式,以支持高效的检索:
import weaviate
from weaviate.classes.config import Configure, Property, DataType, Tokenization
from weaviate.classes.init import Auth
def create_knowledge_base_collection(client: weaviate.WeaviateClient,
collection_name: str = "KnowledgeBase"):
"""创建知识库集合"""
# 检查集合是否已存在
existing_collections = client.collections.list_all()
if collection_name in existing_collections:
print(f"集合 '{collection_name}' 已存在,跳过创建")
return client.collections.get(collection_name)
# 定义集合属性
properties = [
Property(
name="content",
data_type=DataType.TEXT,
description="文档内容",
tokenization=Tokenization.WORD # 用于关键词搜索
),
Property(
name="title",
data_type=DataType.TEXT,
description="文档标题",
tokenization=Tokenization.WORD
),
Property(
name="category",
data_type=DataType.TEXT,
description="文档分类",
tokenization=Tokenization.WORD
),
Property(
name="tags",
data_type=DataType.TEXT_ARRAY,
description="标签列表"
),
Property(
name="author",
data_type=DataType.TEXT,
description="作者"
),
Property(
name="created_at",
data_type=DataType.DATE,
description="创建时间"
),
Property(
name="doc_id",
data_type=DataType.TEXT,
description="文档唯一标识"
),
Property(
name="chunk_id",
data_type=DataType.TEXT,
description="分块唯一标识"
),
Property(
name="section_header",
data_type=DataType.TEXT,
description="所属章节标题"
)
]
# 创建集合配置
collection_config = Configure.NamedVector.text2vec_openai(
name="content_vector",
source_properties=["content", "title"], # 基于内容和标题生成向量
vectorize_collection_name=False
)
# 创建集合
collection = client.collections.create(
name=collection_name,
properties=properties,
vectorizer_config=collection_config,
# 配置向量索引
vector_index_config=Configure.VectorIndex.hnsw(
distance_metric="cosine",
ef_construction=128,
max_connections=16,
ef=64
),
# 配置倒排索引(用于关键词搜索)
inverted_index_config=Configure.inverted_index(
bm25_b=0.75,
bm25_k1=1.2,
stopwords_preset="en",
index_timestamps=True
)
)
print(f"集合 '{collection_name}' 创建成功")
return collection
4.2 批量导入数据
当处理大量文档时,批量导入是提高效率的关键。Weaviate的批量API支持高效的并发写入:
import uuid
from datetime import datetime
from typing import List, Dict, Any
class WeaviateDataImporter:
"""Weaviate数据导入器"""
def __init__(self, client: weaviate.WeaviateClient):
self.client = client
def import_documents(self, documents: List[Dict[str, Any]],
collection_name: str = "KnowledgeBase",
batch_size: int = 100):
"""导入文档到Weaviate"""
collection = self.client.collections.get(collection_name)
chunker = SmartChunker(chunk_size=500, overlap=50)
total_chunks = 0
batch_objects = []
for doc in documents:
# 文档元数据
doc_metadata = {
"doc_id": doc.get("id", str(uuid.uuid4())),
"title": doc.get("title", ""),
"category": doc.get("category", ""),
"tags": doc.get("tags", []),
"author": doc.get("author", ""),
"created_at": doc.get("created_at", datetime.now().isoformat())
}
# 分块处理
chunks = chunker.chunk_markdown(
content=doc.get("content", ""),
metadata=doc_metadata
)
for chunk in chunks:
# 准备对象数据
properties = {
"content": chunk.content,
"title": doc_metadata["title"],
"category": doc_metadata["category"],
"tags": doc_metadata["tags"],
"author": doc_metadata["author"],
"created_at": doc_metadata["created_at"],
"doc_id": doc_metadata["doc_id"],
"chunk_id": chunk.chunk_id,
"section_header": chunk.metadata.get("section_header", "")
}
# 添加到批量列表
batch_objects.append(properties)
total_chunks += 1
# 达到批次大小时提交
if len(batch_objects) >= batch_size:
self._batch_insert(collection, batch_objects)
batch_objects = []
print(f"已导入 {total_chunks} 个分块...")
# 插入剩余的分块
if batch_objects:
self._batch_insert(collection, batch_objects)
print(f"导入完成!总共导入 {total_chunks} 个分块")
# 刷新索引以确保数据可搜索
collection = self.client.collections.get(collection_name)
collection.config.update(
inverted_index_config=Configure.inverted_index(
bm25_b=0.75,
bm25_k1=1.2,
cleanup_interval_seconds=300
)
)
def _batch_insert(self, collection, objects: List[Dict]):
"""批量插入对象"""
with collection.batch.dynamic() as batch:
for obj in objects:
batch.add_object(
properties=obj,
uuid=uuid.uuid4() # 自动生成UUID
)
# 等待索引更新
import time
time.sleep(0.1) # 短暂延迟确保索引更新
4.3 验证数据导入
导入数据后,我们需要验证数据是否正确存储:
def verify_data_import(client: weaviate.WeaviateClient,
collection_name: str = "KnowledgeBase"):
"""验证数据导入结果"""
collection = client.collections.get(collection_name)
# 获取集合统计信息
stats = collection.aggregate.over_all()
print(f"集合统计:")
print(f" 总对象数: {stats.total_count}")
# 抽样检查数据
response = collection.query.fetch_objects(
limit=3,
include_vector=False # 不返回向量以节省带宽
)
print("\n抽样数据:")
for i, obj in enumerate(response.objects):
print(f"\n对象 {i+1}:")
print(f" ID: {obj.uuid}")
props = obj.properties
print(f" 标题: {props.get('title', 'N/A')}")
print(f" 分类: {props.get('category', 'N/A')}")
print(f" 内容预览: {props.get('content', 'N/A')[:100]}...")
# 检查向量化状态
config = collection.config.get()
print(f"\n向量化配置:")
print(f" 向量化器: {config.vectorizer.vectorizer}")
print(f" 向量索引类型: {config.vector_index_config.type}")
return stats.total_count
5. 实现智能检索系统
5.1 基础语义搜索
语义搜索是RAG系统的核心。Weaviate提供了多种搜索方式,我们先从最简单的语义搜索开始:
class SemanticSearcher:
"""语义搜索器"""
def __init__(self, client: weaviate.WeaviateClient):
self.client = client
def semantic_search(self, query: str,
collection_name: str = "KnowledgeBase",
limit: int = 5,
certainty: float = 0.7) -> List[Dict]:
"""纯语义搜索"""
collection = self.client.collections.get(collection_name)
# 执行语义搜索
response = collection.query.near_text(
query=query,
limit=limit,
certainty=certainty,
return_metadata=weaviate.classes.query.MetadataQuery(
distance=True,
certainty=True
)
)
results = []
for obj in response.objects:
result = {
"content": obj.properties.get("content", ""),
"title": obj.properties.get("title", ""),
"category": obj.properties.get("category", ""),
"author": obj.properties.get("author", ""),
"distance": obj.metadata.distance,
"certainty": obj.metadata.certainty,
"uuid": str(obj.uuid)
}
results.append(result)
return results
def search_with_filters(self, query: str,
filters: Dict = None,
collection_name: str = "KnowledgeBase",
limit: int = 5) -> List[Dict]:
"""带过滤条件的语义搜索"""
collection = self.client.collections.get(collection_name)
# 构建过滤条件
filter_conditions = None
if filters:
filter_conditions = self._build_filters(filters)
# 执行搜索
response = collection.query.near_text(
query=query,
limit=limit,
filters=filter_conditions,
return_metadata=weaviate.classes.query.MetadataQuery(
distance=True
)
)
return self._format_results(response)
def _build_filters(self, filters: Dict):
"""构建Weaviate过滤条件"""
from weaviate.classes.query import Filter
conditions = []
for field, value in filters.items():
if field == "category":
conditions.append(Filter.by_property("category").equal(value))
elif field == "tags":
conditions.append(Filter.by_property("tags").contains_any(value))
elif field == "author":
conditions.append(Filter.by_property("author").equal(value))
elif field == "date_range":
start_date, end_date = value
conditions.append(
Filter.by_property("created_at").greater_or_equal(start_date)
& Filter.by_property("created_at").less_or_equal(end_date)
)
# 组合所有条件
if conditions:
combined_filter = conditions[0]
for condition in conditions[1:]:
combined_filter = combined_filter & condition
return combined_filter
return None
def _format_results(self, response):
"""格式化搜索结果"""
results = []
for obj in response.objects:
results.append({
"content": obj.properties.get("content", ""),
"title": obj.properties.get("title", ""),
"category": obj.properties.get("category", ""),
"author": obj.properties.get("author", ""),
"distance": obj.metadata.distance,
"uuid": str(obj.uuid)
})
return results
5.2 混合搜索实现
混合搜索结合了语义搜索和关键词搜索的优势,通常能提供更准确的检索结果:
class HybridSearcher:
"""混合搜索器"""
def __init__(self, client: weaviate.WeaviateClient):
self.client = client
def hybrid_search(self, query: str,
collection_name: str = "KnowledgeBase",
alpha: float = 0.5,
limit: int = 5,
filters: Dict = None) -> List[Dict]:
"""
混合搜索:结合语义搜索和关键词搜索
参数:
query: 搜索查询
alpha: 混合权重 (0-1),0表示纯关键词搜索,1表示纯语义搜索
limit: 返回结果数量
filters: 过滤条件
"""
collection = self.client.collections.get(collection_name)
# 构建过滤条件
filter_conditions = None
if filters:
filter_conditions = self._build_filters(filters)
# 执行混合搜索
response = collection.query.hybrid(
query=query,
alpha=alpha, # 控制语义搜索和关键词搜索的权重
limit=limit,
filters=filter_conditions,
return_metadata=weaviate.classes.query.MetadataQuery(
score=True,
explain_score=True
)
)
results = []
for obj in response.objects:
# 解析得分解释
explanation = obj.metadata.explain_score
semantic_score = explanation.get("vector_distance", 0)
keyword_score = explanation.get("bm25_score", 0)
total_score = obj.metadata.score
results.append({
"content": obj.properties.get("content", ""),
"title": obj.properties.get("title", ""),
"category": obj.properties.get("category", ""),
"score": total_score,
"semantic_score": semantic_score,
"keyword_score": keyword_score,
"explanation": explanation,
"uuid": str(obj.uuid)
})
# 按总分排序
results.sort(key=lambda x: x["score"], reverse=True)
return results
def adaptive_hybrid_search(self, query: str,
collection_name: str = "KnowledgeBase",
limit: int = 5) -> List[Dict]:
"""
自适应混合搜索:根据查询类型动态调整alpha值
这个功能在实际项目中特别有用,因为不同类型的查询
可能更适合语义搜索或关键词搜索
"""
# 分析查询类型
query_type = self._analyze_query_type(query)
# 根据查询类型调整alpha
if query_type == "factual":
# 事实型查询:更依赖关键词匹配
alpha = 0.3
elif query_type == "conceptual":
# 概念型查询:更依赖语义理解
alpha = 0.8
else:
# 默认平衡
alpha = 0.5
print(f"查询类型: {query_type}, 使用alpha: {alpha}")
return self.hybrid_search(query, collection_name, alpha, limit)
def _analyze_query_type(self, query: str) -> str:
"""分析查询类型"""
query_lower = query.lower()
# 事实型查询关键词
factual_keywords = ["什么", "何时", "哪里", "谁", "如何", "多少", "定义", "解释"]
# 概念型查询关键词
conceptual_keywords = ["为什么", "比较", "区别", "关系", "影响", "意义", "理解"]
factual_count = sum(1 for keyword in factual_keywords if keyword in query_lower)
conceptual_count = sum(1 for keyword in conceptual_keywords if keyword in query_lower)
if factual_count > conceptual_count:
return "factual"
elif conceptual_count > factual_count:
return "conceptual"
else:
return "balanced"
def _build_filters(self, filters: Dict):
"""构建过滤条件(与SemanticSearcher相同)"""
from weaviate.classes.query import Filter
conditions = []
for field, value in filters.items():
if field == "category":
conditions.append(Filter.by_property("category").equal(value))
elif field == "tags":
conditions.append(Filter.by_property("tags").contains_any(value))
if conditions:
combined_filter = conditions[0]
for condition in conditions[1:]:
combined_filter = combined_filter & condition
return combined_filter
return None
5.3 重排序优化
为了提高检索质量,我们可以对初步检索结果进行重排序:
class Reranker:
"""重排序器"""
def __init__(self, model_name: str = "BAAI/bge-reranker-base"):
"""初始化重排序模型"""
# 这里可以使用本地模型或API服务
# 实际项目中可能需要集成专门的reranker服务
self.model_name = model_name
def rerank(self, query: str, candidates: List[Dict], top_k: int = 3) -> List[Dict]:
"""
对候选结果进行重排序
参数:
query: 原始查询
candidates: 候选结果列表
top_k: 返回的重排序结果数量
"""
if not candidates:
return []
# 计算每个候选与查询的相关性分数
scored_candidates = []
for candidate in candidates:
score = self._calculate_relevance_score(query, candidate["content"])
candidate["rerank_score"] = score
scored_candidates.append(candidate)
# 按重排序分数排序
scored_candidates.sort(key=lambda x: x["rerank_score"], reverse=True)
# 返回top_k结果
return scored_candidates[:top_k]
def _calculate_relevance_score(self, query: str, document: str) -> float:
"""
计算查询与文档的相关性分数
注意:这是一个简化实现
实际项目中应该使用专门的reranker模型
"""
# 简单的基于重叠的评分
query_words = set(query.lower().split())
doc_words = set(document.lower().split())
# Jaccard相似度
intersection = len(query_words.intersection(doc_words))
union = len(query_words.union(doc_words))
if union == 0:
return 0.0
jaccard_similarity = intersection / union
# 考虑查询词在文档中的位置
# 出现在开头的词通常更重要
position_bonus = 0.0
for word in query_words:
if word in document.lower():
# 找到第一次出现的位置
position = document.lower().find(word)
if position != -1:
# 位置越靠前,加分越多
position_bonus += max(0, 1 - position / len(document))
# 综合分数
final_score = 0.7 * jaccard_similarity + 0.3 * (position_bonus / len(query_words))
return min(1.0, final_score)
6. 集成生成式AI构建完整RAG系统
6.1 构建RAG问答引擎
现在我们将检索系统与生成式AI结合,构建完整的RAG问答引擎:
from typing import Optional, List, Dict, Any
import openai
class RAGQASystem:
"""RAG问答系统"""
def __init__(self,
weaviate_client: weaviate.WeaviateClient,
openai_api_key: str,
model: str = "gpt-3.5-turbo"):
"""
初始化RAG问答系统
参数:
weaviate_client: Weaviate客户端
openai_api_key: OpenAI API密钥
model: 使用的GPT模型
"""
self.client = weaviate_client
self.openai_client = openai.OpenAI(api_key=openai_api_key)
self.model = model
# 初始化搜索器
self.semantic_searcher = SemanticSearcher(weaviate_client)
self.hybrid_searcher = HybridSearcher(weaviate_client)
self.reranker = Reranker()
# 系统提示词模板
self.system_prompt = """你是一个专业的AI助手,基于提供的上下文信息回答问题。
请遵循以下规则:
1. 只基于提供的上下文信息回答问题
2. 如果上下文信息不足以回答问题,请明确说明
3. 保持回答准确、简洁、有帮助
4. 不要编造信息
5. 如果上下文中有多个相关信息,请综合回答
上下文信息:
{context}
用户问题:{question}
请基于以上上下文信息回答问题:"""
def answer_question(self,
question: str,
collection_name: str = "KnowledgeBase",
search_type: str = "hybrid",
top_k: int = 5,
rerank: bool = True) -> Dict[str, Any]:
"""
回答问题
参数:
question: 用户问题
collection_name: 集合名称
search_type: 搜索类型(semantic/hybrid)
top_k: 检索文档数量
rerank: 是否进行重排序
"""
# 步骤1:检索相关文档
if search_type == "semantic":
retrieved_docs = self.semantic_searcher.semantic_search(
question, collection_name, limit=top_k * 2 # 检索更多以便重排序
)
else: # hybrid
retrieved_docs = self.hybrid_searcher.adaptive_hybrid_search(
question, collection_name, limit=top_k * 2
)
# 步骤2:重排序(可选)
if rerank and len(retrieved_docs) > top_k:
retrieved_docs = self.reranker.rerank(question, retrieved_docs, top_k=top_k)
else:
retrieved_docs = retrieved_docs[:top_k]
# 步骤3:构建上下文
context = self._build_context(retrieved_docs)
# 步骤4:生成回答
answer = self._generate_answer(question, context)
# 步骤5:收集元数据
metadata = {
"retrieved_docs": len(retrieved_docs),
"search_type": search_type,
"rerank_applied": rerank,
"source_documents": [
{
"title": doc.get("title", ""),
"category": doc.get("category", ""),
"content_preview": doc.get("content", "")[:200] + "...",
"score": doc.get("score", doc.get("certainty", 0))
}
for doc in retrieved_docs[:3] # 只返回前3个源文档
]
}
return {
"answer": answer,
"metadata": metadata,
"context": context[:1000] # 返回部分上下文用于调试
}
def _build_context(self, documents: List[Dict]) -> str:
"""构建上下文字符串"""
context_parts = []
for i, doc in enumerate(documents, 1):
content = doc.get("content", "")
title = doc.get("title", "")
category = doc.get("category", "")
context_part = f"[文档 {i} - {title} ({category})]\n{content}\n"
context_parts.append(context_part)
return "\n".join(context_parts)
def _generate_answer(self, question: str, context: str) -> str:
"""使用GPT生成回答"""
# 构建提示词
prompt = self.system_prompt.format(
context=context,
question=question
)
try:
response = self.openai_client.chat.completions.create(
model=self.model,
messages=[
{"role": "system", "content": "你是一个基于上下文信息回答问题的AI助手。"},
{"role": "user", "content": prompt}
],
temperature=0.3, # 较低的温度以获得更确定的回答
max_tokens=500
)
answer = response.choices[0].message.content.strip()
return answer
except Exception as e:
return f"生成回答时出错: {str(e)}"
def evaluate_answer(self, question: str, answer: str, context: str) -> Dict[str, Any]:
"""
评估回答质量
在实际项目中,这可以用于监控和优化系统性能
"""
evaluation_prompt = f"""
请评估以下问答对的质量:
问题:{question}
上下文信息:
{context[:500]}...
回答:
{answer}
请从以下维度评分(1-5分):
1. 相关性:回答是否与问题相关
2. 准确性:回答是否基于上下文信息
3. 完整性:回答是否完整回答了问题
4. 清晰度:回答是否清晰易懂
请给出每个维度的分数和简要理由。
"""
try:
response = self.openai_client.chat.completions.create(
model=self.model,
messages=[
{"role": "system", "content": "你是一个问答质量评估专家。"},
{"role": "user", "content": evaluation_prompt}
],
temperature=0.1,
max_tokens=300
)
evaluation_text = response.choices[0].message.content.strip()
# 解析评估结果(简化版)
# 实际项目中可能需要更复杂的解析逻辑
scores = {
"relevance": 3,
"accuracy": 3,
"completeness": 3,
"clarity": 3,
"overall": 3,
"feedback": evaluation_text
}
return scores
except Exception as e:
return {
"error": str(e),
"relevance": 0,
"accuracy": 0,
"completeness": 0,
"clarity": 0,
"overall": 0,
"feedback": "评估失败"
}
6.2 实现对话历史管理
对于多轮对话场景,我们需要管理对话历史:
class ConversationManager:
"""对话管理器"""
def __init__(self, max_history: int = 10):
self.max_history = max_history
self.conversations = {} # conversation_id -> history
def add_message(self, conversation_id: str, role: str, content: str):
"""添加消息到对话历史"""
if conversation_id not in self.conversations:
self.conversations[conversation_id] = []
self.conversations[conversation_id].append({
"role": role,
"content": content,
"timestamp": datetime.now().isoformat()
})
# 保持历史长度
if len(self.conversations[conversation_id]) > self.max_history * 2: # *2因为包含用户和助手消息
self.conversations[conversation_id] = self.conversations[conversation_id][-self.max_history*2:]
def get_history(self, conversation_id: str, max_messages: int = None) -> List[Dict]:
"""获取对话历史"""
if conversation_id not in self.conversations:
return []
history = self.conversations[conversation_id]
if max_messages:
history = history[-max_messages:]
return history
def clear_history(self, conversation_id: str):
"""清空对话历史"""
if conversation_id in self.conversations:
self.conversations[conversation_id] = []
def format_history_for_context(self, conversation_id: str) -> str:
"""格式化对话历史作为上下文"""
history = self.get_history(conversation_id)
if not history:
return ""
formatted = []
for msg in history[-5:]: # 只使用最近5条消息
role = "用户" if msg["role"] == "user" else "助手"
formatted.append(f"{role}: {msg['content']}")
return "\n".join(formatted)
6.3 完整RAG系统集成
现在让我们把所有组件集成在一起:
class CompleteRAGSystem:
"""完整的RAG系统"""
def __init__(self,
weaviate_client: weaviate.WeaviateClient,
openai_api_key: str,
collection_name: str = "KnowledgeBase"):
self.client = weaviate_client
self.collection_name = collection_name
# 初始化各个组件
self.qa_system = RAGQASystem(weaviate_client, openai_api_key)
self.conversation_manager = ConversationManager()
# 缓存常见问题答案
self.cache = {}
self.cache_ttl = 3600 # 1小时
def chat(self,
question: str,
conversation_id: str = "default",
use_cache: bool = True,
search_type: str = "hybrid") -> Dict[str, Any]:
"""
处理用户问题
参数:
question: 用户问题
conversation_id: 对话ID,用于维护上下文
use_cache: 是否使用缓存
search_type: 搜索类型
"""
# 检查缓存
cache_key = f"{conversation_id}:{question}"
if use_cache and cache_key in self.cache:
cached_result = self.cache[cache_key]
if time.time() - cached_result["timestamp"] < self.cache_ttl:
print(f"使用缓存答案")
return cached_result["result"]
# 添加上下文到问题中
enhanced_question = self._enhance_question_with_context(
question, conversation_id
)
# 获取答案
result = self.qa_system.answer_question(
question=enhanced_question,
collection_name=self.collection_name,
search_type=search_type,
top_k=5,
rerank=True
)
# 更新对话历史
self.conversation_manager.add_message(
conversation_id, "user", question
)
self.conversation_manager.add_message(
conversation_id, "assistant", result["answer"]
)
# 添加额外信息到结果
result["conversation_id"] = conversation_id
result["timestamp"] = datetime.now().isoformat()
# 缓存结果
if use_cache:
self.cache[cache_key] = {
"result": result,
"timestamp": time.time()
}
return result
def _enhance_question_with_context(self, question: str, conversation_id: str) -> str:
"""使用对话历史增强问题"""
history_context = self.conversation_manager.format_history_for_context(
conversation_id
)
if not history_context:
return question
# 基于历史上下文重新表述问题
enhancement_prompt = f"""
基于以下对话历史,重新表述用户的最新问题,使其更完整:
对话历史:
{history_context}
最新问题:{question}
重新表述的问题(保持原意,但考虑历史上下文):
"""
try:
# 使用GPT重新表述问题
response = self.qa_system.openai_client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "你是一个问题重新表述助手。"},
{"role": "user", "content": enhancement_prompt}
],
temperature=0.1,
max_tokens=100
)
enhanced = response.choices[0].message.content.strip()
return enhanced
except Exception:
# 如果重新表述失败,返回原问题
return question
def get_conversation_summary(self, conversation_id: str) -> Dict[str, Any]:
"""获取对话摘要"""
history = self.conversation_manager.get_history(conversation_id)
if not history:
return {"summary": "无对话历史", "message_count": 0}
# 生成对话摘要
summary_prompt = f"""
请总结以下对话的主要内容:
{self.conversation_manager.format_history_for_context(conversation_id)}
摘要(50字以内):
"""
try:
response = self.qa_system.openai_client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "你是一个对话摘要助手。"},
{"role": "user", "content": summary_prompt}
],
temperature=0.1,
max_tokens=100
)
summary = response.choices[0].message.content.strip()
return {
"summary": summary,
"message_count": len(history),
"last_message": history[-1]["content"][:100] if history else "",
"start_time": history[0]["timestamp"] if history else "",
"end_time": history[-1]["timestamp"] if history else ""
}
except Exception as e:
return {
"summary": f"生成摘要失败: {str(e)}",
"message_count": len(history)
}
7. 性能优化与监控
7.1 查询性能优化
在实际生产环境中,查询性能至关重要。以下是一些优化策略:
class PerformanceOptimizer:
"""性能优化器"""
def __init__(self, client: weaviate.WeaviateClient):
self.client = client
self.query_cache = {}
self.cache_size = 1000
self.stats = {
"total_queries": 0,
"cache_hits": 0,
"avg_response_time": 0,
"error_count": 0
}
def optimize_query(self, query: str, collection_name: str, **kwargs) -> Dict:
"""
优化查询性能
实现策略:
1. 查询缓存
2. 自适应搜索参数
3. 连接池管理
4. 批量查询优化
"""
start_time = time.time()
# 检查缓存
cache_key = self._generate_cache_key(query, collection_name, kwargs)
if cache_key in self.query_cache:
self.stats["cache_hits"] += 1
result = self.query_cache[cache_key]
result["cached"] = True
return result
# 自适应调整搜索参数
optimized_kwargs = self._adjust_search_params(query, kwargs)
try:
# 执行查询
collection = self.client.collections.get(collection_name)
# 根据查询复杂度选择搜索方式
if self._is_complex_query(query):
# 复杂查询使用混合搜索
response = collection.query.hybrid(
query=query,
**optimized_kwargs
)
else:
# 简单查询使用语义搜索
response = collection.query.near_text(
query=query,
**optimized_kwargs
)
# 处理结果
results = self._process_results(response)
# 计算响应时间
response_time = time.time() - start_time
# 更新统计信息
self.stats["total_queries"] += 1
self.stats["avg_response_time"] = (
(self.stats["avg_response_time"] * (self.stats["total_queries"] - 1) + response_time)
/ self.stats["total_queries"]
)
# 缓存结果
self._add_to_cache(cache_key, {
"results": results,
"response_time": response_time,
"timestamp": time.time()
})
return {
"results": results,
"response_time": response_time,
"cached": False,
"optimization_applied": True
}
except Exception as e:
self.stats["error_count"] += 1
raise e
def _generate_cache_key(self, query: str, collection_name: str, params: Dict) -> str:
"""生成缓存键"""
import hashlib
# 规范化参数
normalized_params = json.dumps(params, sort_keys=True)
key_string = f"{query}:{collection_name}:{normalized_params}"
return hashlib.md5(key_string.encode()).hexdigest()
def _adjust_search_params(self, query: str, params: Dict) -> Dict:
"""根据查询调整搜索参数"""
adjusted = params.copy()
# 根据查询长度调整limit
query_length = len(query.split())
if query_length < 3:
adjusted["limit"] = min(params.get("limit", 10), 5)
elif query_length > 20:
adjusted["limit"] = params.get("limit", 10) + 5
# 根据查询复杂度调整certainty
if self._is_complex_query(query):
adjusted["certainty"] = params.get("certainty", 0.7) * 0.9 # 降低阈值
return adjusted
def _is_complex_query(self, query: str) -> bool:
"""判断是否为复杂查询"""
# 基于启发式规则
complex_indicators = [
"和", "或", "与", "比较", "区别", "关系",
"如何", "为什么", "哪些", "什么情况下"
]
query_lower = query.lower()
return any(indicator in query_lower for indicator in complex_indicators)
def _process_results(self, response):
"""处理查询结果"""
results = []
for obj in response.objects:
result = {
"content": obj.properties.get("content", ""),
"title": obj.properties.get("title", ""),
"score": getattr(obj.metadata, "score", getattr(obj.metadata, "certainty", 0)),
"uuid": str(obj.uuid)
}
results.append(result)
return results
def _add_to_cache(self, key: str, value: Dict):
"""添加到缓存"""
if len(self.query_cache) >= self.cache_size:
# 移除最旧的缓存项
oldest_key = min(self.query_cache.keys(),
key=lambda k: self.query_cache[k]["timestamp"])
del self.query_cache[oldest_key]
self.query_cache[key] = value
def get_performance_stats(self) -> Dict:
"""获取性能统计"""
return {
**self.stats,
"cache_size": len(self.query_cache),
"cache_hit_rate": self.stats["cache_hits"] / max(self.stats["total_queries"], 1)
}
7.2 监控与日志
完善的监控系统对于生产环境至关重要:
import logging
from datetime import datetime
from typing import Dict, Any
class RAGMonitor:
"""RAG系统监控器"""
def __init__(self, log_file: str = "rag_system.log"):
self.logger = self._setup_logger(log_file)
self.metrics = {
"queries": [],
"response_times": [],
"error_counts": [],
"cache_performance": []
}
def _setup_logger(self, log_file: str) -> logging.Logger:
"""设置日志记录器"""
logger = logging.getLogger("RAGSystem")
logger.setLevel(logging.INFO)
# 文件处理器
file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.INFO)
# 控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
# 格式化器
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
logger.addHandler(file_handler)
logger.addHandler(console_handler)
return logger
def log_query(self, query: str, response_time: float,
result_count: int, error: str = None):
"""记录查询日志"""
log_entry = {
"timestamp": datetime.now().isoformat(),
"query": query[:100], # 只记录前100个字符
"response_time": response_time,
"result_count": result_count,
"error": error
}
self.metrics["queries"].append(log_entry)
if error:
self.logger.error(f"查询失败: {query[:50]}... - 错误: {error}")
else:
self.logger.info(
f"查询: {query[:50]}... - "
f"响应时间: {response_time:.3f}s - "
f"结果数: {result_count}"
)
# 保持最近的1000条记录
if len(self.metrics["queries"]) > 1000:
self.metrics["queries"] = self.metrics["queries"][-1000:]
def log_performance_metrics(self, metrics: Dict[str, Any]):
"""记录性能指标"""
self.metrics["response_times"].append({
"timestamp": datetime.now().isoformat(),
"avg_response_time": metrics.get("avg_response_time", 0),
"p95_response_time": metrics.get("p95_response_time", 0)
})
self.metrics["error_counts"].append({
"timestamp": datetime.now().isoformat(),
"error_count": metrics.get("error_count", 0)
})
# 定期写入汇总报告
if len(self.metrics["response_times"]) % 100 == 0:
self._write_summary_report()
def _write_summary_report(self):
"""写入汇总报告"""
if not self.metrics["response_times"]:
return
# 计算统计信息
response_times = [m["avg_response_time"] for m in self.metrics["response_times"][-100:]]
avg_response = sum(response_times) / len(response_times) if response_times else 0
error_counts = [m["error_count"] for m in self.metrics["error_counts"][-100:]]
total_errors = sum(error_counts)
report = f"""
RAG系统性能报告 - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
===================================================
平均响应时间: {avg_response:.3f}s
最近100次查询错误数: {total_errors}
总查询次数: {len(self.metrics['queries'])}
缓存命中率: {self._calculate_cache_hit_rate():.2%}
===================================================
"""
self.logger.info(report)
def _calculate_cache_hit_rate(self) -> float:
"""计算缓存命中率"""
if not self.metrics["queries"]:
return 0.0
recent_queries = self.metrics["queries"][-100:]
cached_queries = [q for q in recent_queries if q.get("cached", False)]
return len(cached_queries) / len(recent_queries) if recent_queries else 0.0
def get_system_health(self) -> Dict[str, Any]:
"""获取系统健康状态"""
if not self.metrics["queries"]:
return {"status": "unknown", "message": "No queries recorded"}
recent_queries = self.metrics["queries"][-50:]
# 计算错误率
error_count = sum(1 for q in recent_queries if q.get("error"))
error_rate = error_count / len(recent_queries) if recent_queries else 0
# 计算平均响应时间
response_times = [q["response_time"] for q in recent_queries if q.get("response_time")]
avg_response = sum(response_times) / len(response_times) if response_times else 0
# 判断系统状态
if error_rate > 0.1: # 错误率超过10%
status = "degraded"
elif avg_response > 2.0: # 平均响应时间超过2秒
status = "slow"
else:
status = "healthy"
return {
"status": status,
"error_rate": error_rate,
"avg_response_time": avg_response,
"total_queries": len(self.metrics["queries"]),
"last_query_time": recent_queries[-1]["timestamp"] if recent_queries else None
}
8. 部署与生产环境考虑
8.1 配置生产环境Weaviate
对于生产环境,我们需要更稳健的配置:
# docker-compose.prod.yml
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:1.33.1
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
ports:
- "8080:8080"
- "50051:50051"
volumes:
- weaviate_data:/var/lib/weaviate
- ./backups:/backups
restart: unless-stopped
environment:
# 基础配置
QUERY_DEFAULTS_LIMIT: 100
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
# 认证配置(生产环境必须启用)
AUTHENTICATION_APIKEY_ENABLED: 'true'
AUTHENTICATION_APIKEY_ALLOWED_KEYS: '${WEAVIATE_API_KEYS}'
AUTHENTICATION_APIKEY_USERS: '${WEAVIATE_API_USERS}'
AUTHORIZATION_ADMINLIST_ENABLED: 'true'
AUTHORIZATION_ADMINLIST_USERS: '${WEAVIATE_ADMIN_USERS}'
# 集群配置
CLUSTER_HOSTNAME: 'node1'
CLUSTER_GOSSIP_BIND_PORT: '7100'
CLUSTER_DATA_BIND_PORT: '7101'
# 性能优化
MAX_IMPORT_GOROUTINES_FACTOR: '2'
DISABLE_GRAPHQL_INTROSPECTION: 'false'
DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
ENABLE_MODULES: 'text2vec-openai,generative-openai,qna-openai'
# OpenAI集成
OPENAI_APIKEY: '${OPENAI_API_KEY}'
OPENAI_ORGANIZATION: '${OPENAI_ORG_ID}'
# 资源限制
MAX_CONCURRENT_READ_REQUESTS: '100'
MAX_CONCURRENT_WRITE_REQUESTS: '20'
# 备份配置
BACKUP_FILESYSTEM_PATH: '/backups'
BACKUP_S3_BUCKET: '${BACKUP_S3_BUCKET}'
BACKUP_S3_ENDPOINT: '${BACKUP_S3_ENDPOINT}'
BACKUP_S3_ACCESS_KEY_ID: '${BACKUP_S3_ACCESS_KEY}'
BACKUP_S3_SECRET_ACCESS_KEY: '${BACKUP_S3_SECRET_KEY}'
deploy:
resources:
limits:
memory: 8G
reservations:
memory: 4G
restart_policy:
condition: on-failure
max_attempts: 3
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8080/v1/.well-known/ready"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
# 可选:添加监控服务
prometheus:
image: prom/prometheus:latest
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
ports:
- "9090:9090"
restart: unless-stopped
grafana:
image: grafana/grafana:latest
volumes:
- grafana_data:/var/lib/grafana
ports:
- "3000:3000"
restart: unless-stopped
environment:
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
volumes:
weaviate_data:
prometheus_data:
grafana_data:
8.2 备份与恢复策略
数据安全至关重要,以下是备份策略的实现:
class WeaviateBackupManager:
"""Weaviate备份管理器"""
def __init__(self, client: weaviate.WeaviateClient, backup_dir: str = "./backups"):
self.client = client
self.backup_dir = backup_dir
os.makedirs(backup_dir, exist_ok=True)
def create_backup(self, backup_name: str = None) -> Dict[str, Any]:
"""创建备份"""
if backup_name is None:
backup_name = f"backup_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
try:
# 创建备份
result = self.client.backup.create(
backup_id=backup_name,
backend="filesystem",
include=[] # 空列表表示备份所有集合
)
# 等待备份完成
status = self._wait_for_backup(backup_name)
if status == "SUCCESS":
backup_path = os.path.join(self.backup_dir, backup_name)
backup_info = {
"name": backup_name,
"path": backup_path,
"status": status,
"created_at": datetime.now().isoformat(),
"size": self._get_backup_size(backup_name)
}
# 记录备份信息
self._log_backup_info(backup_info)
return backup_info
else:
raise Exception(f"备份失败,状态: {status}")
except Exception as e:
raise Exception(f"创建备份时出错: {str(e)}")
def restore_backup(self, backup_name: str,
collections: List[str] = None) -> Dict[str, Any]:
"""恢复备份"""
try:
# 检查备份是否存在
if not self._backup_exists(backup_name):
raise Exception(f"备份 '{backup_name}' 不存在")
# 执行恢复
result = self.client.backup.restore(
backup_id=backup_name,
backend="filesystem",
include=collections if collections else []
)
# 等待恢复完成
status = self._wait_for_restore(backup_name)
if status == "SUCCESS":
return {
"backup_name": backup_name,
"status": status,
"restored_at": datetime.now().isoformat(),
"collections": collections if collections else "所有集合"
}
else:
raise Exception(f"恢复失败,状态: {status}")
except Exception as e:
raise Exception(f"恢复备份时出错: {str(e)}")
def list_backups(self) -> List[Dict[str, Any]]:
"""列出所有备份"""
try:
result = self.client.backup.get_all()
backups = []
for backup in result:
backup_info = {
"id": backup.id,
"backend": backup.backend,
"status": backup.status,
"created_at": backup.created_at,
"collections": backup.collections
}
backups.append(backup_info)
return backups
except Exception as e:
print(f"获取备份列表时出错: {e}")
return []
def _wait_for_backup(self, backup_name: str, timeout: int = 300) -> str:
"""等待备份完成"""
start_time = time.time()
while time.time() - start_time < timeout:
try:
status = self.client.backup.get_create_status(backup_name)
if status.status in ["SUCCESS", "FAILED"]:
return status.status
time.sleep(5)
except Exception as e:
print(f"检查备份状态时出错: {e}")
time.sleep(5)
return "TIMEOUT"
def _wait_for_restore(self, backup_name: str, timeout: int = 300) -> str:
"""等待恢复完成"""
start_time = time.time()
while time.time() - start_time < timeout:
try:
status = self.client.backup.get_restore_status(backup_name)
if status.status in ["SUCCESS", "FAILED"]:
return status.status
time.sleep(5)
except Exception as e:
print(f"检查恢复状态时出错: {e}")
time.sleep(5)
return "TIMEOUT"
def _backup_exists(self, backup_name: str) -> bool:
"""检查备份是否存在"""
backups = self.list_backups()
return any(backup["id"] == backup_name for backup in backups)
def _get_backup_size(self, backup_name: str) -> int:
"""获取备份大小(字节)"""
backup_path = os.path.join(self.backup_dir, backup_name)
if os.path.exists(backup_path):
return sum(os.path.getsize(os.path.join(dirpath, filename))
for dirpath, dirnames, filenames in os.walk(backup_path)
for filename in filenames)
return 0
def _log_backup_info(self, backup_info: Dict[str, Any]):
"""记录备份信息"""
log_file = os.path.join(self.backup_dir, "backup_log.json")
try:
if os.path.exists(log_file):
with open(log_file, "r") as f:
logs = json.load(f)
else:
logs = []
logs.append(backup_info)
# 只保留最近100条记录
if len(logs) > 100:
logs = logs[-100:]
with open(log_file, "w") as f:
json.dump(logs, f, indent=2, ensure_ascii=False)
except Exception as e:
print(f"记录备份信息时出错: {e}")
8.3 自动化测试脚本
在部署前,运行自动化测试确保系统正常工作:
class RAGSystemTester:
"""RAG系统测试器"""
def __init__(self, rag_system: CompleteRAGSystem):
self.rag_system = rag_system
self.test_cases = self._load_test_cases()
def _load_test_cases(self) -> List[Dict]:
"""加载测试用例"""
return [
{
"question": "什么是机器学习?",
"expected_keywords": ["人工智能", "学习", "数据", "算法"],
"category": "基础概念"
},
{
"question": "监督学习和无监督学习有什么区别?",
"expected_keywords": ["标签", "训练数据", "聚类", "分类"],
"category": "对比问题"
},
{
"question": "神经网络有哪些类型?",
"expected_keywords": ["前馈", "卷积", "循环", "RNN", "CNN"],
"category": "列举问题"
},
{
"question": "过拟合是什么意思?",
"expected_keywords": ["训练数据", "泛化", "新数据", "表现不佳"],
"category": "定义问题"
}
]
def run_tests(self) -> Dict[str, Any]:
"""运行所有测试"""
results = {
"total_tests": len(self.test_cases),
"passed_tests": 0,
"failed_tests": 0,
"test_details": [],
"average_response_time": 0,
"success_rate": 0
}
response_times = []
for i, test_case in enumerate(self.test_cases, 1):
print(f"运行测试 {i}/{len(self.test_cases)}: {test_case['question']}")
start_time = time.time()
try:
# 执行查询
result = self.rag_system.chat(
question=test_case["question"],
conversation_id=f"test_{i}",
use_cache=False
)
response_time = time.time() - start_time
response_times.append(response_time)
# 验证结果
test_result = self._validate_result(
test_case, result, response_time
)
results["test_details"].append(test_result)
if test_result["passed"]:
results["passed_tests"] += 1
print(f" ✓ 通过")
else:
results["failed_tests"] += 1
print(f" ✗ 失败: {test_result.get('failure_reason', '未知原因')}")
except Exception as e:
results["failed_tests"] += 1
error_result = {
"test_case": test_case["question"],
"passed": False,
"error": str(e),
"response_time": 0
}
results["test_details"].append(error_result)
print(f" ✗ 异常: {e}")
# 计算统计信息
if response_times:
results["average_response_time"] = sum(response_times) / len(response_times)
if results["total_tests"] > 0:
results["success_rate"] = results["passed_tests"] / results["total_tests"]
return results
def _validate_result(self, test_case: Dict, result: Dict, response_time: float) -> Dict:
"""验证测试结果"""
validation = {
"test_case": test_case["question"],
"category": test_case["category"],
"response_time": response_time,
"passed": False,
"validation_details": {}
}
try:
answer = result.get("answer", "").lower()
expected_keywords = [kw.lower() for kw in test_case["expected_keywords"]]
# 检查是否包含预期关键词
found_keywords = []
missing_keywords = []
for keyword in expected_keywords:
if keyword in answer:
found_keywords.append(keyword)
else:
missing_keywords.append(keyword)
# 检查是否有检索结果
metadata = result.get("metadata", {})
retrieved_docs = metadata.get("retrieved_docs", 0)
# 验证规则
validation_rules = {
"has_answer": bool(answer and answer.strip()),
"min_keywords_found": len(found_keywords) >= max(1, len(expected_keywords) // 2),
"has_retrieved_docs": retrieved_docs > 0,
"response_time_ok": response_time < 5.0 # 响应时间小于5秒
}
360



被折叠的 条评论
为什么被折叠?



