Spring AI与RAG技术实战：构建企业级智能文档问答系统
引言
随着人工智能技术的快速发展，企业对于智能化文档处理的需求日益增长。传统的文档检索方式往往效率低下，无法满足快速获取准确信息的需求。Spring AI结合RAG（检索增强生成）技术，为企业提供了构建智能文档问答系统的完美解决方案。本文将深入探讨如何利用Spring AI框架和RAG技术构建高效的企业级文档问答系统。
技术架构概述
核心组件
我们的智能文档问答系统主要包含以下核心组件：
- 文档处理模块：负责文档的加载、解析和预处理
- 向量化引擎：将文本转换为向量表示
- 向量数据库：存储和管理文档向量
- 检索增强生成模块：结合检索和生成技术提供准确回答
- Spring AI集成层：提供统一的AI服务接口
技术栈选择
- Spring Boot 3.x：作为基础框架
- Spring AI：提供AI能力集成
- OpenAI Embedding：用于文本向量化
- Milvus向量数据库：存储和检索向量数据
- RAG架构：检索增强生成模式
- Docker & Kubernetes：容器化部署
系统设计与实现
1. 文档处理流程
@Service
public class DocumentProcessor {
@Autowired
private EmbeddingClient embeddingClient;
@Autowired
private VectorStore vectorStore;
public void processDocument(MultipartFile file) {
// 文档解析
String content = parseDocument(file);
// 文本分块
List<TextSegment> segments = splitText(content);
// 向量化处理
List<Embedding> embeddings = embeddingClient.embed(segments);
// 存储到向量数据库
vectorStore.add(embeddings, segments);
}
private String parseDocument(MultipartFile file) {
// 支持多种文档格式解析
String fileName = file.getOriginalFilename();
if (fileName.endsWith(".pdf")) {
return parsePdf(file);
} else if (fileName.endsWith(".docx")) {
return parseDocx(file);
} else if (fileName.endsWith(".txt")) {
return parseText(file);
}
throw new UnsupportedOperationException("Unsupported document format");
}
private List<TextSegment> splitText(String content) {
// 基于语义的文本分块
return TextSplitter.splitBySemantic(content, 500); // 每块约500字符
}
}
2. 向量数据库集成
@Configuration
public class VectorStoreConfig {
@Value("${milvus.host:localhost}")
private String milvusHost;
@Value("${milvus.port:19530}")
private int milvusPort;
@Bean
public VectorStore vectorStore(EmbeddingClient embeddingClient) {
MilvusVectorStoreConfig config = MilvusVectorStoreConfig.builder()
.host(milvusHost)
.port(milvusPort)
.collectionName("document_vectors")
.dimension(1536) // OpenAI embedding维度
.build();
return new MilvusVectorStore(config, embeddingClient);
}
}
3. RAG问答服务实现
@Service
public class RagQAService {
@Autowired
private VectorStore vectorStore;
@Autowired
private ChatClient chatClient;
public String answerQuestion(String question) {
// 1. 检索相关文档片段
List<Document> relevantDocs = retrieveRelevantDocuments(question);
// 2. 构建提示词
String prompt = buildPrompt(question, relevantDocs);
// 3. 生成回答
return generateAnswer(prompt);
}
private List<Document> retrieveRelevantDocuments(String question) {
// 使用问题向量进行相似度检索
Embedding questionEmbedding = embeddingClient.embed(question);
return vectorStore.similaritySearch(
SearchRequest.defaults()
.queryEmbedding(questionEmbedding)
.topK(5) // 返回最相关的5个文档片段
.build()
);
}
private String buildPrompt(String question, List<Document> contextDocs) {
StringBuilder promptBuilder = new StringBuilder();
promptBuilder.append("基于以下文档内容，请回答用户的问题：\n\n");
// 添加上下文文档
for (int i = 0; i < contextDocs.size(); i++) {
promptBuilder.append("文档片段").append(i + 1).append(":\n");
promptBuilder.append(contextDocs.get(i).getText()).append("\n\n");
}
promptBuilder.append("用户问题: ").append(question).append("\n\n");
promptBuilder.append("请根据上述文档内容提供准确、详细的回答。");
return promptBuilder.toString();
}
private String generateAnswer(String prompt) {
return chatClient.call(prompt);
}
}
4. REST API设计
@RestController
@RequestMapping("/api/rag")
public class RagController {
@Autowired
private RagQAService ragQAService;
@Autowired
private DocumentProcessor documentProcessor;
@PostMapping("/upload")
public ResponseEntity<String> uploadDocument(@RequestParam("file") MultipartFile file) {
try {
documentProcessor.processDocument(file);
return ResponseEntity.ok("文档上传和处理成功");
} catch (Exception e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("文档处理失败: " + e.getMessage());
}
}
@PostMapping("/ask")
public ResponseEntity<AnswerResponse> askQuestion(@RequestBody QuestionRequest request) {
try {
String answer = ragQAService.answerQuestion(request.getQuestion());
return ResponseEntity.ok(new AnswerResponse(answer));
} catch (Exception e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(new AnswerResponse("系统繁忙，请稍后重试"));
}
}
@GetMapping("/health")
public ResponseEntity<HealthCheck> healthCheck() {
return ResponseEntity.ok(new HealthCheck("OK", System.currentTimeMillis()));
}
}
@Data
class QuestionRequest {
private String question;
}
@Data
class AnswerResponse {
private String answer;
private long timestamp;
public AnswerResponse(String answer) {
this.answer = answer;
this.timestamp = System.currentTimeMillis();
}
}
@Data
class HealthCheck {
private String status;
private long timestamp;
public HealthCheck(String status, long timestamp) {
this.status = status;
this.timestamp = timestamp;
}
}
性能优化策略
1. 缓存机制
@Configuration
@EnableCaching
public class CacheConfig {
@Bean
public CacheManager cacheManager() {
CaffeineCacheManager cacheManager = new CaffeineCacheManager();
cacheManager.setCaffeine(Caffeine.newBuilder()
.expireAfterWrite(30, TimeUnit.MINUTES)
.maximumSize(1000));
return cacheManager;
}
}
@Service
public class CachedRagService {
@Autowired
private RagQAService ragQAService;
@Cacheable(value = "answers", key = "#question")
public String getCachedAnswer(String question) {
return ragQAService.answerQuestion(question);
}
}
2. 异步处理
@Async
public CompletableFuture<String> processDocumentAsync(MultipartFile file) {
return CompletableFuture.supplyAsync(() -> {
documentProcessor.processDocument(file);
return "处理完成";
});
}
3. 批量操作优化
public void processDocumentsInBatch(List<MultipartFile> files) {
files.parallelStream().forEach(file -> {
try {
documentProcessor.processDocument(file);
} catch (Exception e) {
log.error("文档处理失败: {}", file.getOriginalFilename(), e);
}
});
}
安全考虑
1. 输入验证
public void validateQuestion(String question) {
if (question == null || question.trim().isEmpty()) {
throw new IllegalArgumentException("问题不能为空");
}
if (question.length() > 1000) {
throw new IllegalArgumentException("问题长度超过限制");
}
// 防止注入攻击
if (containsMaliciousContent(question)) {
throw new SecurityException("检测到恶意内容");
}
}
2. 访问控制
@PreAuthorize("hasRole('USER')")
@PostMapping("/ask")
public ResponseEntity<AnswerResponse> askQuestion(@RequestBody QuestionRequest request) {
// 方法实现
}
监控与日志
1. Micrometer监控
@Configuration
public class MetricsConfig {
@Bean
public MeterRegistry meterRegistry() {
return new PrometheusMeterRegistry(PrometheusConfig.DEFAULT);
}
}
@Service
public class MonitoredRagService {
private final Counter questionCounter;
private final Timer responseTimer;
public MonitoredRagService(MeterRegistry registry) {
questionCounter = registry.counter("rag.questions.total");
responseTimer = registry.timer("rag.response.time");
}
public String answerQuestionWithMetrics(String question) {
return responseTimer.record(() -> {
questionCounter.increment();
return ragQAService.answerQuestion(question);
});
}
}
2. 结构化日志
@Slf4j
@Service
public class LoggingRagService {
public String answerQuestionWithLogging(String question) {
log.info("处理用户问题: {}", question);
try {
String answer = ragQAService.answerQuestion(question);
log.info("问题回答成功，答案长度: {}", answer.length());
return answer;
} catch (Exception e) {
log.error("问题处理失败: {}", question, e);
throw e;
}
}
}
部署架构
Docker容器化
FROM openjdk:17-jdk-slim
WORKDIR /app
COPY target/rag-system.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "app.jar"]
Kubernetes部署
apiVersion: apps/v1
kind: Deployment
metadata:
name: rag-system
spec:
replicas: 3
selector:
matchLabels:
app: rag-system
template:
metadata:
labels:
app: rag-system
spec:
containers:
- name: rag-app
image: rag-system:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "1Gi"
cpu: "500m"
limits:
memory: "2Gi"
cpu: "1000m"
---
apiVersion: v1
kind: Service
metadata:
name: rag-service
spec:
selector:
app: rag-system
ports:
- port: 80
targetPort: 8080
测试策略
1. 单元测试
@ExtendWith(MockitoExtension.class)
class RagQAServiceTest {
@Mock
private VectorStore vectorStore;
@Mock
private ChatClient chatClient;
@InjectMocks
private RagQAService ragQAService;
@Test
void testAnswerQuestion() {
// 准备测试数据
String question = "什么是Spring AI？";
Document doc = new Document("Spring AI是Spring框架的AI扩展...");
when(vectorStore.similaritySearch(any())).thenReturn(List.of(doc));
when(chatClient.call(anyString())).thenReturn("Spring AI是...");
// 执行测试
String answer = ragQAService.answerQuestion(question);
// 验证结果
assertNotNull(answer);
verify(vectorStore).similaritySearch(any());
verify(chatClient).call(anyString());
}
}
2. 集成测试
@SpringBootTest
@AutoConfigureMockMvc
class RagControllerIntegrationTest {
@Autowired
private MockMvc mockMvc;
@Test
void testAskQuestion() throws Exception {
QuestionRequest request = new QuestionRequest();
request.setQuestion("测试问题");
mockMvc.perform(post("/api/rag/ask")
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(request)))
.andExpect(status().isOk())
.andExpect(jsonPath("$.answer").exists());
}
}
总结与展望
本文详细介绍了如何使用Spring AI和RAG技术构建企业级智能文档问答系统。通过合理的架构设计、性能优化和安全考虑，我们构建了一个高效、可靠的系统。
未来的改进方向包括：
- 多模态支持：支持图片、表格等非文本内容的处理
- 实时学习：系统能够从用户反馈中持续学习优化
- 个性化推荐：基于用户历史提供个性化答案
- 多语言支持：扩展对多语言文档的处理能力
- 边缘计算：支持在边缘设备上部署轻量级版本
Spring AI和RAG技术的结合为企业智能化转型提供了强大的技术支撑，相信随着技术的不断发展，这类系统将在企业中发挥越来越重要的作用。
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
