MyBatis-Plus与Elasticsearch集成：实现全文搜索和数据分析

MyBatis-Plus与Elasticsearch集成：实现全文搜索和数据分析

【免费下载链接】mybatis-plus An powerful enhanced toolkit of MyBatis for simplify development 项目地址: https://gitcode.com/gh_mirrors/my/mybatis-plus

引言：为什么需要集成Elasticsearch？

在现代应用开发中，传统的关系型数据库虽然能够很好地处理结构化数据，但在全文搜索、复杂查询和实时数据分析方面存在局限性。Elasticsearch作为一款分布式搜索引擎，能够提供：

• 🔍 高性能全文搜索：毫秒级的搜索响应
• 📊 实时数据分析：强大的聚合和统计功能
• 📈 可扩展性：分布式架构支持海量数据
• 🔄 数据同步：与关系型数据库无缝对接

本文将详细介绍如何将MyBatis-Plus与Elasticsearch集成，构建一个既保持事务一致性又具备强大搜索能力的系统架构。

架构设计：双写模式与数据同步

系统架构图

集成模式对比

模式	优点	缺点	适用场景
双写模式	实时性高，架构简单	数据一致性难保证	写操作少的场景
异步同步	性能影响小	数据延迟	读多写少场景
CDC模式	完全解耦，实时同步	架构复杂	大型系统

环境准备与依赖配置

Maven依赖配置

<!-- MyBatis-Plus Spring Boot Starter -->
<dependency>
    <groupId>com.baomidou</groupId>
    <artifactId>mybatis-plus-boot-starter</artifactId>
    <version>3.5.6</version>
</dependency>

<!-- Elasticsearch Java Client -->
<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.13.0</version>
</dependency>

<!-- Jackson Databind -->
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.16.1</version>
</dependency>

<!-- Spring Data Elasticsearch (可选) -->
<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-elasticsearch</artifactId>
    <version>5.3.0</version>
</dependency>

配置文件示例

# application.yml
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/mybatis_plus_demo
    username: root
    password: 123456
    driver-class-name: com.mysql.cj.jdbc.Driver
  
  elasticsearch:
    uris: ["http://localhost:9200"]
    username: elastic
    password: changeme

mybatis-plus:
  configuration:
    map-underscore-to-camel-case: true
    log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
  global-config:
    db-config:
      id-type: auto
      logic-delete-field: deleted
      logic-delete-value: 1
      logic-not-delete-value: 0

核心实现：数据实体与映射

基础实体类设计

@Data
@TableName("user_info")
public class User implements Serializable {
    
    @TableId(type = IdType.AUTO)
    private Long id;
    
    private String username;
    
    private String email;
    
    private Integer age;
    
    private String address;
    
    @TableField(fill = FieldFill.INSERT)
    private Date createTime;
    
    @TableField(fill = FieldFill.INSERT_UPDATE)
    private Date updateTime;
    
    @TableLogic
    private Integer deleted;
}

// Elasticsearch文档实体
@Data
@Document(indexName = "user_index")
public class UserDocument {
    
    @Id
    private Long id;
    
    @Field(type = FieldType.Text, analyzer = "ik_max_word")
    private String username;
    
    @Field(type = FieldType.Keyword)
    private String email;
    
    @Field(type = FieldType.Integer)
    private Integer age;
    
    @Field(type = FieldType.Text, analyzer = "ik_smart")
    private String address;
    
    @Field(type = FieldType.Date)
    private Date createTime;
    
    @Field(type = FieldType.Date)
    private Date updateTime;
}

MyBatis-Plus Mapper接口

public interface UserMapper extends BaseMapper<User> {
    
    @Select("SELECT * FROM user_info WHERE age > #{age}")
    List<User> selectUsersByAge(@Param("age") Integer age);
    
    // 复杂查询示例
    @Select("<script>" +
            "SELECT * FROM user_info " +
            "<where>" +
            "   <if test='username != null'>AND username LIKE CONCAT('%', #{username}, '%')</if>" +
            "   <if test='minAge != null'>AND age >= #{minAge}</if>" +
            "   <if test='maxAge != null'>AND age <= #{maxAge}</if>" +
            "</where>" +
            "ORDER BY create_time DESC" +
            "</script>")
    List<User> searchUsers(@Param("username") String username,
                          @Param("minAge") Integer minAge,
                          @Param("maxAge") Integer maxAge);
}

数据同步策略实现

方案一：基于AOP的双写模式

@Aspect
@Component
@Slf4j
public class ElasticsearchSyncAspect {
    
    @Autowired
    private ElasticsearchService elasticsearchService;
    
    // 插入后同步
    @AfterReturning(pointcut = "execution(* com..mapper.*Mapper.insert*(..))", 
                   returning = "result")
    public void afterInsert(Object result) {
        if (result instanceof Integer && (Integer)result > 0) {
            // 获取插入的实体并同步到ES
            syncToElasticsearch(getEntityFromContext());
        }
    }
    
    // 更新后同步
    @AfterReturning(pointcut = "execution(* com..mapper.*Mapper.update*(..))", 
                   returning = "result")
    public void afterUpdate(Object result) {
        if (result instanceof Integer && (Integer)result > 0) {
            syncToElasticsearch(getEntityFromContext());
        }
    }
    
    // 删除后同步
    @AfterReturning(pointcut = "execution(* com..mapper.*Mapper.delete*(..))", 
                   returning = "result")
    public void afterDelete(Object result) {
        if (result instanceof Integer && (Integer)result > 0) {
            deleteFromElasticsearch(getEntityIdFromContext());
        }
    }
    
    private void syncToElasticsearch(Object entity) {
        try {
            elasticsearchService.indexDocument(entity);
        } catch (Exception e) {
            log.error("同步数据到Elasticsearch失败", e);
            // 可以加入重试机制或记录到日志表
        }
    }
}

方案二：基于事务事件监听器

@Component
@TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
public class ElasticsearchTransactionListener {
    
    @Autowired
    private ElasticsearchSyncService syncService;
    
    public void handleAfterCommit(DataChangeEvent event) {
        switch (event.getOperationType()) {
            case INSERT:
                syncService.syncInsert(event.getEntity());
                break;
            case UPDATE:
                syncService.syncUpdate(event.getEntity());
                break;
            case DELETE:
                syncService.syncDelete(event.getEntityId());
                break;
        }
    }
}

// 事件类
@Data
@AllArgsConstructor
public class DataChangeEvent {
    private OperationType operationType;
    private Object entity;
    private Long entityId;
}

public enum OperationType {
    INSERT, UPDATE, DELETE
}

Elasticsearch服务层实现

基础CRUD操作

@Service
@Slf4j
public class ElasticsearchService {
    
    @Autowired
    private ElasticsearchClient elasticsearchClient;
    
    /**
     * 索引文档
     */
    public <T> void indexDocument(T document) {
        try {
            IndexResponse response = elasticsearchClient.index(i -> i
                .index(getIndexName(document.getClass()))
                .id(getDocumentId(document))
                .document(document));
            
            log.info("文档索引成功: {}", response.id());
        } catch (IOException e) {
            throw new RuntimeException("索引文档失败", e);
        }
    }
    
    /**
     * 根据ID查询文档
     */
    public <T> Optional<T> getDocumentById(Class<T> clazz, String id) {
        try {
            GetResponse<T> response = elasticsearchClient.get(g -> g
                .index(getIndexName(clazz))
                .id(id), clazz);
            
            if (response.found()) {
                return Optional.of(response.source());
            }
            return Optional.empty();
        } catch (IOException e) {
            throw new RuntimeException("查询文档失败", e);
        }
    }
    
    /**
     * 全文搜索
     */
    public <T> SearchResponse<T> searchDocuments(Class<T> clazz, 
                                                String query, 
                                                String... fields) {
        try {
            return elasticsearchClient.search(s -> s
                .index(getIndexName(clazz))
                .query(q -> q
                    .multiMatch(m -> m
                        .query(query)
                        .fields(Arrays.asList(fields))
                    )
                ), clazz);
        } catch (IOException e) {
            throw new RuntimeException("搜索文档失败", e);
        }
    }
    
    /**
     * 高级搜索：多条件组合查询
     */
    public <T> SearchResponse<T> advancedSearch(Class<T> clazz, 
                                               AdvancedSearchRequest request) {
        try {
            List<Query> mustQueries = new ArrayList<>();
            List<Query> shouldQueries = new ArrayList<>();
            List<Query> filterQueries = new ArrayList<>();
            
            // 构建查询条件
            buildQueries(request, mustQueries, shouldQueries, filterQueries);
            
            return elasticsearchClient.search(s -> s
                .index(getIndexName(clazz))
                .query(q -> q
                    .bool(b -> b
                        .must(mustQueries)
                        .should(shouldQueries)
                        .filter(filterQueries)
                        .minimumShouldMatch(request.getMinimumShouldMatch())
                    )
                )
                .from(request.getFrom())
                .size(request.getSize())
                .sort(buildSorts(request.getSorts())), clazz);
        } catch (IOException e) {
            throw new RuntimeException("高级搜索失败", e);
        }
    }
}

复杂查询构建器

@Component
public class ElasticsearchQueryBuilder {
    
    /**
     * 构建范围查询
     */
    public Query buildRangeQuery(String field, Object from, Object to) {
        return Query.of(q -> q
            .range(r -> r
                .field(field)
                .gte(JsonData.of(from))
                .lte(JsonData.of(to))
            )
        );
    }
    
    /**
     * 构建术语查询
     */
    public Query buildTermQuery(String field, Object value) {
        return Query.of(q -> q
            .term(t -> t
                .field(field)
                .value(FieldValue.of(value))
            )
        );
    }
    
    /**
     * 构建匹配查询
     */
    public Query buildMatchQuery(String field, String value) {
        return Query.of(q -> q
            .match(m -> m
                .field(field)
                .query(value)
            )
        );
    }
    
    /**
     * 构建多字段匹配查询
     */
    public Query buildMultiMatchQuery(String query, String... fields) {
        return Query.of(q -> q
            .multiMatch(m -> m
                .query(query)
                .fields(Arrays.asList(fields))
            )
        );
    }
    
    /**
     * 构建聚合查询
     */
    public Aggregation buildTermsAggregation(String field, int size) {
        return Aggregation.of(a -> a
            .terms(t -> t
                .field(field)
                .size(size)
            )
        );
    }
}

业务场景实现示例

场景一：用户全文搜索

@Service
public class UserSearchService {
    
    @Autowired
    private ElasticsearchService elasticsearchService;
    
    @Autowired
    private UserMapper userMapper;
    
    /**
     * 用户综合搜索
     */
    public SearchResponse<UserDocument> searchUsers(UserSearchRequest request) {
        AdvancedSearchRequest searchRequest = new AdvancedSearchRequest();
        
        // 构建查询条件
        if (StringUtils.hasText(request.getKeyword())) {
            searchRequest.addShouldQuery(
                elasticsearchQueryBuilder.buildMultiMatchQuery(
                    request.getKeyword(), 
                    "username", "email", "address"
                )
            );
        }
        
        // 年龄范围过滤
        if (request.getMinAge() != null || request.getMaxAge() != null) {
            searchRequest.addFilterQuery(
                elasticsearchQueryBuilder.buildRangeQuery(
                    "age", 
                    request.getMinAge(), 
                    request.getMaxAge()
                )
            );
        }
        
        // 时间范围过滤
        if (request.getStartTime() != null || request.getEndTime() != null) {
            searchRequest.addFilterQuery(
                elasticsearchQueryBuilder.buildRangeQuery(
                    "createTime", 
                    request.getStartTime(), 
                    request.getEndTime()
                )
            );
        }
        
        // 设置分页和排序
        searchRequest.setFrom(request.getPageable().getOffset());
        searchRequest.setSize(request.getPageable().getPageSize());
        searchRequest.addSort(new Sort("createTime", SortOrder.DESC));
        
        return elasticsearchService.advancedSearch(UserDocument.class, searchRequest);
    }
    
    /**
     * 搜索建议（自动补全）
     */
    public List<String> suggestUsernames(String prefix) {
        try {
            SearchResponse<UserDocument> response = elasticsearchClient.search(s -> s
                .index("user_index")
                .query(q -> q
                    .prefix(p -> p
                        .field("username.keyword")
                        .value(prefix)
                    )
                )
                .source(sc -> sc
                    .filter(f -> f
                        .includes("username")
                    )
                )
                .size(10), UserDocument.class);
            
            return response.hits().hits().stream()
                .map(hit -> hit.source().getUsername())
                .collect(Collectors.toList());
        } catch (IOException e) {
            throw new RuntimeException("获取搜索建议失败", e);
        }
    }
}

场景二：数据分析与统计

@Service
public class UserAnalyticsService {
    
    @Autowired
    private ElasticsearchClient elasticsearchClient;
    
    /**
     * 用户年龄分布统计
     */
    public Map<String, Long> getUserAgeDistribution() {
        try {
            SearchResponse<Void> response = elasticsearchClient.search(s -> s
                .index("user_index")
                .size(0)
                .aggregations("age_distribution", a -> a
                    .terms(t -> t
                        .field("age")
                        .size(100)
                    )
                ), Void.class);
            
            return response.aggregations().get("age_distribution")
                .sterms().buckets().array().stream()
                .collect(Collectors.toMap(
                    bucket -> String.valueOf(bucket.key()),
                    bucket -> bucket.docCount()
                ));
        } catch (IOException e) {
            throw new RuntimeException("获取年龄分布失败", e);
        }
    }
    
    /**
     * 用户注册趋势分析
     */
    public Map<String, Long> getUserRegistrationTrend(Period period) {
        try {
            DateHistogramAggregation dateAggregation = DateHistogramAggregation.of(d -> d
                .field("createTime")
                .calendarInterval(period.getCalendarInterval())
                .format("yyyy-MM-dd"));
            
            SearchResponse<Void> response = elasticsearchClient.search(s -> s
                .index("user_index")
                .size(0)
                .aggregations("registration_trend", a -> a
                    .dateHistogram(dateAggregation)
                ), Void.class);
            
            return response.aggregations().get("registration_trend")
                .dateHistogram().buckets().array().stream()
                .collect(Collectors.toMap(
                    bucket -> bucket.key(),
                    bucket -> bucket.docCount()
                ));
        } catch (IOException e) {
            throw new RuntimeException("获取注册趋势失败", e);
        }
    }
}

性能优化与最佳实践

索引优化策略

@Configuration
public class ElasticsearchIndexConfig {
    
    @PostConstruct
    public void initIndices() {
        createUserIndexWithOptimizedSettings();
    }
    
    private void createUserIndexWithOptimizedSettings() {
        try {
            // 检查索引是否存在
            BooleanResponse exists = elasticsearchClient.indices()
                .exists(e -> e.index("user_index"));
            
            if (!exists.value()) {
                CreateIndexResponse response = elasticsearchClient.indices()
                    .create(c -> c
                        .index("user_index")
                        .settings(s -> s
                            .numberOfShards(3)
                            .numberOfReplicas(1)
                            .analysis(a -> a
                                .analyzer("ik_smart", ana -> ana
                                    .custom(cus -> cus
                                        .tokenizer("ik_smart")
                                    )
                                )
                                .analyzer("ik_max_word", ana -> ana
                                    .custom(cus -> cus
                                        .tokenizer("ik_max_word")
                                    )
                                )
                            )
                        )
                        .mappings(m -> m
                            .properties("username", p -> p
                                .text(t -> t
                                    .analyzer("ik_max_word")
                                    .searchAnalyzer("ik_smart")
                                    .fields("keyword", f -> f
                                        .keyword(k -> k)
                                    )
                                )
                            )
                            .properties("email", p -> p
                                .keyword(k -> k)
                            )
                            .properties("age", p -> p
                                .integer(i -> i)
                            )
                            .properties("address", p -> p
                                .text(t -> t
                                    .analyzer("ik_smart")
                                )
                            )
                        )
                    );
                
                log.info("用户索引创建成功: {}", response.index());
            }
        } catch (IOException e) {
            log.error("创建索引失败", e);
        }
    }
}

查询性能优化

@Service
public class QueryOptimizationService {
    
    /**
     * 使用过滤器上下文提升性能
     */
    public SearchResponse<UserDocument> searchWithFilterContext(UserSearchRequest request) {
        try {
            return elasticsearchClient.search(s -> s
                .index("user_index")
                .query(q -> q
                    .bool(b -> b
                        .must(m -> m
                            .multiMatch(mm -> mm
                                .query(request.getKeyword())
                                .fields("username", "address")
                            )
                        )
                        .filter(f -> f
                            .range(r -> r
                                .field("age")
                                .gte(JsonData.of(request.getMinAge()))
                                .lte(JsonData.of(request.getMaxAge()))
                            )
                        )
                    )
                )
                // 使用文档值字段提升排序性能
                .docvalueFields("age", "createTime")
                // 使用查询结果缓存
                .requestCache(true), UserDocument.class);
        } catch (IOException e) {
            throw new RuntimeException("搜索失败", e);
        }
    }
    
    /**
     * 批量操作提升性能
     */
    public void bulkIndexUsers(List<UserDocument> users) {
        try {
            BulkRequest.Builder br = new BulkRequest.Builder();
            
            for (UserDocument user : users) {
                br.operations(op -> op
                    .index(idx -> idx
                        .index("user_index")
                        .id(user.getId().toString())
                        .document(user)
                    )
                );
            }
            
            BulkResponse response = elasticsearchClient.bulk(br.build());
            
            if (response.errors()) {
                log.error("批量索引存在错误: {}", response.items());
            }
        } catch (IOException e) {
            throw new RuntimeException("批量索引失败", e);
        }
    }
}

监控与故障处理

健康检查与监控

【免费下载链接】mybatis-plus An powerful enhanced toolkit of MyBatis for simplify development 项目地址: https://gitcode.com/gh_mirrors/my/mybatis-plus