Elasticsearch 7.X-8.0 AggregationBuliders 相关聚合函数(二)桶聚合

本文详细介绍了Elasticsearch 7.x-8.0中桶聚合的使用,包括filter聚合对结果进行条件过滤,filters聚合实现多条件桶过滤,adjacencyMatrix聚合形成邻接矩阵,sampler聚合对高得分文档采样,以及diversifiedSampler聚合确保样本多样性的功能。通过实例展示了各聚合在大数据分析中的应用场景和效果。

二、查询相关,返回桶

  • filter

    • 单个的桶聚合,将结果进行条件过滤
    • 请求示例,过滤商品为短袖的类型,再进行价格平均值计算:
    • POST /sales/_search?size=0&filter_path=aggregations
      {
        "aggs": {
          "avg_price": { "avg": { "field": "price" } },
          "t_shirts": {
            "filter": { "term": { "type": "t-shirt" } },
            "aggs": {
              "avg_price": { "avg": { "field": "price" } }
            }
          }
        }
      }

      返回结果:

    • 所有商品的平均价格为140.71428...,其中短袖的总共3件,平均价格为128.33333...

    • {
        "aggregations": {
          "avg_price": { "value": 140.71428571428572 },
          "t_shirts": {
            "doc_count": 3,
            "avg_price": { "value": 128.33333333333334 }
          }
        }
      }

  • filters

    • 多个桶聚合过滤,支持同时过滤不同条件的不同桶返回
    • 请求示例:
    • 查询errors桶,为日志中含有error错误信息的;
    • 查询warnings桶,查询日志中含warning信息的;
    • GET logs/_search
      {
        "size": 0,
        "aggs" : {
          "messa
<think>我们遇到了一个关于Elasticsearch Java客户端的问题:代码中使用的RestHighLevelClient在较新版本的Elasticsearch中已被弃用,并在8.x版本中被移除。因此,我们需要将代码迁移到新的Java客户端API。 ### 问题分析 Elasticsearch7.15.0版本开始标记RestHighLevelClient为弃用,并在8.0版本中完全移除。官方推荐使用新的Java客户端,该客户端具有以下特点: 1.Elasticsearch服务器版本兼容(当前最新为8.x) 2. 更简洁的API设计 3. 基于Java 8函数式编程风格 4. 支持异步和同步操作 ### 解决方案:迁移到新的Elasticsearch Java客户端 我们将使用Elasticsearch官方的新Java客户端(`elasticsearch-java`)重写示例代码。 #### 步骤1:添加Maven依赖 ```xml <dependency> <groupId>co.elastic.clients</groupId> <artifactId>elasticsearch-java</artifactId> <version>8.12.2</version> </dependency> <!-- 还需要Jackson依赖进行JSON处理 --> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.15.3</version> </dependency> ``` #### 步骤2:创建客户端 新的客户端使用`RestClient`作为底层HTTP通信,然后构建`ElasticsearchClient`。 #### 步骤3:重写操作 新客户端的API与之前的RestHighLevelClient有较大不同,需要重新编写索引、搜索等操作。 ### 完整代码示例(使用新Java客户端) ```java import co.elastic.clients.elasticsearch.ElasticsearchClient; import co.elastic.clients.elasticsearch.core.*; import co.elastic.clients.elasticsearch.core.search.Hit; import co.elastic.clients.json.JsonData; import co.elastic.clients.json.jackson.JacksonJsonpMapper; import co.elastic.clients.transport.ElasticsearchTransport; import co.elastic.clients.transport.rest_client.RestClientTransport; import org.apache.http.HttpHost; import org.apache.http.auth.AuthScope; import org.apache.http.auth.UsernamePasswordCredentials; import org.apache.http.client.CredentialsProvider; import org.apache.http.impl.client.BasicCredentialsProvider; import org.elasticsearch.client.RestClient; import java.io.IOException; import java.util.Collections; import java.util.List; public class NewElasticsearchDemo { public static void main(String[] args) throws IOException { // 创建客户端 ElasticsearchClient client = createClient(); // 索引名称和文档ID String indexName = "new_java_index"; String docId = "101"; // 1. 索引文档 System.out.println("===== 索引文档 ====="); Product product = new Product("Coffee Maker", 89.99, 15); IndexResponse indexResponse = indexDocument(client, indexName, docId, product); System.out.println("索引结果: " + indexResponse.result().toString()); System.out.println("文档ID: " + indexResponse.id()); // 2. 获取文档 System.out.println("\n===== 获取文档 ====="); Product getProduct = getDocument(client, indexName, docId); if (getProduct != null) { System.out.println("获取到的产品: " + getProduct); } else { System.out.println("文档不存在"); } // 3. 搜索文档 System.out.println("\n===== 搜索文档 ====="); List<Product> products = searchProducts(client, indexName, "Coffee"); System.out.println("搜索到 " + products.size() + " 个产品"); for (Product p : products) { System.out.println(p); } // 4. 更新文档(增加库存) System.out.println("\n===== 更新文档 ====="); updateDocument(client, indexName, docId, "stock", 20); getProduct = getDocument(client, indexName, docId); System.out.println("更新后的产品: " + getProduct); // 5. 删除文档 System.out.println("\n===== 删除文档 ====="); DeleteResponse deleteResponse = deleteDocument(client, indexName, docId); System.out.println("删除结果: " + deleteResponse.result().toString()); getProduct = getDocument(client, indexName, docId); if (getProduct == null) { System.out.println("文档已成功删除"); } client.shutdown(); } // 创建新的Elasticsearch客户端 private static ElasticsearchClient createClient() { // 配置认证信息(如果需要) final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "password")); // 替换为实际凭证 // 创建底层RestClient RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)) .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder .setDefaultCredentialsProvider(credentialsProvider)) .build(); // 使用Jackson映射器创建Transport ElasticsearchTransport transport = new RestClientTransport( restClient, new JacksonJsonpMapper() ); // 创建API客户端 return new ElasticsearchClient(transport); } // 索引文档 private static IndexResponse indexDocument(ElasticsearchClient client, String index, String id, Product product) throws IOException { IndexRequest<Product> request = IndexRequest.of(b -> b .index(index) .id(id) .document(product) ); return client.index(request); } // 获取文档 private static Product getDocument(ElasticsearchClient client, String index, String id) throws IOException { GetRequest request = GetRequest.of(b -> b.index(index).id(id)); GetResponse<Product> response = client.get(request, Product.class); if (response.found()) { return response.source(); } else { return null; } } // 搜索文档 private static List<Product> searchProducts(ElasticsearchClient client, String index, String keyword) throws IOException { SearchRequest request = SearchRequest.of(b -> b .index(index) .query(q -> q .match(m -> m .field("name") .query(keyword) ) ) ); SearchResponse<Product> response = client.search(request, Product.class); return response.hits().hits().stream() .map(Hit::source) .toList(); } // 更新文档(部分更新) private static void updateDocument(ElasticsearchClient client, String index, String id, String field, Object value) throws IOException { UpdateRequest<Product, Object> request = UpdateRequest.of(b -> b .index(index) .id(id) .doc(Collections.singletonMap(field, value)) ); client.update(request, Object.class); } // 删除文档 private static DeleteResponse deleteDocument(ElasticsearchClient client, String index, String id) throws IOException { DeleteRequest request = DeleteRequest.of(b -> b.index(index).id(id)); return client.delete(request); } // 产品类(用于文档映射) static class Product { private String name; private double price; private int stock; public Product() { // 无参构造函数用于反序列化 } public Product(String name, double price, int stock) { this.name = name; this.price = price; this.stock = stock; } // getters and setters public String getName() { return name; } public void setName(String name) { this.name = name; } public double getPrice() { return price; } public void setPrice(double price) { this.price = price; } public int getStock() { return stock; } public void setStock(int stock) { this.stock = stock; } @Override public String toString() { return "Product{" + "name='" + name + '\'' + ", price=" + price + ", stock=" + stock + '}'; } } } ``` ### 代码说明 1. **客户端创建**:使用新的`ElasticsearchClient`替代`RestHighLevelClient`。 2. **文档操作**: - 索引文档:使用`IndexRequest`构建请求,通过`client.index()`发送。 - 获取文档:使用`GetRequest`,并通过指定文档类(如`Product.class`)进行反序列化。 - 搜索文档:使用`SearchRequest`构建查询,返回的搜索结果可以通过`Hit.source()`转换为对象。 - 更新文档:使用`UpdateRequest`进行部分更新(传递一个Map)。 - 删除文档:使用`DeleteRequest`。 3. **POJO映射**:使用Jackson将JSON文档映射到Java对象(`Product`类),需要无参构造函数和getter/setter。 ### 迁移注意事项 1. **版本兼容性**:确保客户端版本与Elasticsearch服务器版本一致(主版本号相同)。 2. **异常处理**:新API可能会抛出`ElasticsearchException`,需要适当处理。 3. **异步操作**:新客户端支持异步操作(如`client.indexAsync()`),可提高性能。 ### 在线运行说明 此代码可以在支持Java 8+和Maven的在线环境中运行(如Replit)。需要设置正确的Elasticsearch服务器地址和认证信息。 --- §§相关问题§§: 1. 如何在新的Elasticsearch Java客户端中执行批量操作(Bulk)? 2. 使用新客户端时如何处理连接池和重试策略? 3. 如何配置新客户端的SSL/TLS加密连接? 4. 在新客户端中如何实现聚合查询(Aggregations)?</think>### 解决 Elasticsearch RestHighLevelClient 弃用问题 Elasticsearch8.0 版本中移除了 `RestHighLevelClient`,改用新的 Java API 客户端。以下是完全兼容 Elasticsearch 8.x 的解决方案: #### 完整解决方案代码 ```java import co.elastic.clients.elasticsearch.ElasticsearchClient; import co.elastic.clients.elasticsearch.core.*; import co.elastic.clients.elasticsearch.core.search.Hit; import co.elastic.clients.json.JsonData; import co.elastic.clients.json.jackson.JacksonJsonpMapper; import co.elastic.clients.transport.ElasticsearchTransport; import co.elastic.clients.transport.rest_client.RestClientTransport; import org.apache.http.HttpHost; import org.apache.http.auth.AuthScope; import org.apache.http.auth.UsernamePasswordCredentials; import org.apache.http.client.CredentialsProvider; import org.apache.http.impl.client.BasicCredentialsProvider; import org.elasticsearch.client.RestClient; import java.io.IOException; import java.util.List; import java.util.Map; public class ElasticsearchJavaClientV8 { public static void main(String[] args) { // 1. 创建客户端 try (ElasticsearchClient client = createClient()) { String indexName = "java_products"; String docId = "p1001"; // 2. 创建索引文档 System.out.println("===== 创建文档 ====="); Map<String, Object> product = Map.of( "name", "Java Programming Book", "price", 49.99, "stock", 100, "tags", List.of("java", "programming", "book") ); indexDocument(client, indexName, docId, product); // 3. 获取文档 System.out.println("\n===== 获取文档 ====="); getDocument(client, indexName, docId); // 4. 搜索文档 System.out.println("\n===== 搜索文档 ====="); searchDocuments(client, indexName, "java"); // 5. 更新文档 System.out.println("\n===== 更新文档 ====="); updateDocument(client, indexName, docId, "stock", 95); getDocument(client, indexName, docId); // 验证更新 // 6. 删除文档 System.out.println("\n===== 删除文档 ====="); deleteDocument(client, indexName, docId); getDocument(client, indexName, docId); // 验证删除 } catch (Exception e) { e.printStackTrace(); } } // 创建 Elasticsearch 8.x 客户端 private static ElasticsearchClient createClient() { // 配置认证信息 final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "your_password")); // 替换为实际凭证 // 创建底层 REST 客户端 RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200)) .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)) .build(); // 使用 Jackson 映射器创建传输层 ElasticsearchTransport transport = new RestClientTransport( restClient, new JacksonJsonpMapper()); return new ElasticsearchClient(transport); } // 索引文档 private static void indexDocument(ElasticsearchClient client, String index, String id, Map<String, Object> source) throws IOException { IndexRequest<Map<String, Object>> request = IndexRequest.of(b -> b .index(index) .id(id) .document(source) ); IndexResponse response = client.index(request); System.out.println("结果: " + response.result()); System.out.println("文档ID: " + response.id()); } // 获取文档 private static void getDocument(ElasticsearchClient client, String index, String id) throws IOException { GetRequest request = GetRequest.of(b -> b .index(index) .id(id) ); GetResponse<Map> response = client.get(request, Map.class); if (response.found()) { System.out.println("文档内容: " + response.source()); System.out.println("版本: " + response.version()); } else { System.out.println("文档不存在"); } } // 搜索文档 private static void searchDocuments(ElasticsearchClient client, String index, String query) throws IOException { SearchRequest request = SearchRequest.of(b -> b .index(index) .query(q -> q .match(m -> m .field("name") .query(query) ) ) ); SearchResponse<Map> response = client.search(request, Map.class); System.out.println("命中数量: " + response.hits().total().value()); for (Hit<Map> hit : response.hits().hits()) { System.out.println("ID: " + hit.id() + " | 分数: " + hit.score()); System.out.println("内容: " + hit.source()); } } // 更新文档 private static void updateDocument(ElasticsearchClient client, String index, String id, String field, Object value) throws IOException { UpdateRequest<Map, Object> request = UpdateRequest.of(b -> b .index(index) .id(id) .doc(Map.of(field, value)) ); UpdateResponse<Map> response = client.update(request, Map.class); System.out.println("结果: " + response.result()); System.out.println("新版本: " + response.version()); } // 删除文档 private static void deleteDocument(ElasticsearchClient client, String index, String id) throws IOException { DeleteRequest request = DeleteRequest.of(b -> b .index(index) .id(id) ); DeleteResponse response = client.delete(request); System.out.println("结果: " + response.result()); } } ``` ### 迁移说明和关键变化 #### 1. 客户端创建方式变化 ```java // 旧方式 (已弃用) RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200))); // 新方式 RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build(); ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper()); ElasticsearchClient client = new ElasticsearchClient(transport); ``` #### 2. API 设计变化 - **强类型 DSL**:使用 builder 模式创建请求 - **函数式风格**:通过 lambda 表达式配置请求参数 - **响应对象简化**:直接返回操作结果而非 HTTP 状态 #### 3. Maven 依赖变更 ```xml <dependencies> <!-- Elasticsearch Java API 客户端 --> <dependency> <groupId>co.elastic.clients</groupId> <artifactId>elasticsearch-java</artifactId> <version>8.12.2</version> </dependency> <!-- 底层 REST 客户端 --> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-client</artifactId> <version>8.12.2</version> </dependency> <!-- JSON 处理 --> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.15.3</version> </dependency> </dependencies> ``` ### 新旧 API 对比表 | 操作 | 旧 API (RestHighLevelClient) | 新 API (ElasticsearchClient) | |------|-----------------------------|------------------------------| | **创建客户端** | `new RestHighLevelClient(...)` | 通过 `RestClientTransport` 创建 | | **索引文档** | `IndexRequest` + `client.index()` | `IndexRequest.of(b -> b...)` | | **获取文档** | `GetRequest` + `client.get()` | `GetRequest.of(b -> b...)` | | **搜索文档** | `SearchRequest` + `SearchSourceBuilder` | DSL builder + lambda 表达式 | | **更新文档** | `UpdateRequest` + `client.update()` | `UpdateRequest.of(b -> b...)` | | **删除文档** | `DeleteRequest` + `client.delete()` | `DeleteRequest.of(b -> b...)` | | **响应处理** | 手动解析 JSON/Map | 自动反序列化为 POJO 或 Map | ### 常见迁移问题解决 #### 问题1:依赖冲突 **解决方案**:排除旧依赖 ```xml <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.17.3</version> <exclusions> <exclusion> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </exclusion> </exclusions> </dependency> ``` #### 问题2:认证配置 **解决方案**:使用 `CredentialsProvider` ```java CredentialsProvider credsProvider = new BasicCredentialsProvider(); credsProvider.setCredentials( AuthScope.ANY, new UsernamePasswordCredentials("user", "password") ); RestClient restClient = RestClient.builder(new HttpHost("host", 9200)) .setHttpClientConfigCallback(hcb -> hcb.setDefaultCredentialsProvider(credsProvider)) .build(); ``` #### 问题3:SSL/TLS 配置 **解决方案**:配置 SSL 上下文 ```java SSLContext sslContext = SSLContextBuilder.create() .loadTrustMaterial(new TrustSelfSignedStrategy()) .build(); RestClient restClient = RestClient.builder(new HttpHost("host", 9200, "https")) .setHttpClientConfigCallback(hcb -> hcb.setSSLContext(sslContext)) .build(); ``` ### 在线测试说明 此代码可直接在支持 Java 17+ 的环境中运行: 1. 确保 Elasticsearch 8.x 服务运行 2. 更新 `createClient()` 中的连接信息 3. 添加上述 Maven 依赖 4. 运行程序查看完整文档生命周期操作 ---
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值