ES-aggregation聚合分析

最新推荐文章于 2025-06-07 15:21:14 发布

冲上云霄的Jayden

最新推荐文章于 2025-06-07 15:21:14 发布

阅读量625

点赞数

CC 4.0 BY-SA版权

分类专栏： ES 文章标签： es aggregation 聚合分析

本文链接：https://blog.youkuaiyun.com/NF_ALONG/article/details/124308932

ES 专栏收录该内容

19 篇文章

订阅专栏

本文解析了如何使用Kibana的Aggregation功能，如BucketAggregation、MetricAggregation和PipelineAggregation，来从飞行数据中获取目的地国家的统计信息，包括平均票价、最高最低价格和天气分布。通过实例展示了如何计算航班数据的集中趋势和多样性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

集合的分类

Bucket Aggregation

一些列满足特定条件的文档集。

Metric Aggregation

一些数学运算，可以对文档字段进行统计分析。

Metric会基于数据集计算结果，除了支持在字段上进行计算，同样也支持在脚步（painless script）产生的结果之上进行计算。

大多数Metric是数学计算，仅输出一个值。
min / max / sum / avg / cardinality

部分Metric支持输出多个数值。
stats / percentiles /percentile_ranks

实战：查询航班目的地/出发地的统计信息、机票平均价、最贵机票价、最便宜机票价

get kibana_sample_data_flights/_mappings
# 输出
{
  "kibana_sample_data_flights" : {
    "mappings" : {
      "properties" : {
        "AvgTicketPrice" : {
          "type" : "float"
        },
        "Cancelled" : {
          "type" : "boolean"
        },
        "Carrier" : {
          "type" : "keyword"
        },
        "Dest" : {
          "type" : "keyword"
        },
        "DestAirportID" : {
          "type" : "keyword"
        },
        "DestCityName" : {
          "type" : "keyword"
        },
        "DestCountry" : {
          "type" : "keyword"
        },
        "DestLocation" : {
          "type" : "geo_point"
        },
        "DestRegion" : {
          "type" : "keyword"
        },
        "DestWeather" : {
          "type" : "keyword"
        },
        "DistanceKilometers" : {
          "type" : "float"
        },
        "DistanceMiles" : {
          "type" : "float"
        },
        "FlightDelay" : {
          "type" : "boolean"
        },
        "FlightDelayMin" : {
          "type" : "integer"
        },
        "FlightDelayType" : {
          "type" : "keyword"
        },
        "FlightNum" : {
          "type" : "keyword"
        },
        "FlightTimeHour" : {
          "type" : "keyword"
        },
        "FlightTimeMin" : {
          "type" : "float"
        },
        "Origin" : {
          "type" : "keyword"
        },
        "OriginAirportID" : {
          "type" : "keyword"
        },
        "OriginCityName" : {
          "type" : "keyword"
        },
        "OriginCountry" : {
          "type" : "keyword"
        },
        "OriginLocation" : {
          "type" : "geo_point"
        },
        "OriginRegion" : {
          "type" : "keyword"
        },
        "OriginWeather" : {
          "type" : "keyword"
        },
        "dayOfWeek" : {
          "type" : "integer"
        },
        "timestamp" : {
          "type" : "date"
        }
      }
    }
  }
}

get kibana_sample_data_flights/_search
{
  "size":0,
  "aggs":{
    "flight_dest":{
      //类似 select DestCountry count(*) from x group by DestCountry
      "terms":{
        "field": "DestCountry"
      },
      //aggs 开启统计
      "aggs": {
        // average_price 存储统计结果名称
        "average_price": {
          //avg 为统计函数
          "avg": {
            //根据AvgTicketPrice字段进行统计
            "field": "AvgTicketPrice"
          }
        },
        "max_price":{
          "max": {
            "field": "AvgTicketPrice"
          }
        },
        "min_price":{
          "min": {
            "field": "AvgTicketPrice"
          }
        },
        "stat_price":{
          //stats 包含min/max/avg/sum统计
          "stats": {
            "field": "AvgTicketPrice"
          }
        },
        "weather":{
          "terms": {
            "field": "DestWeather"
          }
        }
      }
    }
  }
}

结果输出：

{
  "took" : 18,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "flight_dest" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 3187,
      "buckets" : [
        {
          "key" : "IT",
          "doc_count" : 2371,
          "max_price" : {
            "value" : 1195.3363037109375
          },
          "min_price" : {
            "value" : 100.57646942138672
          },
          "weather" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "Clear",
                "doc_count" : 428
              },
              {
                "key" : "Sunny",
                "doc_count" : 424
              },
              {
                "key" : "Rain",
                "doc_count" : 417
              },
              {
                "key" : "Cloudy",
                "doc_count" : 414
              },
              {
                "key" : "Heavy Fog",
                "doc_count" : 182
              },
              {
                "key" : "Damaging Wind",
                "doc_count" : 173
              },
              {
                "key" : "Hail",
                "doc_count" : 169
              },
              {
                "key" : "Thunder & Lightning",
                "doc_count" : 164
              }
            ]
          },
          "average_price" : {
            "value" : 586.9627099618385
          },
          "stat_price" : {
            "count" : 2371,
            "min" : 100.57646942138672,
            "max" : 1195.3363037109375,
            "avg" : 586.9627099618385,
            "sum" : 1391688.585319519
          }
        }
      ]
    }
  }
}