ElassticSearch对字段截取后再聚合

最新推荐文章于 2024-05-04 21:06:20 发布

渊云

最新推荐文章于 2024-05-04 21:06:20 发布

阅读量2.3k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： ElasticSearch 文章标签： elasticsearch 大数据数据库

本文链接：https://blog.youkuaiyun.com/qq_41878532/article/details/109239328

ElasticSearch 专栏收录该内容

13 篇文章

订阅专栏

本文探讨了在Elasticsearch 6.7中如何处理查询时使用字段值截取遇到的空值问题，并针对Date类型字段提供了解决方案。通过实例和问题分析，分享了如何避免非法操作并优化聚合查询的实践技巧。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

环境：ElasticSearch6.7
需求：利用ElasticSearch某个字段值的前几位进行聚合
Restful查询语句：

GET /ads_lading_trade_brief_es/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "country": "US"
          }
        }
      ]
    }
  },
  "aggs": {
    "company": {
      "terms": {
        "script": "doc['prod_desc'].value.substring(0,1)"
        
      }
    }
  }
}

出现的问题1：

{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:779)",
          "org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:116)",
          "org.elasticsearch.index.query.QueryShardContext.lambda$lookup$0(QueryShardContext.java:283)",
          "org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:88)",
          "org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:85)",
          "java.base/java.security.AccessController.doPrivileged(Native Method)",
          "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:85)",
          "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:39)",
          "doc['prod_desc'].value.substring(0,1)",
          "    ^---- HERE"
        ],
        "script": "doc['prod_desc'].value.substring(0,1)",
        "lang": "painless"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "ads_lading_trade_brief_es_02",
        "node": "fFDobwtSQvS1ishWYtWQcg",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "script_stack": [
            "org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:779)",
            "org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:116)",
            "org.elasticsearch.index.query.QueryShardContext.lambda$lookup$0(QueryShardContext.java:283)",
            "org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:88)",
            "org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:85)",
            "java.base/java.security.AccessController.doPrivileged(Native Method)",
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:85)",
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:39)",
            "doc['prod_desc'].value.substring(0,1)",
            "    ^---- HERE"
          ],
          "script": "doc['prod_desc'].value.substring(0,1)",
          "lang": "painless",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [prod_desc] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
          }
        }
      }
    ]
  },
  "status": 500
}

问题分析1：
这是因为这个截取的字段在有的记录中存在，在有的记录中不存在，即存在空值问题。
问题解决：
将空值字段用特使值表示，不再使用空值，目前本人没有找到好的方法（可能已有解决方法），如果有朋友有好的解决方法，还请留言告知下。

出现的问题2：

{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "doc['by_date'].value.asstring.substring(0,1)",
          "                    ^---- HERE"
        ],
        "script": "doc['by_date'].value.asstring.substring(0,1)",
        "lang": "painless"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "ads_lading_trade_brief_es_02",
        "node": "fFDobwtSQvS1ishWYtWQcg",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "script_stack": [
            "doc['by_date'].value.asstring.substring(0,1)",
            "                    ^---- HERE"
          ],
          "script": "doc['by_date'].value.asstring.substring(0,1)",
          "lang": "painless",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "dynamic getter [org.elasticsearch.script.JodaCompatibleZonedDateTime, asstring] not found"
          }
        }
      }
    ]
  },
  "status": 500
}