关于fielddata数据占用内存过大的解决方法

关注

原创最新推荐文章于 2026-07-29 11:12:50 发布 1w 阅读 AI 写同款· GEO 优化 ›

【Elasticsearch】es fielddata 字段数据在es中，text类型的字段使用一种叫做fielddata的查询时内存数据结构。当字段被排序，聚合或者通过脚本访问时这种数据结构会被创建。它是通过从磁盘读取每个段的整个反向索引来构建的，然后存存储在java的堆内存中。Fielddata针对text字段在默认时是禁用的，Fielddata会占用大量堆空间，尤其是在加载大量的文本字段时。一旦将字段数据加载到堆中，它在该段的生命周期内将一直保留在那里。同样，加载字段数据是一个昂贵的过程，可能导致用户遇到延迟的情况。这就是默认情况下禁用字段数据的原因。阅读详情

参考文章

Support in the Wild: My Biggest Elasticsearch Problem at Scale

http://blog.youkuaiyun.com/jiao_fuyou/article/details/50478198

Understanding Fielddata

By default, fielddata is loaded on demand, which means that you will not see it until you are using it. Also, by being loaded per segment, it means that new segments that get created will slowly add to your overall memory usage until the field’s fielddata is evicted from memory. Eviction happens in only a few ways:

1、Deleting the index or indices that contains it.
2、Closing the index or indices that contains it.
3、Segment fielddata is removed when segments are removed (e.g., background merging).
* This usually just means that the problem is moving rather than going away.

4、Restarting the node containing the fielddata.
5、Clearing the relevant fielddata cache.
6、Automatically evicting the fielddata to make room for other fielddata.
* This will not happen with default settings.

Use Doc Values

The solution to this fielddata problem is to avoid it altogether. Fortunately, you can avoid the use of fielddata bymanually mapping all of your fields to use doc values. Without repeating too much from the guide, doc values offload this burden by writing the fielddata to disk at index time, thereby allowing Elasticsearch to load the values outside of your Java heap as they are needed.

You might be asking: How do I enable these doc values? It’s as simple as adding this to every field’s mapping, exceptanalyzed string fields:

"doc_values" : true

Unfortunately, you must do this before you index any data. This means that, for any existing index that is not already using doc values, you cannot flip the switch to enable them. You would have to reindex.

控制fielddata允许内存大小

控制fielddata允许内存大小，达到HEAP 40% 自动清理旧cache

curl -XPUT http://localhost:9200/_cluster/settings
{
    "persistent" : {
        "indices.fielddata.cache.size" : "40%"
    }
}

如果旧缓存不需要，如日志，只查询最新一天数据，那可以强制清除旧缓存：

POST /_cache/clear

开启字段doc_values

对fielddata战用很大内存的字段，设置：”doc_values” : true，ES2.0以后，该属性默认开启。

PUT /music/_mapping/song
{
  "properties" : {
    "tag": {
      "type":       "string",
      "index" :     "not_analyzed",
      "doc_values": true 
    }
  }
}