influxdb {"error":"partial write: max-values-per-tag limit exceeded (100009/100000)

最新推荐文章于 2024-06-19 09:22:40 发布

原创最新推荐文章于 2024-06-19 09:22:40 发布 · 1w 阅读

4 ·

CC 4.0 BY-SA版权

文章标签：

#influxdb

InfluxDB 专栏收录该内容

3 篇文章

订阅专栏

本文探讨了InfluxDB中关于tag值数量限制的问题，解释了为何设置此限制及其潜在影响，并提供了如何调整该限制的方法。

部署运行你感兴趣的模型镜像

在用influxdb写数据的时候，突然出现如下的错误，我用的是到目前最新的版本1.2：

java.lang.RuntimeException: {"error":"partial write: max-values-per-tag limit exceeded (100009/100000): measurement=\"t1\" tag=\"openId\" value=\"oGeG_0emqmIf5xakHAZq5_NEbcJ0\"dropped=4"}

        at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:466)
        at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:267)
        at com.ihangmei.datapro.consumer.influxdb.InfluxdbInstance.writeInfluxdb(InfluxdbInstance.java:97)
        at com.ihangmei.datapro.consumer.kafka.WechatRunnable.run(WechatRunnable.java:50)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.RuntimeException: {"error":"partial write: max-values-per-tag limit exceeded (100009/100000): measurement=\"t1\" tag=\"openId\" value=\"oGeG_0RngX2VL7c_wmzDNyacMQHI\"dropped=4"}

        at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:466)
        at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:267)
        at com.ihangmei.datapro.consumer.influxdb.InfluxdbInstance.writeInfluxdb(InfluxdbInstance.java:97)
        at com.ihangmei.datapro.consumer.kafka.WechatRunnable.run(WechatRunnable.java:50)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

在官方文档中是这么说的：

max-values-per-tag = 100000

The maximum number of tag values allowed per tag key. The default setting is 100000. Change the setting to 0 to allow an unlimited number of tag values per tag key. If a tag value causes the number of tag values of a tag key to exceed max-values-per-tag InfluxDB will not write the point, and it returns a partial write error.

Any existing tag keys with tag values that exceed max-values-per-tag will continue to accept writes, but writes that create a new tag value will fail.

Environment variable: INFLUXDB_DATA_MAX_VALUES_PER_TAG

默认情况下每个tag key只允许最多有10w个tag values，如果超过了10w个就会出现如上的错误，当然在另一篇文章中，我们找到个这么做的原因：

influxdb增加了对tag的值的大小的校验，最大值是10000.若写入的数据的tag的值超限，则调用方会收到报错。用来防止写入到measurement的数据的series-cardinality(series-cardinality)过大。如果我们写入了大量的series-cardinality很高的数据，那么当我们删除数据的时候，InfluxDB会OOM。

原因是InfluxDB在内存中维护了系统中每个series数据的索引。随着具有唯一性的series数据数量的增长，RAM的使用也会增长。过高的series cardinality会导致操作系统kill掉InfluxDB进程，抛出OOM异常。

参考：https://docs.influxdata.com/influxdb/v1.2/administration/config#max-values-per-tag-100000

http://xiajunhust.github.io/2016/12/13/InfluxDB%E5%86%99%E5%85%A5%E5%A4%B1%E8%B4%A5-partial-write-max-values-per-tag-limit-exceeded/

您可能感兴趣的与本文相关的镜像