hbase rs启动的内存要求

最新推荐文章于 2022-09-23 14:53:19 发布

原创最新推荐文章于 2022-09-23 14:53:19 发布 · 182 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#hbase

hbase 专栏收录该内容

76 篇文章

订阅专栏

本文探讨了HBase集群中regionserver内存配置的关键参数，包括memstore和blockcache的上限限制，以及如何确保这些配置不会导致集群运行失败。

hbase rs启动的内存要求

1-hbase.regionserver.global.memstore.upperLimit-hfile.block.cache.size<=(HBASE_CLUSTER_MINIMUM_MEMORY_THRESHOLD = 0.2f)

也就是memstore和blockcache要小于等于heap的80%

private static void checkForClusterFreeMemoryLimit(Configuration conf) {
      float globalMemstoreLimit = conf.getFloat("hbase.regionserver.global.memstore.upperLimit", 0.4f);
      int gml = (int)(globalMemstoreLimit * CONVERT_TO_PERCENTAGE);
      float blockCacheUpperLimit =
        conf.getFloat(HConstants.HFILE_BLOCK_CACHE_SIZE_KEY,
          HConstants.HFILE_BLOCK_CACHE_SIZE_DEFAULT);
      int bcul = (int)(blockCacheUpperLimit * CONVERT_TO_PERCENTAGE);
      if (CONVERT_TO_PERCENTAGE - (gml + bcul)
              < (int)(CONVERT_TO_PERCENTAGE *
                      HConstants.HBASE_CLUSTER_MINIMUM_MEMORY_THRESHOLD)) {//1-0.4-0.4<0.2 设置的刚好在边缘，呵~
          throw new RuntimeException(
            "Current heap configuration for MemStore and BlockCache exceeds " +
            "the threshold required for successful cluster operation. " +
            "The combined value cannot exceed 0.8. Please check " +
            "the settings for hbase.regionserver.global.memstore.upperLimit and " +
            "hfile.block.cache.size in your configuration. " +
            "hbase.regionserver.global.memstore.upperLimit is " +
            globalMemstoreLimit +
            " hfile.block.cache.size is " + blockCacheUpperLimit);
      }
  }