Hive动态分区参数配置

最新推荐文章于 2025-02-16 12:19:01 发布

Lemon.Warn

最新推荐文章于 2025-02-16 12:19:01 发布

阅读量440

点赞数

分类专栏：大数据文章标签：大数据 hive spark

本文链接：https://blog.youkuaiyun.com/Fread325/article/details/112183298

版权

大数据专栏收录该内容

14 篇文章

订阅专栏

Caused by: org.apache.hadoop.hive.ql.metadata.HiveFatalException: [Error 20004]: Fatal error occurred when node tried to create too many dynamic partitions. The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to: 5000

set hive.exec.dynamic.partition=true;  
set hive.exec.dynamic.partition.mode=nonstrict;  
  
-- 特别大时，执行以下参数设置
set hive.exec.dynamic.partitions.pernode=1024;  
set hive.exec.dynamic.partitions.partitions=2048;  
set hive.exec.max.created.files=20000;