analyze table {table} compute statistics 批量更新Hive元数据,老是报错 return code 1 / Cannot run program “chmod”: error=24, Too many open files
具体报错如下
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. java.util.concurrent.ExecutionException: java.io.IOException: Cannot run program “chmod”: error=24, Too many open files
搞了好几天,map,yarn,hive配置设置了个遍,还没解决。之前不知道怎么设置了打开文件限制,平时运行没有问题,一次数据迁移,批量更新元数据就报错啦。
具体解决办法如下:
1,vim /etc/sysctl.conf
fs.nr_open=100000
fs.file-max=100000
2,vim /etc/my.cnf
open_files_limit=65536
然后重启服务器,或者重启MySQL和hdfs,hive
可以参考:
https://www.cnblogs.com/ivictor/p/14900930.html