hadoop 不使用hostname，使用ip遇到的问题（Datanode denied communication with namenode because hostname cann）及解决方案

最新推荐文章于 2025-06-05 14:17:07 发布

翻译最新推荐文章于 2025-06-05 14:17:07 发布 · 8k 阅读

hadoop 专栏收录该内容

0 篇文章

订阅专栏

本文详细解释了如何使用hosts.deny和hosts.allow文件来限制对Hadoop集群的访问，以及如何通过修改配置避免DNS解析问题，并介绍了如何通过配置避免数据节点注册检查以适应特定环境如AWS VPC。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

“Hadoop provides a mechanism to prevent unauthorized access. User can add hosts.deny and hosts.allow files to limit the access to a hadoop cluster. When a data node joins the name node, the name node will check the DNS name of the data node, if the DNS name cannot be resolved or the host is not allowed based on hosts.deny and hosts.allow files, the NN will reject the data node join.

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException): Datanode denied communication with namenode because hostname cannot be resolved (ip=10.5.x.x, hostname=10.5.x.x): DatanodeRegistration(0.0.0.0, datanodeUuie12e2d92-e215-4af9-a09b-d22ef4e3cc5ainfoPort=9102, ipcPort=9201, storageInfo=lv=-55;cid=CID-f50d3808-5d7f-4520-b696-0234f07f89f7;nsid=1758720717;c=0)

If set dfs.namenode.datanode.registration.ip-hostname-check=false, the name node will not do the check, which could be useful if your cluster is inside AWS VPC and you do not have proper reverse DNS.

Another usage of this feature is to decommission a data node by listing it in the hosts.deny file.”

也就是说在namenode的hdfs-site.xml 里面添加

<property>
<name>dfs.namenode.datanode.registration.ip-hostname-check</name>
<value>false</value>
</property>

即可