hdfs client access the hdfs cluster not in one domain

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html#Clients_use_Hostnames_when_connecting_to_DataNodes

sometimes when you access your hadoop cluster outside the internal cluster, you maybe encounter some route problem.

1, how to expose you hdfs service on internet?

the hdfs client communicate the master( if the master can access) ,but the master return datanodes's internal ip or hostname,

the client can not do the correct name resolution for the host name or can not access the internal ip.

for the name resolution, if the data node ip can access, you should set the parameter

dfs.client.use.datanode.hostname

false

Whether clients should use datanode hostnames when connecting to datanodes.

2.if you use python to access hdfs, you should pay attention to another configure item.

  dfs.webhdfs.enabled  -> true

because python hdfs module use webhdfs to access hadoop by default.

转载于:https://www.cnblogs.com/huaxiaoyao/p/6007795.html