HDFS集群机架感知配置

最新推荐文章于 2024-05-07 09:48:41 发布

原创最新推荐文章于 2024-05-07 09:48:41 发布 · 517 阅读

1 ·

CC 4.0 BY-SA版权

进入配置文件目录

cd /root/apps/hadoop-2.7.5/etc/hadoop

1、修改配置文件core-site.xml
vim core-site.xml
<configuration>
<property>
<name>topology.script.file.name</name>
<value>/root/apps/hadoop-2.7.5/etc/hadoop/topology.sh</value>
</property>
</configuration>

2、创建topology.sh
vim topology.sh
#!/bin/bash
HADOOP_CONF=/root/apps/hadoop-2.7.5/etc/hadoop
while [ $# -gt 0 ] ;
do
nodeArg=$1
exec<${HADOOP_CONF}/topology.data
result=""
while read line
do
ar=( $line )
if [ "${ar[0]}" = "$nodeArg" ]||[ "${ar[1]}" = "$nodeArg" ]
then
result="${ar[2]}"
fi
done
shift
if [ -z "$result" ]
then
echo -n "/default-rack"
else
echo -n "$result"
fi
done

3、创建topology.data
vim topology.data
192.168.123.101 hadoop01 /switch1/rack1
192.168.123.102 hadoop02 /switch1/rack1
192.168.123.103 hadoop03 /switch2/rack2
192.168.123.104 hadoop04 /switch2/rack2

4、修改文件权限
chmod 777 topology.sh topology.data

5、验证机架感知

hdfs dfsadmin -printTopology