add new node in hadoop without restarting the cluster

hegang1314

于 2013-05-23 10:35:26 发布

阅读量921

点赞数

分类专栏：问题总结

本文链接：https://blog.youkuaiyun.com/hero_hegang/article/details/8966687

版权

问题总结专栏收录该内容

26 篇文章

订阅专栏

本文详细介绍了在Hadoop集群中配置并启动新节点的过程，包括修改主机名、hosts文件、slaves文件，分布式配置文件传输，以及新节点的启动步骤。确保集群平衡利用磁盘资源。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

For the new node
step 1:modify /etc/hostname
slave0X
=============================================
For the master node
step1:modify /etc/hosts
ip masters
ip slave01
ip slave02
...
ip slave0X

step2:modify the configure file : conf/slaves
slave01
slave02
...
slave0X
==============================================

Distribute the configure files of the master to other slave nodes.
run batch file "batch" in hadoop-1.0.3 directory with hadoop user.
$ su hadoop
$ cd ~
$ cd hadoop/hadoop-1.0.3
$ chown -R hadoop:hadoop /home/batch
$ ./home/batch

#The contents of the batch file are as follows:

scp conf/slaves hadoop@slave0x:hadoop-1.0.3/conf/

scp /etc/hosts root@slave0x:/etc/hosts

==============================================

Start the new or broken node in the new node.

bin/Hadoop-daemon.sh start datanode
bin/Hadoop-daemon.sh start tasktracker

Balancing the utilization of the disks in the master node.

bin/start-balancer.sh -threshold 0.1

!attention the default threshold value is 0.1, and the lower the more balance, but the more time will be used.