CentOS6.5基于Hadoop用户的HDFS伪分布式部署(a Single Node Cluster)

前言:
  上一篇博客HDFS伪分布式部署基于管理员root部署成功，但是实际生产中使用 Hadoop 用户操作及配置
 官网参考：http://hadoop.apache.org/docs/r2.8.4/hadoop-project-dist/hadoop-  common/SingleCluster.html

1、检查hadoop用户是否存在

1. [root@hadoop001 ~]# id hadoop
   
2. uid=516(hadoop) gid=516(hadoop) groups=516(hadoop)

2、创建hadoop用户

1. [root@hadoop001 ~]# useradd hadoop
   
2. [root@hadoop001 ~]# passwd hadoop
   
3. Changing password for user hadoop.
   
4. New password:

3、停止hadoop服务及权限

1. [root@hadoop001 ~]# kill -9 $(pgrep -f hadoop)
   
2. [root@hadoop001 ~]# jps
3. 15803 Jps
4. [root@hadoop001 software]# pwd
   /opt/software
   
5. [root@hadoop001 software]# chown -R hadoop:hadoop hadoop-2.8.1

4、配置hadoop用户SSH互信
# 设置hadoop用户shh互信（新用户需要重新设置ssh）
# 其他用户启动hadoop，authorized_keys必须授权0600 

1. [root@hadoop001 .ssh]# su - hadoop
   
2. [hadoop@hadoop001 ~]$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
   
3. Generating public/private rsa key pair.
   
4. Your identification has been saved in /home/hadoop/.ssh/id_rsa.
   
5. Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
   
6. The key fingerprint is:
   
7. 68:5e:33:dd:2a:d2:01:e7:4e:4e:7b:4a:72:7d:67:83 hadoop@hadoop001
   
8. The key's randomart image is:
   +--[ RSA 2048]----+
   |                 |
   |                 |
   |      . .        |
   |       = . .     |
   |      o S . .    |
   |     o B * . .   |
   |      + O + E +  |
   |       = + . o . |
   |        .        |
   +-----------------+
   [hadoop@hadoop001 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
   [hadoop@hadoop001 ~]$ chmod 0600 ~/.ssh/authorized_keys
   
   #检查通过hadoop用户互信
   [hadoop@hadoop001 ~]$ chmod 0600 ~/.ssh/authorized_keys
   [hadoop@hadoop001 ~]$ ssh -p2222 localhost date      --ssh端口为2222
   Fri May 18 17:32:59 EDT 2018
9. 
   

4、修改HDFS三大进程对外服务IP
#读写都是通过NameNode名称节点访问IP

1. [root@hadoop001 hadoop]# pwd
   
2. /opt/software/hadoop-2.8.1/etc/hadoop
   
3. 
   
4. [root@hadoop001 hadoop]# vim core-site.xml
   
5. --添加<property>,localhost修改为IP
   
6. <configuration>
7.     <property>
   
8.         <name>fs.defaultFS</name>
   
9.         <value>hdfs://localhost:9000</value>    
   
10.     </property>
11. </configuration>

#修改DataNode数据节点访问IP

1. [root@hadoop001 hadoop]# pwd
   
2. /opt/software/hadoop-2.8.1/etc/hadoop
   
3. 
   
4. [root@hadoop001 hadoop]# vim slaves
   
5. localhost --localhost修改为IP,多DN逗号分隔ip  

#修改SecondayNameNode数据节点访问IP

1. [root@hadoop001 hadoop]# vim hdfs-site.xml
   
2. -- 添加<property>
   
3.     <configuration>
   
4.         <property>
   
5.                 <name>dfs.namenode.secondary.http-address</name>
   
6.                 <value>192.168.0.129:50090</value>
   
7.         </property>
   
8. 
   
9.         <property>
   
10.                 <name>dfs.namenode.secondary.https-address</name>
    
11.                  <value>192.168.0.129:50091</value>
    
12.         </property>
    
13.         <property>
    
14.                  <name>dfs.replication</name>
    
15.                   <value>1</value>
    
16.         </property>
    
17.     </configuration>

6、删除root用户的DFS文件及DFS磁盘格式化

1. [root@hadoop001 tmp]# rm -rf /tmp/hadoop-* /tmp/hsperfdata-*
   
2. [root@hadoop001 tmp]# su - hadoop
   
3. [hadoop@hadoop001 hadoop-2.8.1]$ hdfs namenode -format

7、hadoop用户启动

1. [hadoop@hadoop001 sbin]$ pwd
   
2. /opt/software/hadoop-2.8.1/sbin
   
3. [hadoop@hadoop001 sbin]$ ./start-dfs.sh       -- 第一次启动输入密码
   
4. Starting namenodes on [hadoop001]
   
5. hadoop001: starting namenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-hadoop001.out
   
6. 192.168.0.129: starting datanode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-hadoop001.out
   
7. Starting secondary namenodes [hadoop001]
   
8. hadoop001: starting secondarynamenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-secondarynamenode-hadoop001.out

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/31441024/viewspace-2154712/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/31441024/viewspace-2154712/