hadoop2.7.x安装与配置

1.虚拟机

master 192.168.109.200

node1 192.168.109.201

node2 192.168.109.202

2.安装jdk1.8,下载jdk1.8（密码: 7sw5）,多台虚拟机无密码登录

3.安装jdk

4.安装配置hadoop2.7.x

下载hadoop2.7.3 （密码：f8vf）

在虚拟机master的hadoop用户下创建：hadoop文件夹

mkdir hadoop;

创建tmp、hdfs、hdfs/data、hdfs/name

mkdir tmp

mkdir hdfs

mkdir hdfs/name

mkdir hdfs/data

把hadoop2.73上传到虚拟机的hadoop用户的hadoop文件夹下

tar -zxvf hadoop2.7.tar.gz

mv hadoop2.7.3xx hadoop

修改配置文件：

core-site.xml（hadoop/etc/hadoop/core-site.xml）:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.109.200:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/hadoop/hadoop/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
 </configuration>

hdfs-site.xml（hadoop/etc/hadoop/hdfs-site.xml）:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/home/hadoop/hadoop/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/home/hadoop/hadoop/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>192.168.109.200:9001</value>
    </property>
    <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
    </property>
<property> 
     <name>dfs.permissions</name> 
     <value>false</value> 
  </property>

 </configuration>

mapred-site.xml（hadoop/etc/hadoop/mapred-site.xml）:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>192.168.109.200:10020</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>192.168.109.200:19888</value>
    </property>
 </configuration>

yarn-site.xml（hadoop/etc/hadoop/yarn-site.xml）:

<?xml version="1.0"?>
<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>192.168.109.200:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>192.168.109.200:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>192.168.109.200:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>192.168.109.200:8033</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>192.168.109.200:8088</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>64</value>
    </property>
 </configuration>

hadoop-env.sh（hadoop/etc/hadoop/hadoop-env.sh）:(放开25行，修改成你的jdk地址（#$JAVA_HOME可以查看）)

export JAVA_HOME=/usr/local/jdk

yarn-env.sh（hadoop/etc/hadoop/yarn-env.sh）:(放开25行，修改成你的jdk地址（#$JAVA_HOME可以查看）)

export JAVA_HOME=/usr/local/jdk

配置/home/hadoop/hadoop-2.7.0/etc/hadoop目录下的slaves，删除默认的localhost，增加2个从节点，
node1

node2

将配置好的~/hadoop复制到各个节点对应位置上，通过scp传送，
scp -r /home/hadoop/hadoop hadoop@node1:/home/hadoop/hadoop
scp -r /home/hadoop/hadoop hadoop@node2:/home/hadoop/hadoop

在master服务器启动hadoop，从节点会自动启动，进入/home/hadoop/hadoop目录
(1)初始化，输入命令，bin/hdfs namenode -format
(2)全部启动sbin/start-all.sh，也可以分开sbin/start-dfs.sh、sbin/start-yarn.sh
(3)停止的话，输入命令，sbin/stop-all.sh
(4)输入命令，jps，可以看到相关信息

主要参考：http://www.linuxidc.com/Linux/2015-11/124800.htm