linux安装hadoop2.9.0,Linux ubuntu-hadoop-2.9.2安装与配置

获取压缩包

从*https://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz*获得压缩包

$ wget https://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz

创建目录

$ sudo mkdir /opt/hadoop

解压文件

$ tar -xf hadoop-2.9.2.tar.gz

移动文件

$ sudo sudo mv hadoop-2.9.2 /opt/hadoop/

安装JDK

创建jvm目录

$ sudo mkdir /usr/lib/jvm

应该是不用预先创建的

$ sudo apt-get install openjdk-8-jdk

检查JDK的安装

$ java -version

设置JAVA_HOME变量

$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/

检查JAVA_HOME

$ echo $JAVA_HOME

配置hadoop

其他配置方法若配置以后执行start-all.sh出现以下错误，通过本配置方法均可得到解决。

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh

Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.

Starting namenodes on []

localhost: Error: JAVA_HOME is not set and could not be found.

localhost: Error: JAVA_HOME is not set and could not be found.

Starting secondary namenodes [0.0.0.0]

The authenticity of host ‘0.0.0.0 (0.0.0.0)’ can’t be established.

ECDSA key fingerprint is SHA256:SHA256_value

Are you sure you want to continue connecting (yes/no)? no

0.0.0.0: Host key verification failed.

starting yarn daemons

starting resourcemanager, logging to filename.out

localhost: Error: JAVA_HOME is not set and could not be found.

创建hadoop的软连接

$ sudo ln -s /opt/hadoop/hadoop-2.9.2/bin/hadoop /bin/hadoop

测试hadoop命令

有个错误：

/bin/hadoop: 行 27: /bin/…/libexec/hadoop-config.sh: 没有那个文件或目录

解决上述错误的办法是创建一个软连接

$ sudo ln -s /opt/hadoop/hadoop-2.9.2/libexec/ /libexec

进入配置文件目录

$ cd /opt/hadoop/hadoop-2.9.2/etc/hadoop

配置hadoop-env.sh，core-site.xml，hdfs-site.xml

hadoop-env.sh

主要配置JAVA_HOME变量，如果直接使用$JAVA_HOME可能会出现找不到JAVA_HOME变量的错误。

core-site.xml

配置的时候注意路径，如果不是安装在/opt/hadoop下的相对应改变即可。

hdfs-site.xml

初始化配置

$ hadoop namenode -format

启动

$ ./start-dfs.sh

$ jps

为了方便也可以给start-dfs.sh在/bin下创建一个软连接，这样就不用进入安装目录执行了。