CentOS6.5下编译Hadoop-2.7.2

在部署完hadoop-2.7.2伪分布式环境之后，在开启hadoop及执行shell操作的时候，经常会遇到 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable 的报错，有些shell操作甚至无法执行，用hadoop checknative -a命令查看本地库支持，结果如下：
hadoop: false
zlib: false
snappy: false
lz4: false
bzip2: false
openssl: false
在网上查了一下原因，可能是因为hadoop版本是从32位操作系统编译出来的，所以需要在我的机器上重新编译，编译步骤整理如下。

一、下载所需的包

使用yum进行补包，在整个编译的过程中，需要有一个网卡能够连接外网。

yum -y install  svn   ncurses-devel   gcc* 
yum -y install lzo-devel zlib-devel autoconf automake libtool    cmake 

二、安装maven

下载好maven，将其解压到/opt目录下，然后在/etc/profile里追加：

export MAVEN_HOME=/opt/maven/apache-maven-3.3.9
export PATH=$PATH:$MAVEN_HOME/bin

然后source /etc/profile
cd到/opt/maven/apache-maven-3.3.9/bin目录下，执行命令：

chmod 777 *

否则在编译时会出现mvn命令拒绝访问的错误。

三、 安装protocol

将下载好的protocal解压到/opt下，cd到解压后的目录，然后进行源码安装。（./configure && make && make install）

四、编译Hadoop

下载相应的hadoop源码文件，我的hadoop源码文件为hadoop-2.7.2-src.tar.gz，对应着我机器装的hadoop版本。将该源码文件解压到/opt下，然后在解压后的目录中中执行命令：

mvn package -Pdist,native -DskipTests -Dtar

之后进行漫长的等待。。。编译中间报了一个错，导致编译不成功，
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-pipes: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part …… @ 5:122 in /opt/hadoop/hadoop-2.7.2-src/hadoop-tools/hadoop-pipes/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hadoop-pipes
在网上查了一下该报错出现的原因是因为缺少libssl-dev的包，但是yum源里面并没有。原来，在Ubuntu下名字为这个包，在CentOS下该包的名字为openssl-devel, 执行命令：

yum install openssl-devel -y

然后重新编译Hadoop,就可以编译成功了。
[INFO] Apache Hadoop Main …………………………… SUCCESS [ 3.245 s]
[INFO] Apache Hadoop Project POM …………………….. SUCCESS [ 2.311 s]
[INFO] Apache Hadoop Annotations …………………….. SUCCESS [ 6.611 s]
[INFO] Apache Hadoop Assemblies ……………………… SUCCESS [ 0.463 s]
[INFO] Apache Hadoop Project Dist POM ………………… SUCCESS [ 2.883 s]
[INFO] Apache Hadoop Maven Plugins …………………… SUCCESS [ 6.207 s]
[INFO] Apache Hadoop MiniKDC ………………………… SUCCESS [ 5.244 s]
[INFO] Apache Hadoop Auth …………………………… SUCCESS [ 8.285 s]
[INFO] Apache Hadoop Auth Examples …………………… SUCCESS [ 4.947 s]
[INFO] Apache Hadoop Common …………………………. SUCCESS [02:21 min]
[INFO] Apache Hadoop NFS ……………………………. SUCCESS [ 10.084 s]
[INFO] Apache Hadoop KMS ……………………………. SUCCESS [ 14.141 s]
[INFO] Apache Hadoop Common Project ………………….. SUCCESS [ 0.085 s]
[INFO] Apache Hadoop HDFS …………………………… SUCCESS [05:22 min]
[INFO] Apache Hadoop HttpFS …………………………. SUCCESS [ 24.384 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ………….. SUCCESS [ 10.518 s]
[INFO] Apache Hadoop HDFS-NFS ……………………….. SUCCESS [ 6.329 s]
[INFO] Apache Hadoop HDFS Project ……………………. SUCCESS [ 0.102 s]
[INFO] hadoop-yarn …………………………………. SUCCESS [ 0.087 s]
[INFO] hadoop-yarn-api ……………………………… SUCCESS [ 54.766 s]
[INFO] hadoop-yarn-common …………………………… SUCCESS [ 52.697 s]
[INFO] hadoop-yarn-server …………………………… SUCCESS [ 0.096 s]
[INFO] hadoop-yarn-server-common …………………….. SUCCESS [ 21.298 s]
[INFO] hadoop-yarn-server-nodemanager ………………… SUCCESS [ 26.294 s]
[INFO] hadoop-yarn-server-web-proxy ………………….. SUCCESS [ 4.768 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ……. SUCCESS [ 10.088 s]
[INFO] hadoop-yarn-server-resourcemanager …………….. SUCCESS [ 27.993 s]
[INFO] hadoop-yarn-server-tests ……………………… SUCCESS [ 6.569 s]
[INFO] hadoop-yarn-client …………………………… SUCCESS [ 11.344 s]
[INFO] hadoop-yarn-server-sharedcachemanager ………….. SUCCESS [ 5.907 s]
[INFO] hadoop-yarn-applications ……………………… SUCCESS [ 0.060 s]
[INFO] hadoop-yarn-applications-distributedshell ………. SUCCESS [ 4.165 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ….. SUCCESS [ 3.283 s]
[INFO] hadoop-yarn-site …………………………….. SUCCESS [ 0.062 s]
[INFO] hadoop-yarn-registry …………………………. SUCCESS [ 9.204 s]
[INFO] hadoop-yarn-project ………………………….. SUCCESS [ 7.306 s]
[INFO] hadoop-mapreduce-client ………………………. SUCCESS [ 0.119 s]
[INFO] hadoop-mapreduce-client-core ………………….. SUCCESS [ 40.384 s]
[INFO] hadoop-mapreduce-client-common ………………… SUCCESS [ 32.566 s]
[INFO] hadoop-mapreduce-client-shuffle ……………….. SUCCESS [ 6.233 s]
[INFO] hadoop-mapreduce-client-app …………………… SUCCESS [ 17.094 s]
[INFO] hadoop-mapreduce-client-hs ……………………. SUCCESS [ 11.530 s]
[INFO] hadoop-mapreduce-client-jobclient ……………… SUCCESS [ 11.564 s]
[INFO] hadoop-mapreduce-client-hs-plugins …………….. SUCCESS [ 3.699 s]
[INFO] Apache Hadoop MapReduce Examples ………………. SUCCESS [ 10.132 s]
[INFO] hadoop-mapreduce …………………………….. SUCCESS [ 4.850 s]
[INFO] Apache Hadoop MapReduce Streaming ……………… SUCCESS [ 9.519 s]
[INFO] Apache Hadoop Distributed Copy ………………… SUCCESS [ 14.060 s]
[INFO] Apache Hadoop Archives ……………………….. SUCCESS [ 3.988 s]
[INFO] Apache Hadoop Rumen ………………………….. SUCCESS [ 10.398 s]
[INFO] Apache Hadoop Gridmix ………………………… SUCCESS [ 8.661 s]
[INFO] Apache Hadoop Data Join ………………………. SUCCESS [ 5.616 s]
[INFO] Apache Hadoop Ant Tasks ………………………. SUCCESS [ 3.962 s]
[INFO] Apache Hadoop Extras …………………………. SUCCESS [ 5.549 s]
[INFO] Apache Hadoop Pipes ………………………….. SUCCESS [ 11.318 s]
[INFO] Apache Hadoop OpenStack support ……………….. SUCCESS [ 8.387 s]
[INFO] Apache Hadoop Amazon Web Services support ………. SUCCESS [14:45 min]
[INFO] Apache Hadoop Azure support …………………… SUCCESS [ 51.242 s]
[INFO] Apache Hadoop Client …………………………. SUCCESS [ 12.740 s]
[INFO] Apache Hadoop Mini-Cluster ……………………. SUCCESS [ 0.168 s]
[INFO] Apache Hadoop Scheduler Load Simulator …………. SUCCESS [ 7.728 s]
[INFO] Apache Hadoop Tools Dist ……………………… SUCCESS [ 16.374 s]
[INFO] Apache Hadoop Tools ………………………….. SUCCESS [ 0.054 s]
[INFO] Apache Hadoop Distribution ……………………. SUCCESS [ 50.550 s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
然后在启动hadoop，就可以看到之前启动时候警告信息消失了。