CentOS6.5环境下编译hadoop2.8.1源码

一、环境准备：
（1）CentOS6.5虚拟机
（2）hadoop2.8.1源码包下载：http://pan.baidu.com/s/1mikiNgc
（3）jdk安装包下载：http://pan.baidu.com/s/1eRJqFRW
（4）maven安装包下载:http://pan.baidu.com/s/1dEKEGff
（5）protobuf安装包下载：http://pan.baidu.com/s/1qYJmAb6
（6）Findbugs安装包下载：http://pan.baidu.com/s/1i4OAkZj

二、源码编译
1.查看是否安装过hadoop

su - root
ps -ef|grep hadoop
find / -name hadoop

2.在/opt下创建目录，上传源码包，解压

su - root
cd /opt
mkdir sourcecode software
cd sourcecode
tar -xzvf hadoop-2.8.1-src.tar.gz

查看安装要求：

cat BUILDING.txt

Requirements:

• Unix System
• JDK 1.7+
• Maven 3.0 or later
• Findbugs 1.3.9 (if running findbugs)
• ProtocolBuffer 2.5.0
• CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac
• Zlib devel (if compiling native code)
• openssl devel (if compiling native hadoop-pipes and to get the best HDFS encryption performance)
• Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
• Internet connection for first build (to fetch all Maven and Hadoop dependencies)

3.上传jdk安装包：/usr/java，加压安装，配置java环境
新建安装目录

su - root
mkdir -p /usr/java

上传java安装包：/usr/java,修改用户组（解压后需要确保文件夹用户和组为：root:root）

cd /usr/java
tar -xzvf jdk-8u45-linux-x64.gz
chown -R root:root jdk1.8.0_45

配置环境

vi /etc/profile
#add java
export JAVA_HOME=/usr/java/jdk1.8.0_45
export PATH=$JAVA_HOME/bin:$PATH
source /etc/profile

which java
java -version

4.上传maven安装包：/opt/software，解压安装，配置maven环境

cd /opt/software
unzip apache-maven-3.3.9-bin.zip

vi /etc/profile
#add mvn
export MAVEN_HOME=/opt/software/apache-maven-3.3.9
export PATH=$MAVEN_HOME/bin:$PATH
source /etc/profile

which mvn
mvn -version

5.上传protobuf安装包：/opt/software，解压安装，配置protobuf环境

cd /opt/software
tar -xzvf protobuf-2.5.0.tar.gz

cd /opt/software/protobuf-2.5.0
#安装cmake
yum install -y gcc gcc-c++ make cmake
./configure --prefix=/usr/local/protobuf
make && make install

vi /etc/profile
#add protoc
export PROTOC_HOME=/usr/local/protobuf
export PATH=$PROTOC_HOME/bin/:$PATH
source /etc/profile

which protoc
protoc --version

6.上传Findbugs安装包：/opt/software，解压安装，配置Findbugs环境

cd /opt/software
unzip findbugs-1.3.9.zip

vi /etc/profile
#add findbugs
export FINDBUGS_HOME=/opt/software/findbugs-1.3.9
export PATH=$FINDBUGS_HOME/bin:$PATH
source /etc/profile

which findbugs
findbugs -version

7.其他依赖

yum install -y openssl openssl-devel svn ncurses-devel zlib-devel libtool
yum install -y snappy snappy-devel bzip2 bzip2-devel lzo lzo-devel lzop autoconf automake

8.编译

cd /opt/sourcecode/hadoop-2.8.1-src
mvn clean package -Pdist,native -DskipTests -Dtar

注意：
（1）有时候编译过程中会出现下载某个包的时间太久，这是由于连接网站的过程中会出现假死，此时按ctrl+c，重新运行编译命令。
（2）如果出现缺少了某个文件的情况，则要先清理maven(使用命令 mvn clean) 再重新编译。

9.针对编译过程中卡死问题，请更新配置文件连接阿里云的源

cd /opt/software/apache-maven-3.3.9/conf
vi settings.xml

<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
</mirror>

cd /opt/sourcecode/hadoop-2.8.1-src
vi pom.xml

<repository>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
</repository>

再次执行编译命令：

cd /opt/sourcecode/hadoop-2.8.1-src
mvn clean package -Pdist,native,docs -DskipTests -Dtar

10.编译完成，查看编译成功的tar包

ll /opt/sourcecode/hadoop-2.8.1-src/hadoop-dist/target/hadoop-2.8.1.tar.gz

至此，CentOS6.5环境下编译hadoop2.8.1源码我们已经完成