1.环境搭建
镜像版本: CentOS-7-x86_64-DVD-1908.iso
SOFTWARE SELECTION:GNOME Desktop(桌面版,比Minimal Install版多部分软件)
Hadoop版本:hadoop-3.2.1-src.tar.gz
下载地址:All previous releases of Hadoop are available from the Apache release archive site.
hadoop源码编译所需的其他软件在源码包的BUILDING.txt文件中有详细介绍,编译过程中发生错误,99%是因为所需软件未安装,根据错误描述安装对应版本软件即可顺利编译。
Build instructions for Hadoop
----------------------------------------------------------------------------------
Requirements:
* Unix System
* JDK 1.8
* Maven 3.3 or later
* ProtocolBuffer 2.5.0
* CMake 3.1 or newer (if compiling native code)
* Zlib devel (if compiling native code)
* Cyrus SASL devel (if compiling native code)
* One of the compilers that support thread_local storage: GCC 4.8.1 or later, Visual Studio,
Clang (community version), Clang (version for iOS 9 and later) (if compiling native code)
* openssl devel (if compiling native hadoop-pipes and to get the best HDFS encryption performance)
* Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
* Doxygen ( if compiling libhdfspp and generating the documents )
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* python (for releasedocs)
* bats (for shell code testing)
* Node.js / bower / Ember-cli (for YARN UI v2 building)
2.软件安装
总结:
①不编译native code和documentation
Create binary distribution without native code and without documentation:
$ mvn package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
则仅需安装JDK 1.8,Maven 3.3 or later,ProtocolBuffer 2.5.0,gcc,gcc-c++,patch即可
②编译native code和documentation
Create binary distribution with native code and with documentation:
$ mvn package -Pdist,native,docs -DskipTests -Dtar
除JDK 1.8,Maven 3.3 or later,ProtocolBuffer 2.5.0,gcc,gcc-c++,patch还需要安装CMake 3.1 or newer,zlib-devel,openssl-devel,cyrus-sasl-devel
1.JDK安装
查看系统自带JDK,自