windows7 64位cygwin环境maven编译hadoop2.7.1

最新推荐文章于 2021-05-09 00:15:10 发布

原创最新推荐文章于 2021-05-09 00:15:10 发布 · 1.8k 阅读

0 ·

CC 4.0 BY-SA版权

搜索引擎专栏收录该内容

6 篇文章

订阅专栏

本文档详细介绍了如何在Windows7 64位系统中，利用Cygwin环境编译Hadoop 2.7.1。主要步骤包括下载源码、修改特定文件以解决权限问题、执行Maven打包命令以及安装Hadoop Maven插件。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

主要参照BUILDING.txt的操作来

* JDK 1.7+
* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer
* Windows SDK 7.1 or Visual Studio 2010 Professional
* Windows SDK 8.1 (if building CPU rate control for the container executor)
* zlib headers (if building native code bindings for zlib)
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip. These tools must be present on your PATH. 如cygwin、mingw
*从cygwin安装包安装openssl包
   Net 下的：openssh,openssl
   Base 下的：sed （若需要Eclipse，必须sed）
   Devel 下的：subversion（建议安装）
* svn command
* git command
* vs2010 professional

1、下载hadoop-2.7.1-src.tar.gz并解压到D根目录，建议文件名越短越好，太长windows报错

2、修改源文件hadoop-2.7.1-src\hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\fs\FileUtil.java，防止出现权限错误

  private static void checkReturnValue(boolean rv, File p, 
                                       FsPermission permission
                                       ) throws IOException {
    //if (!rv) {
    //  throw new IOException("Failed to set permissions of path: " + p + 
    //                        " to " + 
    //                        String.format("%04o", permission.toShort()));
    //}
  }

3、可能会碰到hadoop-common-project\hadoop-common\src\main\native\native.sln编译不过去，主要是找不到zlib.h，手动打开sln，添加include路径即可

4、

* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer

这几个命令一定要加载PATH路径下面.

5、源文件根目录执行mvn package -Pdist,native-win,docs -DskipTests -Dtar等待build sucess

6、cd hadoop-maven-plugins，执行mvn install

7、嫌默认maven源下载慢的可以修改conf/settings.xml

<mirrors>
    <mirror>
		 <id>nexus-osc</id>
         <mirrorOf>*</mirrorOf>
		 <name>Nexusosc</name>
		 <url>http://maven.oschina.net/content/groups/public/</url>
    </mirror>
     
  </mirrors>

  <profiles>
	<profile>
       <id>jdk-1.8</id>
       <activation>
         <jdk>1.8</jdk>
       </activation>
       <repositories>
         <repository>
           <id>nexus</id>
           <name>local private nexus</name>
           <url>http://maven.oschina.net/content/groups/public/</url>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
         </repository>
       </repositories>
       <pluginRepositories>
         <pluginRepository>
           <id>nexus</id>
          <name>local private nexus</name>
           <url>http://maven.oschina.net/content/groups/public/</url>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
         </pluginRepository>
       </pluginRepositories>
     </profile>
  </profiles>