最近公司hadoop集群准备升级到CHD4U6,要进行升级前的job测试。需要依赖一些新的公共jar包,比如:
- hadoop-annotations-2.0.0-cdh4.6.0.jar
- hadoop-auth-2.0.0-cdh4.6.0.jar
- hadoop-common-2.0.0-cdh4.6.0.jar
与之前的hadoop版本(chd3u5)依赖不同。但用nexus搭建的maven库并没有创建该公共repository代理。
查得:http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.1/CDH4-Installation-Guide/cdh4ig_topic_31.html
根据其中描述得到id:cloudera, remote storage location:https://repository.cloudera.com/artifactory/cloudera-repos/,于是创建代理。
创建后编译不行,看了下setting文件:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<proxies>
<proxy>
<id>normal</id>
<active>true</active>
<protocol>http</protocol>
<username>Deployment</username>
<host>10.16.1.240:8081/nexus</host>
<!-- <port>80</port> -->
<nonProxyHosts>10.16.1.240:8081/nexus</nonProxyHosts>
</proxy>
</proxies>
<servers>
<server>
<id>releases</id>
<username>Deployment</username>
</server>
<server>
<id>snapshots</id>
<username>Deployment</username>
</server>
</servers>
<mirrors>
<mirror>
<!--This is used to direct the public snapshots repo in the
profile below over to a different nexus group -->
<id>nexus-public-snapshots</id>
<mirrorOf>public-snapshots</mirrorOf>
<url>http://10.16.1.240:8081/nexus/content/groups/public-snapshots</url>
</mirror>
<span style="color:#ff0000;"> <mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://10.16.1.240:8081/nexus/content/groups/public</url>
</mirror></span>
</mirrors>
<profiles>
<profile>
<id>development</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
<profile>
<!--this profile will allow snapshots to be searched when activated-->
<id>public-snapshots</id>
<repositories>
<repository>
<id>public-snapshots</id>
<url>http://public-snapshots</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>public-snapshots</id>
<url>http://public-snapshots</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>development</activeProfile>
</activeProfiles>
</settings>
本文详细介绍了如何在已有Nexus搭建的Maven库中,为Hadoop集群升级到CDH4U6时,解决由于依赖库版本不同导致的编译问题。通过创建Cloudera的repository代理,并将其加入到Nexus的group中,最终成功解决了依赖问题,确保了集群升级的顺利进行。
2592

被折叠的 条评论
为什么被折叠?



