spark编译报错信息简介

最新推荐文章于 2023-01-16 13:31:18 发布

weixin_30323961

最新推荐文章于 2023-01-16 13:31:18 发布

阅读量158

点赞数

CC 4.0 BY-SA版权

文章标签：大数据 java git

原文链接：http://www.cnblogs.com/zhzhang/p/7347523.html

本文介绍了在编译Spark过程中遇到的三种常见错误及其解决办法，包括Maven版本不匹配、R包安装问题及Scala编译失败等，并提供了相应的解决方案。

spark编译需要环境
git
java1.7+
maven
R


报错信息1：
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52.661s (Wall Clock)
[INFO] Finished at: Thu Aug 10 11:04:00 CST 2017
[INFO] Final Memory: 25M/200M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-versions) on project spark-parent_2.11: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

报错信息1的解决办法：maven 版本需要是3.3.9

报错信息2：
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:25 min (Wall Clock)
[INFO] Finished at: 2017-08-10T11:25:18+08:00
[INFO] Final Memory: 50M/633M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.11: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :spark-core_2.11

报错信息2的解决办法
yum install -y R

报错信息3
[INFO] Spark Project Java 8 Tests ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38:34 min (Wall Clock)
[INFO] Finished at: 2017-08-10T17:21:52+08:00
[INFO] Final Memory: 69M/664M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) on project spark-hive_2.11: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed. CompileFailed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :spark-hive_2.11
Destroying 1 processes
Destroying process..
Destroyed 1 processes

报错信息3的解决办法
编译过程中加上参数 -DskipTests