spark version : spark-2.0.0
maven : Apache Maven 3.5.0
cd /usr/local/spark-2.0.0
1
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
Note:
For Java 8 and above this step is not required.
If using build/mvn with no MAVEN_OPTS set, the script will automate this for you.
2
mvn -Pyarn -Phadoop-2.7 -Pspark-ganglia-lgpl -Pkinesis-asl -Phive -DskipTests clean package
3
./dev/make-distribution.sh --name custom-spark --tgz -Psparkr -Phadoop-2.7 -Phive -Phive-thriftserver -Pyarn
Note:
if you come across spark-core_2.11: Command execution failed
may be you should try some ways the following
On Debian GNU/Linux (in particular 8/jessie) you would need to run
apt-get install r-base r-base-dev
or
aptitude install r-base r-base-dev
compile spark2.0.0 with maven
最新推荐文章于 2026-01-03 08:52:33 发布
本文介绍如何安装配置 Spark 2.0.0 版本,并提供详细的步骤说明,包括 Maven 的配置及 Spark 相关插件的启用。此外还提供了针对 Debian GNU/Linux 特别是 8/jessie 版本的解决方案。
7143

被折叠的 条评论
为什么被折叠?



