cdh安装spark遇到的几个BUG

最新推荐文章于 2024-05-20 17:45:23 发布

aisi5656

最新推荐文章于 2024-05-20 17:45:23 发布

阅读量508

点赞数

文章标签：大数据 java

原文链接：http://www.cnblogs.com/zhaojinyan/p/9599521.html

版权

本文介绍了解决Spark启动失败的问题，原因是缺少Hadoop与Spark间通信所需的jar包。提供了具体解决方案，包括所需下载的jar包名称及下载地址，并指导如何正确配置以确保Spark正常启动。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

spark安装后启动：

[zdwy@master spark]$ sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.master.Master-1-master.out
failed to launch org.apache.spark.deploy.master.Master:
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
full log in /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.master.Master-1-master.out
slave1: starting org.apache.spark.deploy.worker.Worker, logging to /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.worker.Worker-1-slave1.out
slave2: starting org.apache.spark.deploy.worker.Worker, logging to /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.worker.Worker-1-slave2.out
slave1: failed to launch org.apache.spark.deploy.worker.Worker:
slave1: at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
slave1: ... 6 more
slave1: full log in /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.worker.Worker-1-slave1.out
slave2: failed to launch org.apache.spark.deploy.worker.Worker:
slave2: at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
slave2: ... 6 more
slave2: full log in /home/zdwy/cdh5.9/spark/logs/spark-zdwy-org.apache.spark.deploy.worker.Worker-1-slave2.out

原因：缺少hadoop和spark之间通信的jar包

解决方案：下载3个jar包：jackson-core-xxx.jar,jackson-annotations-xxx.jar,jackson-databind-xxx.jar,下载地址：http://mvnrepository.com/artifact/com.fasterxml.jackson.core/

下载后将jar包放入到hadoop/share/hadoop/commom/目录下，重新启动spark即可。

转载于:https://www.cnblogs.com/zhaojinyan/p/9599521.html