/usr/local/spark/sbin/start-all.sh

最新推荐文章于 2023-06-24 22:30:23 发布

原创最新推荐文章于 2023-06-24 22:30:23 发布 · 1k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#spark

本文详细介绍了如何使用`/usr/local/spark/sbin/start-all.sh`命令启动Spark集群，包括集群启动过程、相关配置解析以及常见问题排查。通过阅读，读者将了解启动Spark集群的完整步骤和注意事项。

data2: failed to launch: nice -n 0 /usr/local/spark/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://master:7077

hduser@master:/usr/local/spark/logs$ ls
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out.1
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out.2
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out.3
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out.4
spark-hduser-org.apache.spark.deploy.master.Master-1-master.out.5
hduser@master:/usr/local/spark/logs$ vim spark-hduser-org.apache.spark.deploy.master.Master-1-master.out
hduser@master:/usr/local/spark/logs$ cat spark-hduser-org.apache.spark.deploy.master.Master-1-master.out
Spark Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp /usr/local/spark/conf/:/usr/local/spark/jars/*:/usr/local/hadoop/etc/hadoop/ -Xmx1g org.apache.spark.deploy.master.Master --host master --port 7077 --webui-port 8080
========================================
18/06/26 09:09:31 WARN MasterArguments: SPARK_MASTER_IP is deprecated, please use SPARK_MASTER_HOST
18/06/26 09:09:44 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hduser@master:/usr/local/spark/logs$

18/06/26 09:38:39 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources