sql_python的博客

Job的提交 任何一个行动算子，都会执行：runJob() val results = sc.runJob() runJob()在DAGScheduler类中 1. 提交job，是一个阻塞线程 val waiter = submitJob(rdd, func, partitions, callSite, resultHandler, properties) //742 2. submitJob中： val jobId = nextJobId.getAndIncrement() //生成JobId，

Submit流程(yarn) #执行提交命令： bin/spark-submit --master yarn --deploy-mode cluster --class com.Tencent.spark.WordCount /opt/module/spark-standalone/bin/spark-1.0-SNAPSHOT.jar hdfs://hadoop:9820/input 执行脚本实际是执行： bin/spark-class org.apache.spark.dep