Hadoop 3.1.0 ErasureCodingPolicy导致spark streaming的任务失败问题分析

最新推荐文章于 2023-12-07 19:24:31 发布

T-Janey

最新推荐文章于 2023-12-07 19:24:31 发布

阅读量2.1k

点赞数 2

CC 4.0 BY-SA版权

分类专栏： BigData

本文链接：https://blog.youkuaiyun.com/anitinaj/article/details/84754413

一、ErasureCodePolicy问题，导致streaming任务退出

1、任务失败的原因，当执行block时有uncaught 异常时，stop sparkcontext，如下

具体源码错误路径，感兴趣的，可以根据错误日志跟踪一下，这里就不具体跟踪了，只显示比较重要的源码信息
AsyncEventQueue

  private val dispatchThread = new Thread(s"spark-listener-group-$name") {
    setDaemon(true)
    override def run(): Unit = Utils.tryOrStopSparkContext(sc) {
      dispatch()
    }
  }

  private def dispatch(): Unit = LiveListenerBus.withinListenerThread.withValue(true) {
    try {
      var next: SparkListenerEvent = eventQueue.take()
      while (next != POISON_PILL) {
        val ctx = processingTime.time()
        try {
          super.postToAll(next)
        } finally {
          ctx.stop()
        }
        eventCount.decrementAndGet()
        next = eventQueue.take()
      }
      eventCount.decrementAndGet()
    } catch {
      case ie: InterruptedException =>
        logInfo(s"Stopping listener queue $name.", ie)
    }
  }

org.apache.spark.util.Utils

  /**
   * Execute a block of code that evaluates to Unit, stop SparkContext if there is any uncaught
   * exception
   *
   * NOTE: This method is to be called by the driver-side components to avoid stopping the
   * user-started JVM process completely; in contrast, tryOrExit is to be called in the
   * spark-started JVM process .
   */
  def tryOrStopSparkContext(sc: SparkContext)(block: => Unit) {
    try {
      block
    } catch {
      case e: ControlThrowable => throw e
      case t: Throwable =>
        val c