Spark 源码分析(二) SparkRpc中Rpc架构解读（正在更新Netty部分~）

本文链接：https://blog.youkuaiyun.com/Peng909157372/article/details/145035828

上一章节序列化解读

二、Spark.rpc中Netty架构源码解读

⑤ RemoteProcessConnected 、RemoteProcessDisconnected 、

RemoteProcessConnectionError

⑥ Inbox 重点！要考！

二、Spark.rpc中Netty架构源码解读

在进行源码解读之前，首先分析类间关系

简单阐述类间关系，后续再深入理解

1、首先是RpcEndpoint

从图中可以看到，RpcEndpoint中包含了一个Inbox专门用于存储消息，一个MessageLoop专门用于消费消息，一个Dispatcher调度器专门用于将消息调度到对应的地址上。、

接下来我们首先对Inbox源码进行解读，看看Inbox是如何存储消息并传输出去的。

(1) Inbox源码解读

① InboxMessage

private[netty] sealed trait InboxMessage

定义了Inbox中的消息存放的基本特质，而且这个特质只能被本文件的类继承

② OneWayMessage

private[netty] case class OneWayMessage(
    senderAddress: RpcAddress,
    content: Any) extends InboxMessage

定义了一种单向消息（也就是没有返回的消息），构造参数是目的地的Rpc地址和传输内容

③ RpcMessage

private[netty] case class RpcMessage(
    senderAddress: RpcAddress,
    content: Any,
    context: NettyRpcCallContext) extends InboxMessage

定义了一个Rpc消息，构造参数是目的地的Rpc地址和传输内容和NettyRpcCallContext类的实例

④ OnStart 和 OnStop

private[netty] case object OnStart extends InboxMessage

private[netty] case object OnStop extends InboxMessage

定义了两种消息类型，OnStart和OnStop是用于标识RpcEndpoint生命周期的关键消息

⑤ RemoteProcessConnected 、RemoteProcessDisconnected 、

RemoteProcessConnectionError

/** A message to tell all endpoints that a remote process has connected. */
private[netty] case class RemoteProcessConnected(remoteAddress: RpcAddress) extends InboxMessage

/** A message to tell all endpoints that a remote process has disconnected. */
private[netty] case class RemoteProcessDisconnected(remoteAddress: RpcAddress) extends InboxMessage

/** A message to tell all endpoints that a network error has happened. */
private[netty] case class RemoteProcessConnectionError(cause: Throwable, remoteAddress: RpcAddress)
  extends InboxMessage

这是三种类似的提示消息类型

RemoteProcessConnected负责告知所有endpoint一个远程进程已连接

RemoteProcessDisconnected负责告知所有endpoint一个远程进程已断开连接

RemoteProcessConnectionError负责告知所有endpoint出现了一个网络问题

⑥ Inbox 重点！要考！

private[netty] class Inbox(val endpointName: String, val endpoint: RpcEndpoint)
  extends Logging {

  inbox =>  // Give this an alias so we can use it more clearly in closures.

  @GuardedBy("this")
  protected val messages = new java.util.LinkedList[InboxMessage]()

  /** True if the inbox (and its associated endpoint) is stopped. */
  @GuardedBy("this")
  private var stopped = false

  /** Allow multiple threads to process messages at the same time. */
  @GuardedBy("this")
  private var enableConcurrent = false

  /** The number of threads processing messages for this inbox. */
  @GuardedBy("this")
  private var numActiveThreads = 0

  // OnStart should be the first message to process
  inbox.synchronized {
    messages.add(OnStart)
  }

/**
   * Process stored messages.
   */
  def process(dispatcher: Dispatcher): Unit = {
    var message: InboxMessage = null
    inbox.synchronized {
      if (!enableConcurrent && numActiveThreads != 0) {
        return
      }
      message = messages.poll()
      if (message != null) {
        numActiveThreads += 1
      } else {
        return
      }
    }
    while (true) {
      safelyCall(endpoint) {
        message match {
          case RpcMessage(_sender, content, context) =>
            try {
              endpoint.receiveAndReply(context).applyOrElse[Any, Unit](content, { msg =>
                throw new SparkException(s"Unsupported message $message from ${_sender}")
              })
            } catch {
              case e: Throwable =>
                context.sendFailure(e)
                // Throw the exception -- this exception will be caught by the safelyCall function.
                // The endpoint's onError function will be called.
                throw e
            }

          case OneWayMessage(_sender, content) =>
            endpoint.receive.applyOrElse[Any, Unit](content, { msg =>
              throw new SparkException(s"Unsupported message $message from ${_sender}")
            })

          case OnStart =>
            endpoint.onStart()
            if (!endpoint.isInstanceOf[ThreadSafeRpcEndpoint]) {
              inbox.synchronized {
                if (!stopped) {
                  enableConcurrent = true
                }
              }
            }

          case OnStop =>
            val activeThreads = inbox.synchronized { inbox.numActiveThreads }
            assert(activeThreads == 1,
              s"There should be only a single active thread but found $activeThreads threads.")
            dispatcher.removeRpcEndpointRef(endpoint)
            endpoint.onStop()
            assert(isEmpty, "OnStop should be the last message")

          case RemoteProcessConnected(remoteAddress) =>
            endpoint.onConnected(remoteAddress)

          case RemoteProcessDisconnected(remoteAddress) =>
            endpoint.onDisconnected(remoteAddress)

          case RemoteProcessConnectionError(cause, remoteAddress) =>
            endpoint.onNetworkError(cause, remoteAddress)
        }
      }

      inbox.synchronized {
        // "enableConcurrent" will be set to false after `onStop` is called, so we should check it
        // every time.
        if (!enableConcurrent && numActiveThreads != 1) {
          // If we are not the only one worker, exit
          numActiveThreads -= 1
          return
        }
        message = messages.poll()
        if (message == null) {
          numActiveThreads -= 1
          return
        }
      }
    }
  }

def post(message: InboxMessage): Unit = inbox.synchronized {
    if (stopped) {
      // We already put "OnStop" into "messages", so we should drop further messages
      onDrop(message)
    } else {
      messages.add(message)
      false
    }
  }

  def stop(): Unit = inbox.synchronized {
    // The following codes should be in `synchronized` so that we can make sure "OnStop" is the last
    // message
    if (!stopped) {
      // We should disable concurrent here. Then when RpcEndpoint.onStop is called, it's the only
      // thread that is processing messages. So `RpcEndpoint.onStop` can release its resources
      // safely.
      enableConcurrent = false
      stopped = true
      messages.add(OnStop)
      // Note: The concurrent events in messages will be processed one by one.
    }
  }

  def isEmpty: Boolean = inbox.synchronized { messages.isEmpty }

  /**
   * Called when we are dropping a message. Test cases override this to test message dropping.
   * Exposed for testing.
   */
  protected def onDrop(message: InboxMessage): Unit = {
    logWarning(s"Drop $message because endpoint $endpointName is stopped")
  }

  /**
   * Calls action closure, and calls the endpoint's onError function in the case of exceptions.
   */
  private def safelyCall(endpoint: RpcEndpoint)(action: => Unit): Unit = {
    def dealWithFatalError(fatal: Throwable): Unit = {
      inbox.synchronized {
        assert(numActiveThreads > 0, "The number of active threads should be positive.")
        // Should reduce the number of active threads before throw the error.
        numActiveThreads -= 1
      }
      logError(s"An error happened while processing message in the inbox for $endpointName", fatal)
      throw fatal
    }

    try action catch {
      case NonFatal(e) =>
        try endpoint.onError(e) catch {
          case NonFatal(ee) =>
            if (stopped) {
              logDebug("Ignoring error", ee)
            } else {
              logError("Ignoring error", ee)
            }
          case fatal: Throwable =>
            dealWithFatalError(fatal)
        }
      case fatal: Throwable =>
        dealWithFatalError(fatal)
    }
  }

可以看到这个Inbox内部的具体数据类型是存储信息的messages是一个链表，然后依靠process方法，不断的获取信息进行处理。用了scala中的match表达式。

还要注意的是，一个inbox被创建的时候必然会有OnStart方法被调用，也就会被加入messages中。

主要是要理解inbox的process方法

这个 process 方法是一个用于处理消息并且与 RpcEndpoint 进行交互的关键方法，通常是在处理 RPC 消息时使用。它的主要功能是在接收到消息时对消息进行处理，并根据消息类型调用不同的 RpcEndpoint 方法。它还管理线程的活跃状态，并确保在消息处理过程中线程安全。

本节的任务重点理解上面的架构图。

今天暂时更新到这了，喜欢的小伙伴们，点赞关注哦，谢谢