HDFS EC Reconstruction

最新推荐文章于 2025-09-16 19:32:28 发布

原创

最新推荐文章于 2025-09-16 19:32:28 发布 · 760 阅读

3 ·

CC 4.0 BY-SA版权

本文深入剖析HDFS中Erasure Coding (EC) 重构机制的实现细节，从DN接收到NN的BlockECReconstructionCommand命令开始，详细介绍如何启动重构过程，包括线程池的配置与任务调度。

hdfs ec重构块的代码设计巧妙，本文总结其设计思想。
下文先分析DN重构块的流程。
DN收到NN下达的命令后，判断如果是 BlockECReconstructionCommand 命令，DN则开始重构工作。

// BPOfferService.java
case DatanodeProtocol.DNA_ERASURE_CODING_RECONSTRUCTION:
  LOG.info("DatanodeCommand action: DNA_ERASURE_CODING_RECOVERY");
  Collection<BlockECReconstructionInfo> ecTasks =
      ((BlockECReconstructionCommand) cmd).getECTasks(); //1
  dn.getErasureCodingWorker().processErasureCodingTasks(ecTasks); //2
  break;

我们看一下重构的对象 Collection ecTasks 的结构，包括块组id和源及目标DN信息等：

public static class BlockECReconstructionInfo {
   
   
 private final ExtendedBlock block;
 private final DatanodeInfo[] sources;
 private DatanodeInfo[] targets;
 private String[] targetStorageIDs;
 private StorageType[] targetStorageTypes;
 private final byte[] liveBlockIndices;
 private final ErasureCodingPolicy ecPolicy;
 ...

dn.getErasureCodingWorker()获得 private ErasureCodingWorker ecWorker; 那么，ecWorker是一个什么角色呢？我们看定义：

/**
 * ErasureCodingWorker handles the erasure coding reconstruction work commands.
 * These commands would be issued from Namenode as part of Datanode's heart beat
 * response. BPOfferService delegates the work to this class for handling EC
 * commands.
 */
public final class ErasureCodingWorker {
   
   
  private static final Logger LOG = DataNode.LOG;
 
  private final DataNode datanode;
  private final Configuration conf;
  private final float xmitWeight;
 
  private ThreadPoolExecutor stripedReconstructionPool;
  private ThreadPoolExecutor stripedReadPool;

ErasureCodingWorker 用于处理EC的重构命令，该命令是NN给DN的心跳回复。

ECWorker在整个架构中的角色如下图所示，用于服务DN有关块构建恢复工作。
在这里插入图片描述

线程池篇

我们看一下ecWorker中的两个线程池：

private ThreadPoolExecutor stripedReconstructionPool;
private ThreadPoolExecutor stripedReadPool;

看线程池的初始化：

  // ErasureCodingWorker#initializeStripedReadThreadPool
  private void initializeStripedReadThreadPool() {
   
   
    LOG.debug("Using striped reads");

    // Essentially, this is a cachedThreadPool.
    stripedReadPool = new ThreadPoolExecutor(0, Integer.MAX_VALUE