Hadoop NameNode启动之DecommissionManager$Monitor（八）_namenode decommissionmonitor-优快云博客

本文链接：https://blog.youkuaiyun.com/lihm0_1/article/details/10587679

本文详细解析了HDFS集群中节点退役的检查过程，包括节点退役的条件、检查间隔、内部检查函数以及复制进度的检查。通过DecommissionManager的内部类Monitor实现节点退役的监控。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

该线程负责节点退役的检查，HDFS集群内的节点是不能随便退役的，因为DD退役会造成某些文件的副本数下降，所以在退役前会把这个节点上的block复制到其他节点，然后才能退役。

这些检查由DecommissionManager的内部类Monitor来实现

public void run() {
      for(; fsnamesystem.isRunning(); ) {
        synchronized(fsnamesystem) {
          check();
        }
        try {
          Thread.sleep(recheckInterval);//检测间隔，对应参数dfs.namenode.decommission.interval
        } catch (InterruptedException ie) {
          LOG.info("Interrupted " + this.getClass().getSimpleName(), ie);
        }
      }
}

主要看check函数的实现

private void check() {
      int count = 0;
      //循环检测dd状态，注意CyclicIteration数据结构，是环形的
      for(Map.Entry<String, DatanodeDescriptor>entry
          : new CyclicIteration<String,DatanodeDescriptor>(
              fsnamesystem.datanodeMap, firstkey)) {
        final DatanodeDescriptor d = entry.getValue();
        firstkey = entry.getKey();
        //如果正在退役，则进行内部检查
        if (d.isDecommissionInProgress()) {
          try {
            fsnamesystem.checkDecommissionStateInternal(d);
          } catch(Exception e) {
            LOG.warn("entry=" + entry, e);
          }
          if (++count == numNodesPerCheck) {
            return;
          }
        }
      }
}

内部检查函数checkDecommissionStateInternal如下

boolean checkDecommissionStateInternal(DatanodeDescriptor node) {
//如果节点正在退役，并且该节点没有块复制的需求，那么他可以直接退役了
//因为他上面的数据已经没啥价值了
    if (node.isDecommissionInProgress()) {
      if (!isReplicationInProgress(node)) {
        node.setDecommissioned();
        LOG.info("Decommission complete for node " + node.getName());
      }
    }
    if (node.isDecommissioned()) {
      return true;
    }
    return false;
}

下面看下检查复制进度的函数isReplicationInProgress，这是个大块头

private boolean isReplicationInProgress(DatanodeDescriptorsrcNode) {
    boolean status = false;
    int underReplicatedBlocks = 0;
    int decommissionOnlyReplicas = 0;
    int underReplicatedInOpenFiles = 0;
 
    for(final Iterator<Block> i = srcNode.getBlockIterator(); i.hasNext(); ) {
      final Block block = i.next();
      INode fileINode = blocksMap.getINode(block);
 
      if (fileINode != null) {
        NumberReplicas num = countNodes(block);
       //当前副本数
        int curReplicas = num.liveReplicas();
        //目标副本数
        int curExpectedReplicas = getReplication(block);
        if (curExpectedReplicas > curReplicas) {
          // Log info about one block for this nodewhich needs replication
          if (!status) {
            status = true;
            //日志当中记录一把
            logBlockReplicationInfo(block,srcNode, num);
          }
          underReplicatedBlocks++;
          if ((curReplicas == 0) &&(num.decommissionedReplicas() > 0)) {
            decommissionOnlyReplicas++;
          }
          if (fileINode.isUnderConstruction()) {
            underReplicatedInOpenFiles++;
          }
 
          if (!neededReplications.contains(block) &&
            pendingReplications.getNumReplicas(block) == 0) {
            //如果这个块还没被复制则加入neededReplications队列
            neededReplications.add(block,
                                   curReplicas,
                                  num.decommissionedReplicas(),
                                  curExpectedReplicas);
          }
        }
      }
    }//根据上面的结果，设置退役状态

    srcNode.decommissioningStatus.set(underReplicatedBlocks,
        decommissionOnlyReplicas,underReplicatedInOpenFiles);
 
    return status;
  }