HDFS报错--org.apache.hdfs.BlockMissingException

#问题：访问HDFS文件的时候报错org.apache.hdfs.BlockMissingException

#问题分析：目前博主遇到的问题有两种

一：问题分析

A：存放指定块的DateNode进程都死了，客户端的请求没有响应肯定获取不到HDFS块

#解决办法：首先可以在yarn的管理界面上查看集群的状况，那些Datenode节点死了。然后去对应的主机把进程起起来

下面是博主自己的本机的环境，供参考

B：遇到BlockMissingException的第二个原因--在客户端向Namenode提交存储任务，对应负责存储响应文件的datanode出现断电。导致块没有真正存储！

#解决办法，参考公司大牛写的hdfstool中的代码，内部其实就是调用了DistributedFileSystem中的recoverLease()方法

二：DistributeFileSystem源码分析

/** 
 * Start the lease recovery of a file
 *
 * @param f a file
 * @return true if the file is already closed
 * @throws IOException if an error occurs
 */
public boolean recoverLease(final Path f) throws IOException {
  Path absF = fixRelativePart(f);
  return new FileSystemLinkResolver<Boolean>() {
    @Override
    public Boolean doCall(final Path p)
        throws IOException, UnresolvedLinkException {
      return dfs.recoverLease(getPathName(p));
    }
    @Override
    public Boolean next(final FileSystem fs, final Path p)
        throws IOException {
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem myDfs = (DistributedFileSystem)fs;
        return myDfs.recoverLease(p);
      }
      throw new UnsupportedOperationException("Cannot recoverLease through" +
          " a symlink to a non-DistributedFileSystem: " + f + " -> " + p);
    }
  }.resolve(this, absF);
}

上面是recoverLease()方法的代码，该方法的作用就是判断指定的文件是否是已经关闭的的如果正常关闭返回true,否则返回false

内部原理还没搞清楚，得进一步深入理解

主要的代码都在FileSystemLinkResolver抽象类中，该类的主要作用如下：

public abstract class FileSystemLinkResolver<T> {

  /**
   * FileSystem subclass-specific implementation of superclass method.
   * Overridden on instantiation to perform the actual method call, which throws
   * an UnresolvedLinkException if called on an unresolved {@link Path}.
   * @param p Path on which to perform an operation
   * @return Generic type returned by operation
   * @throws IOException
   * @throws UnresolvedLinkException
   */
  abstract public T doCall(final Path p) throws IOException,
      UnresolvedLinkException;

  /**
   * Calls the abstract FileSystem call equivalent to the specialized subclass
   * implementation in {@link #doCall(Path)}. This is used when retrying the
   * call with a newly resolved Path and corresponding new FileSystem.
   * 
   * @param fs
   *          FileSystem with which to retry call
   * @param p
   *          Resolved Target of path
   * @return Generic type determined by implementation
   * @throws IOException
   */
  abstract public T next(final FileSystem fs, final Path p) throws IOException;

  /**
   * Attempt calling overridden {@link #doCall(Path)} method with
   * specified {@link FileSystem} and {@link Path}. If the call fails with an
   * UnresolvedLinkException, it will try to resolve the path and retry the call
   * by calling {@link #next(FileSystem, Path)}.
   * @param filesys FileSystem with which to try call
   * @param path Path with which to try call
   * @return Generic type determined by implementation
   * @throws IOException
   */
  public T resolve(final FileSystem filesys, final Path path)
      throws IOException {
    int count = 0;
    T in = null;
    Path p = path;
    // Assumes path belongs to this FileSystem.
    // Callers validate this by passing paths through FileSystem#checkPath
    FileSystem fs = filesys;
    for (boolean isLink = true; isLink;) {
      try {
        in = doCall(p);
        isLink = false;
      } catch (UnresolvedLinkException e) {
        if (!filesys.resolveSymlinks) {
          throw new IOException("Path " + path + " contains a symlink"
              + " and symlink resolution is disabled ("
              + CommonConfigurationKeys.FS_CLIENT_RESOLVE_REMOTE_SYMLINKS_KEY
              + ").", e);
        }
        if (!FileSystem.areSymlinksEnabled()) {
          throw new IOException("Symlink resolution is disabled in" +
              " this version of Hadoop.");
        }
        if (count++ > FsConstants.MAX_PATH_LINKS) {
          throw new IOException("Possible cyclic loop while " +
                                "following symbolic link " + path);
        }
        // Resolve the first unresolved path component
        p = FSLinkResolver.qualifySymlinkTarget(fs.getUri(), p,
            filesys.resolveLink(p));
        fs = FileSystem.getFSofPath(p, filesys.getConf());
        // Have to call next if it's a new FS
        if (!fs.equals(filesys)) {
          return next(fs, p);
        }
        // Else, we keep resolving with this filesystem
      }
    }
    // Successful call, path was fully resolved
    return in;
  }
}