[solved] INFO hdfs.DFSClient: Could not obtain block

最新推荐文章于 2024-11-18 14:59:36 发布

原创最新推荐文章于 2024-11-18 14:59:36 发布 · 963 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#大数据 #scala #java

hadoop 同时被 2 个专栏收录

42 篇文章

订阅专栏

hdfs

6 篇文章

订阅专栏

本文介绍了解决Hadoop HDFS数据节点无法提供足够文件服务的问题。通过调整Hadoop配置文件hdfs-site.xml中的参数dfs.datanode.max.xcievers，可以增加数据节点能够同时服务的文件数量。

部署运行你感兴趣的模型镜像

问题：
10/12/08 20:10:31 INFO hdfs.DFSClient: Could not obtain block blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException: No live nodes contain current block

解决：
An Hadoop HDFS datanode has an upper bound on the number of files that it will serve at any one time. The upper bound parameter is called xcievers (yes, this is misspelled). Again, before doing any loading, make sure you have configured Hadoop's conf/hdfs-site.xml setting the xceivers value to at least the following:

<property>
<name>dfs.datanode.max.xcievers</name>
<value>4096</value>
</property>

Be sure to restart your HDFS after making the above configuration.

您可能感兴趣的与本文相关的镜像