could only be replicated to 1 nodes, instead of 2

最新推荐文章于 2019-08-02 14:12:01 发布

weixin_30569033

最新推荐文章于 2019-08-02 14:12:01 发布

阅读量95

点赞数

CC 4.0 BY-SA版权

文章标签： java 大数据运维

原文链接：http://www.cnblogs.com/bokun-wang/archive/2013/04/22/3035876.html

本文记录了一次Hadoop文件系统中文件复制失败的问题排查过程，包括检查防火墙设置、磁盘空间、DataNode状态及SafeMode等步骤。

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /smp/kpi/1366560000/_job_201304221109_0031_kpi_temporary/_attempt_201304221109_0031_r_000003_0/kpi_60/1366599600-1366603200-ci_failure_60-1.kpi could only be replicated to 1 nodes, instead of 2
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1322)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:453)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:962)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:958)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:956)

at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3001)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2880)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2200(DFSClient.java:2157)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2343)

第一个问题“could only be replicated to 0 nodes, instead of 1 ”是怎么解决的？

6.1.1 是否是防火墙未关闭，查看。确实忘记关闭防火墙了，因为我换了几台机器，以前是在虚拟机下用的是redhat，现在用的是ubuntu8.0.4。所以跟这个很可能相关。关闭iptables后，出现的错误信息没那么多了，但是还有错误。如下：
6.1.2 是否还有空间
df –hl
6.1.3 datanode数目是否正常
datanode用jps查看进程
6.1.4 是否在safemode下
hadoop dfsadmin -safemode leave

转载于:https://www.cnblogs.com/bokun-wang/archive/2013/04/22/3035876.html