hadoop2 集群出现过的问题-优快云博客

在Hadoop2集群中遇到了两个问题：1) 数据节点数量达到最小值0，通过执行'hadoop dfsadmin -safemode leave'命令退出安全模式解决；2) 提交任务时遇到内存请求超出配置限制，解决方案是增加虚拟机内存至至少1.5GB，推荐使用2GB内存的虚拟机。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1 The number of live datanodes 2 has reached the minimum number 0

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot delete /tmp/hadoop-yarn/staging/root/.staging/job_1434694058996_0001. Name node is in safe mode.
The reported blocks 244 has reached the threshold 0.9990 of total blocks 244. The number of live datanodes 2 has reached the minimum number 0. Safe mode will be turned off automatically in 3 seconds.
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1130)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:3247)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInt(FSNamesystem.java:3207)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3191)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:706)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:514)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1962)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1958)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1956)

处理方式：

可以关闭安全模式

[root@master mapreduce]# hadoop dfsadmin -safemode leave

2 Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException): Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=1536, maxMemory=1024

处理方式： 1 虚拟机内存弄大点，建议每个节点最少为1.5G以上，这里我用的是2G的内存虚拟机