java.lang.OutOfMemoryError: unable to create new native thread

最新推荐文章于 2025-05-21 09:58:43 发布

原创最新推荐文章于 2025-05-21 09:58:43 发布 · 1.2k 阅读

0 ·

CC 4.0 BY-SA版权

hadoop 专栏收录该内容

52 篇文章

订阅专栏

本文记录了Hadoop集群中出现的内存溢出错误及连接被拒绝的问题，详细展示了错误日志，并探讨了解决这些问题的方法，包括调整ulimit-u参数等。

35227 2014-05-21 13:53:18,504 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Reopen already-open Block for append blk_8901346392456488003_201326
135228 2014-05-21 13:53:18,506 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.1.33.11:50010, storageID=DS-420686803-10.1.33.11-50010-1399181350037, infoPort=50       075, ipcPort=50020):DataXceiver
135229 java.lang.OutOfMemoryError: unable to create new native thread
135230     at java.lang.Thread.start0(Native Method)
135231     at java.lang.Thread.start(Thread.java:714)
135232     at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:576)
135233     at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:404)
135234     at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:112)
135235     at java.lang.Thread.run(Thread.java:745)
135236 2014-05-21 13:53:25,866 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.1.33.11:50010, storageID=DS-420686803-10.1.33.11-50010-1399181350037, infoPort=500       75, ipcPort=50020) Starting thread to transfer blk_-2263414036967179224_200863 to 10.1.33.13:50010
135237 2014-05-21 13:53:25,866 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.1.33.11:50010, storageID=DS-420686803-10.1.33.11-50010-1399181350037, infoPort=500       75, ipcPort=50020) Starting thread to transfer blk_-2231241119796918398_200963 to 10.1.33.15:50010
135238 2014-05-21 13:53:25,877 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.1.33.11:50010, storageID=DS-420686803-10.1.33.11-50010-1399181350037, infoPort=500       75, ipcPort=50020):Failed to transfer blk_-2263414036967179224_200863 to 10.1.33.13:50010 got java.net.ConnectException: Connection refused
135239     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
135240     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
135241     at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
135242     at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)
135243     at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)
135244     at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1511)
135245     at java.lang.Thread.run(Thread.java:745)
135246
135247 2014-05-21 13:53:25,877 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: checkDiskError: exception:
135248 java.net.ConnectException: Connection refused
135249     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
135250     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
135251     at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
135252     at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)
135253     at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)
135254     at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1511)
135255     at java.lang.Thread.run(Thread.java:745)
135256 2014-05-21 13:53:25,877 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Not checking disk as checkDiskError was called on a network related exception
135257 2014-05-21 13:53:25,877 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode

ulimit -u 参数调大