积累hadoop错误，杂乱啊！

最新推荐文章于 2025-08-08 22:01:10 发布

Sweblish

最新推荐文章于 2025-08-08 22:01:10 发布

阅读量2.3k

点赞数

CC 4.0 BY-SA版权

分类专栏： hadoop 文章标签： hadoop exception java file 测试存储

本文链接：https://blog.youkuaiyun.com/Sweblish/article/details/7613270

hadoop 专栏收录该内容

2 篇文章

订阅专栏

本文介绍了Hadoop伪分布式环境的搭建步骤，并详细记录了一次部署过程中遇到的文件复制错误及其解决方法。通过调整配置文件及重新格式化名称节点，成功解决了问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

2012-05-29 16:14:13,885 WARN org.apache.hadoop.mapred.JobTracker: Retrying...
2012-05-29 16:14:13,892 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
   at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)

   at org.apache.hadoop.ipc.Client.call(Client.java:1066)
   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
   at $Proxy5.addBlock(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
   at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
   at $Proxy5.addBlock(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3507)
   at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3370)
   at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2700(DFSClient.java:2586)
   at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2826)

出现如此的错误，因为是自己测试用的，所以狠一点，将两台机器上存储hadoop文件的地方都删掉了。

然后 hadoop namenode -format

然后 bin/start-all.sh

伪分布式安装：

1.要确定可以ssh localhost，apt-get install openssh-server

2.4个配置文件要动

hadoop-env.sh 增加 JAVA_HOME=

core-site.xml 设定namenode：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000/</value>
    </property>

    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/hadoop/data/</value>
    </property>
</configuration>

hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    
    <property>  
        <name>dfs.name.dir</name>
        <value>/home/wangxing/hadoop-0.20.2/tmpdir/hdfs/name</value>  
    </property>  
    
    <property>  
        <name>dfs.data.dir</name>  
        <value>/home/hadoop/data/hdfs/data</value>  
    </property>
</configuration>

mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:9001</value>
    </property>

    <property>  
        <name>mapred.local.dir</name>  
        <value>/home/hadoop/data/mapred/local</value>  
    </property>  
    
    <property>  
        <name>mapred.system.dir</name>  
        <value>/home/hadoop/data/mapred/system</value>  
    </property>

</configuration>

#bin/hadoop namenode -format