Hadoop 2.4.1 搭建Ha遇到问题记录

最新推荐文章于 2023-07-21 11:32:40 发布

大彪先生

最新推荐文章于 2023-07-21 11:32:40 发布

阅读量8.8k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： hadoop 文章标签： hadoop hbase hdfs

本文链接：https://blog.youkuaiyun.com/u010811257/article/details/46809935

15/06/19 17:12:44 ERROR namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: Invalid configuration: <span style="background-color: rgb(255, 102, 102);">a shared edits dir must not be specified if HA is not enabled</span>. // 明明我配置了HA，却说没有,下面的配置写错了
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:710)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO namenode.FSNamesystem: Stopping services started for active state
15/06/19 17:12:44 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.stopActiveServices(FSNamesystem.java:1118)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.close(FSNamesystem.java:1446)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:781)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO util.ExitUtil: Exiting with status 1
15/06/19 17:12:44 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v1/192.168.32.91
************************************************************/

<strong>.写成了-</strong>
<property>
                <name>dfs.ha.automatic-failover-enabled</name>
                <value>true</value>
        </property>
正确如下：
<property>
                <name>dfs.ha.automatic-failover.enabled</name>
                <value>true</value>
        </property>

2.journalnode的端口配置错误，导致不能格式化namenode ,journalnode监听的端口是：8485，而不是8445.

3.格式化standby namnenode时，有两种方式

1.将active namenode格式化完成后，将整个hadoop.tmp.dir路径全部拷贝到standby机器上。

2.在active namenode格式化完成后，并且格式化zkfc(只需要在active上格式化一次，standby上的zkfc不需要再次格式化)启动好了namenode之后。在standby namenode上执行:hdfs namenode -bootstrapStandby 也可以.如果active namenode没有启动的话，这里会报错。因为standby namenode是通过active namenode的9000端口通讯的。若active namenode没有启动，则9000没有程序监听提供服务，standby尝试连接9次后会爆异常并且退出。

没有启动active namenode时，报错如下：

[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:28:45 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = v2/192.168.32.92
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.4.1
STARTUP_MSG:   classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/l