Caused by: java.lang.IllegalArgumentException: bad journal id: qjournal://

最新推荐文章于 2024-05-11 15:17:46 发布

mtj66

最新推荐文章于 2024-05-11 15:17:46 发布

阅读量2.3k

点赞数

分类专栏： CDH

本文链接：https://blog.youkuaiyun.com/mtj66/article/details/90269334

版权

CDH 专栏收录该内容

18 篇文章

订阅专栏

CM 6.2.0 cdh hdfs 配置HA方法:

CDH 5.0 和更高版本不支持使用共享 NFS 挂载的 HDFS High Availability。建议您将 HDFS HA 设置切换为 Quorum-based Storage。必须将 HDFS HA 设置切换为 Quorum-based Storage，然后再尝试升级至 CDH 5。
Suppress... 一直是个警告,真的是很烦啊.看看官方文档吧.

官方参考:https://archive.cloudera.com/cdh6/6.2.0/docs/hadoop-3.0.0-cdh6.2.0/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html

重点配置: quorum journal

不过上面是采用手动配置hdfs-site.xml,但是我们采用 CM进行配置HA:所有的都配置好了啊,为啥不行呢

停止两个namenode ,将备用namenode的节点的

1. 初始化 jn的路径 ,应该会自动启动namenode (否则自己手动启动)

2.引导standby namenode,引导的时候将 standby节点的元数据文件清空,

上面我都是根据CM的日志试出来的,如果出错,自己根据日志尝试解决.

rm -rf /alidata1/hdfs/dfs/editsdir/current/*   
rm -rf /alidata1/hdfs/dfs/nn/current/*

注意:我之前采用手动拷贝nn的元数据到standby节点,还是不行,要按照上面的操作

[root@hzd-t-vbdl-02 dfs]# more  nn/current/VERSION 
#Thu May 16 19:08:04 CST 2019
namespaceID=855951961
clusterID=cluster54
cTime=1557219668083
storageType=NAME_NODE
blockpoolID=BP-1004949537-10.253.76.213-1557219668083
layoutVersion=-64
[root@hzd-t-vbdl-02 dfs]# ssh hzd-t-vbdl-01
Last login: Thu May 16 18:49:27 2019 from 10.253.11.199

Welcome to Alibaba Cloud Elastic Compute Service !

[root@hzd-t-vbdl-01 ~]# cd /alidata1/hdfs/dfs
[root@hzd-t-vbdl-01 dfs]# more nn/current/VERSION 
#Tue May 07 17:01:08 CST 2019
namespaceID=855951961
clusterID=cluster54
cTime=1557219668083
storageType=NAME_NODE
blockpoolID=BP-1004949537-10.253.76.213-1557219668083
layoutVersion=-64
[root@hzd-t-vbdl-01 dfs]#

配置下 Quorum Journal:qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/namenodens (这个是配置到hdfs-site.xml的格式,但是在CM中配置会报错的)

同时取消配置共享编辑目录 :dfs.namenode.shared.edits.dir

Failed to start namenode.
java.lang.IllegalArgumentException: Unable to construct journal, qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/namenodens
   at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1824)
   at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:296)
   at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:274)
   at org.apache.hadoop.hdfs.server.namenode.FSImage.initEditLog(FSImage.java:841)
   at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:683)
   at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:323)
   at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1084)
   at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:709)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:665)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:727)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:950)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:929)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1653)
   at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1720)
Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1811)
   ... 13 more
Caused by: java.lang.IllegalArgumentException: bad journal id: qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/namenodens
   at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.checkJournalId(QuorumJournalManager.java:177)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.parseJournalId(QuorumJournalManager.java:172)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.createLoggers(QuorumJournalManager.java:386)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.createLoggers(QuorumJournalManager.java:163)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.<init>(QuorumJournalManager.java:126)
   at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.<init>(Q


   Exiting with status 1: java.lang.IllegalArgumentException: Unable to construct journal, qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/namenodens

从上面的报错日志判断可能是Quorum Journal名称配置错误,检查CM 的hdfs配置发现之前填写的是:qjournal://gyb-t-vbdl-01:8485;gyb-t-vbdl-02:8485;gyb-t-vbdl-03:8485;hzd-t-vbdl-01:8485;hzd-t-vbdl-02:8485/namenodens

明显多了点啥,修改后如下

正确配置好HA之后,触发日志如下,之前一直卡在:

Triggering log roll on remote NameNode 
A checkpoint was triggered but the Standby Node has not received any transac

晚上7点19:06.885分	INFO	EditLogTailer	
Triggering log roll on remote NameNode
晚上7点19:06.932分	INFO	FSImage	
Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@7ab17db1 expecting start txid #272598
晚上7点19:06.932分	INFO	FSImage	
Start loading edits file http://gyb-t-vbdl-03:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true maxTxnsToRead = 9223372036854775807
晚上7点19:06.933分	INFO	RedundantEditLogInputStream	
Fast-forwarding stream 'http://gyb-t-vbdl-03:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true' to transaction ID 272598
晚上7点19:06.933分	INFO	RedundantEditLogInputStream	
Fast-forwarding stream 'http://gyb-t-vbdl-03:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true' to transaction ID 272598
晚上7点19:07.019分	INFO	FSImage	
Edits file http://gyb-t-vbdl-03:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272598&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true of size 1699 edits # 15 loaded in 0 seconds
晚上7点21:07.055分	INFO	EditLogTailer	
Triggering log roll on remote NameNode
晚上7点21:07.114分	INFO	FSImage	
Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@42ebc6ac expecting start txid #272613
晚上7点21:07.115分	INFO	FSImage	
Start loading edits file http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://gyb-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true maxTxnsToRead = 9223372036854775807
晚上7点21:07.115分	INFO	RedundantEditLogInputStream	
Fast-forwarding stream 'http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://gyb-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true' to transaction ID 272613
晚上7点21:07.115分	INFO	RedundantEditLogInputStream	
Fast-forwarding stream 'http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true' to transaction ID 272613
晚上7点21:07.119分	INFO	FSImage	
Edits file http://hzd-t-vbdl-02:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://hzd-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true, http://gyb-t-vbdl-01:8480/getJournal?jid=namenodens&segmentTxId=272613&storageInfo=-64%3A855951961%3A1557219668083%3Acluster54&inProgressOk=true of size 1591 edits # 14 loaded in 0 seconds

参考:https://cloud.tencent.com/developer/article/1174093