hbase启动master挂掉解决的几种办法 | base 和 Hadoop版本问题

原创已于 2022-05-16 02:33:20 修改 · 2.9k 阅读

31 ·

CC 4.0 BY-SA版权

文章标签：

#hbase #hadoop #big data

于 2022-05-16 02:30:16 首次发布

hbase 专栏收录该内容

1 篇文章

订阅专栏

当HBase在启动后，Master节点因文件系统缺少hsync而挂掉。错误源于HBase集群操作框架需要在写前日志中持久保存状态的能力。本文探讨了不同解决办法，包括检查Hadoop和HBase的版本匹配、解决jar包冲突以及日志框架冲突。特别提示，解决方法3中通过替换Hadoop的jar包版本至与HBase相匹配，成功解决了问题。

0.版本信息

hadoop3.1.3

hbase2.0.5

1.错误描述

启动HBase后,master过了几秒就挂了.去log中看发现

java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1083)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:421)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:611)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1407)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:853)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2241)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:567)
	at java.lang.Thread.run(Thread.java:748)
2022-05-16 00:17:15,618 ERROR [Thread-14] master.HMaster: ***** ABORTING master hadoop102,16000,1652631432898: Unhandled exception. Starting shutdown. *****
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1083)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:421)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:611)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1407)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:853)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2241)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:567)
	at java.lang.Thread.run(Thread.java:748)

查看官网

https://hbase.apache.org/book.html#trouble.master.startup.hsync

146.1.3. Master fails to become active due to lack of hsync for filesystem

HBase’s internal framework for cluster operations requires the ability to durably save state in a write ahead log. When using a version of Apache Hadoop Common’s filesystem API that supports checking on the availability of needed calls, HBase will proactively abort the cluster if it finds it can’t operate safely.

For Master roles, the failure will show up in logs like this:

2018-04-05 11:18:44,653 ERROR [Thread-21] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
        at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1034)
        at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:530)
        at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1267)
        at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1173)
        at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:881)
        at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2048)
        at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:568)
        at java.lang.Thread.run(Thread.java:745)

If you are attempting to run in standalone mode and see this error, please walk back through the section Quick Start - Standalone HBase and ensure you have included all the given configuration settings.

官网的给的解决方法没用,因为我跑的是分布式,如果单机模式的兄弟报错可以试试官网的方法.

2.解决方法

解决方法1:

hbase-site.xml增加配置

<property>
  <name>hbase.unsafe.stream.capability.enforce</name>
  <value>false</value>
</property>

但这种方法有丢数据的风险,生产环境不建议.

解决方法2

检查一下hbase 和hadoop的版本是否为官网推荐版本

https://hbase.apache.org/book.html#basic.prerequisites

如果不是可以下载对应版本.

但我装了新版的hbase,后phoenix也需要升级成对应版本.但新版的phoenix比起5.0.0版的少别的组件的jar包和瘦客户端,所以我就没升级.

解决方法3

自己解决jar包冲突问题,先检查原版hbase lib目录下Hadoop的jar包版本

替换的jar包可以去/opt/module/hadoop-3.1.3/share/hadoop/ 自己hadoop中有,

也可以去Hbase官网上下符合你hadoop版本的hbase 之后到这个新版的hbase /opt/module/hbase/lib/ 下拿

2.0.5版本hbse为2.7.7版本的Hadoop的jar包,比我3.1.3的hadoop低级,所以要替换为3.1.3hadoop版本的jar包

rm /opt/module/hbase/lib/hadoop-*

然后复制jar包到/opt/module/hbase/lib/

最后分发到别的机子上

在这里插入图片描述

或者也可以替换成对应符合你hadoop版本的hbase的jar包

比如我的是3.1.3 去官网下2.4.12的hbase 复制hadoop的jar包到 /opt/module/hbase/lib/

完美解决!!!

还没有解决就检查一下配置hdfs的地址是否写错,自己看一下报错信息.

3.lo4j冲突问题

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
running master, logging to /opt/module/hbase/logs/hbase-atguigu-master-hadoop102.out
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

删除/opt/module/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar

一定要保留Hadoop的