hbase集群错误：WARN [Thread-15] hdfs.DFSClient: DataStreamer Exception

最新推荐文章于 2021-06-21 19:32:19 发布
原创 最新推荐文章于 2021-06-21 19:32:19 发布 · 3k 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#错误记录

在运行HBase集群时遇到错误，提示/hbase/.tmp/hbase.version文件只能复制到0个节点，而最小复制数为1。由于没有datanode正在运行，导致该操作失败。通过检查web集群管理页面，发现DataNode处于Dead状态，尝试通过stop-dfs.sh和start-dfs.sh重启集群来解决问题。如果问题依旧，可能是HDFS搭建存在问题。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1562)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3245)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:663)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

最低0.47元/天 解锁文章

200万优质内容无限畅学
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No leas e on
mtj66的博客，交流WX:SpringBreeze1104
08-10 1248
异常信息： 13/09/11 12:12:06 INFO hdfs.DFSClient: SMALL_BUFFER_SIZE is 512 org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No leas e on /tmp/put_dir/20
hadoop：hbase：apache-phoenix连接hbase，实现类sql查询（附DBeaver连接方式）
不花的花和尚的博客
08-21 2622
学校
WARN hdfs.DataStreamer: Caught exception java.lang.InterruptedException
BigData
07-20 1858
19/07/20 19:20:56 WARN hdfs.DataStreamer: Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1245) at java.lang.Thread.joi...
HDFS上传文件出现WARN hdfs.DataStreamer: Caught exception java.lang.InterruptedException
李子翔
06-06 2256
WARN hdfs.DataStreamer: Caughtexception java.lang.InterruptedException hadoop-2.8.5上传文件到hdfs会出现中断异常，这是这个软件本身的bug,上传文件少的话不碍事，报错也无所谓，一样能上传成功。上传文件大且多的话，可能有点痛苦了，要等很久… ...
WARN org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Region was hijacked
hua840812的专栏
04-15 2403
转自：http://mail-archives.apache.org/mod_mbox/hbase-issues/201112.mbox/%3C7071407.41918.1324627472698.JavaMail.tomcat@hel.zones.apache.org%3E Two adjacent assignments lead region is in PENDING_OPEN s
WARN hdfs.DataStreamer: Caught exception
weixin_30496431的博客
09-22 2498
在向hdfs上传文件的时候，报了这么一个错： Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): WARN hdfs.Data...
【Bug记录】org.apache.hadoop.hdfs.server.namenode.SafeModeException
12-27 1830
情况的发生启动hadoop,查看jps所有进程都是正常运行。 输入hive报错： biglucky@ubuntu:/opt/hadoop-2.7.1/sbin$ hive SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/l
[root@hadoop01 apache-hive-3.1.3-bin]# bin/hive which: no hbase in (/export/servers/hadoop-3.3.5/bin::/export/servers/apache-hive-3.1.3-bin/bin:/export/servers/flume-1.9.0/bin::/export/servers/apache-hive-3.1.3-bin/bin:/export/servers/flume-1.9.0/bin:/export/servers/flume-1.9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/root/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/root/bin) 2025-06-17 19:17:36,429 INFO conf.HiveConf: Found configuration file file:/export/servers/apache-hive-3.1.3-bin/conf/hive-site.xml Hive Session ID = 174edc05-5da2-4b28-a455-d29ac7bdd8fc 2025-06-17 19:17:39,510 INFO SessionState: Hive Session ID = 174edc05-5da2-4b28-a455-d29ac7bdd8fc Logging initialized using configuration in jar:file:/export/servers/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true 2025-06-17 19:17:39,639 INFO SessionState: Logging initialized using configuration in jar:file:/export/servers/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true 2025-06-17 19:17:42,218 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/174edc05-5da2-4b28-a455-d29ac7bdd8fc 2025-06-17 19:17:42,262 INFO session.SessionState: Created local directory: /tmp/root/174edc05-5da2-4b28-a455-d29ac7bdd8fc 2025-06-17 19:17:42,274 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/174edc05-5da2-4b28-a455-d29ac7bdd8fc/_tmp_space.db 2025-06-17 19:17:42,304 INFO conf.HiveConf: Using the default value passed in for log id: 174edc05-5da2-4b28-a455-d29ac7bdd8fc 2025-06-17 19:17:42,304 INFO session.SessionState: Updating thread name to 174edc05-5da2-4b28-a455-d29ac7bdd8fc main 2025-06-17 19:17:44,440 INFO metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore 2025-06-17 19:17:44,512 WARN metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2025-06-17 19:17:44,530 INFO metastore.ObjectStore: ObjectStore, initialize called 2025-06-17 19:17:44,532 INFO conf.MetastoreConf: Found configuration file file:/export/servers/apache-hive-3.1.3-bin/conf/hive-site.xml 2025-06-17 19:17:44,534 INFO conf.MetastoreConf: Unable to find config file hivemetastore-site.xml 2025-06-17 19:17:44,534 INFO conf.MetastoreConf: Found configuration file null 2025-06-17 19:17:44,535 INFO conf.MetastoreConf: Unable to find config file metastore-site.xml 2025-06-17 19:17:44,535 INFO conf.MetastoreConf: Found configuration file null 2025-06-17 19:17:44,958 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored 2025-06-17 19:17:45,507 INFO hikari.HikariDataSource: HikariPool-1 - Starting... 2025-06-17 19:17:46,120 INFO hikari.HikariDataSource: HikariPool-1 - Start completed. 2025-06-17 19:17:46,244 INFO hikari.HikariDataSource: HikariPool-2 - Starting... 2025-06-17 19:17:46,262 INFO hikari.HikariDataSource: HikariPool-2 - Start completed. 2025-06-17 19:17:46,944 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 2025-06-17 19:17:47,177 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL 2025-06-17 19:17:47,179 INFO metastore.ObjectStore: Initialized ObjectStore 2025-06-17 19:17:47,661 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:47,662 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:47,664 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:47,664 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:47,665 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:47,665 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,340 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,341 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,342 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,342 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,342 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:51,342 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:17:56,088 WARN metastore.ObjectStore: Version information not found in metastore. metastore.schema.verification is not enabled so recording the schema version 3.1.0 2025-06-17 19:17:56,088 WARN metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 3.1.0, comment = Set by MetaStore root@192.168.245.131 2025-06-17 19:17:56,489 INFO metastore.HiveMetaStore: Added admin role in metastore 2025-06-17 19:17:56,497 INFO metastore.HiveMetaStore: Added public role in metastore 2025-06-17 19:17:56,607 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty 2025-06-17 19:17:56,969 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=root (auth:SIMPLE) retries=1 delay=1 lifetime=0 2025-06-17 19:17:57,003 INFO metastore.HiveMetaStore: 0: get_all_functions 2025-06-17 19:17:57,011 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_all_functions Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. 2025-06-17 19:17:57,170 INFO CliDriver: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Hive Session ID = 2804a115-a38a-441b-9d42-6abad28c99f8 2025-06-17 19:17:57,170 INFO SessionState: Hive Session ID = 2804a115-a38a-441b-9d42-6abad28c99f8 2025-06-17 19:17:57,216 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/2804a115-a38a-441b-9d42-6abad28c99f8 2025-06-17 19:17:57,222 INFO session.SessionState: Created local directory: /tmp/root/2804a115-a38a-441b-9d42-6abad28c99f8 2025-06-17 19:17:57,228 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/2804a115-a38a-441b-9d42-6abad28c99f8/_tmp_space.db 2025-06-17 19:17:57,231 INFO metastore.HiveMetaStore: 1: get_databases: @hive# 2025-06-17 19:17:57,231 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_databases: @hive# 2025-06-17 19:17:57,233 INFO metastore.HiveMetaStore: 1: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore 2025-06-17 19:17:57,239 INFO metastore.ObjectStore: ObjectStore, initialize called 2025-06-17 19:17:57,272 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL 2025-06-17 19:17:57,272 INFO metastore.ObjectStore: Initialized ObjectStore 2025-06-17 19:17:57,288 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#db_hive1 pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,288 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#db_hive1 pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,314 INFO metastore.HiveMetaStore: 1: get_multi_table : db=db_hive1 tbls= 2025-06-17 19:17:57,314 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=db_hive1 tbls= 2025-06-17 19:17:57,316 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,316 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,321 INFO metastore.HiveMetaStore: 1: get_multi_table : db=default tbls= 2025-06-17 19:17:57,321 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=default tbls= 2025-06-17 19:17:57,322 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#itcast_ods pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,322 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#itcast_ods pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:17:57,326 INFO metastore.HiveMetaStore: 1: get_multi_table : db=itcast_ods tbls= 2025-06-17 19:17:57,326 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=itcast_ods tbls= 2025-06-17 19:17:57,326 INFO metadata.HiveMaterializedViewsRegistry: Materialized views registry has been initialized hive>
最新发布
06-18
我们正在解决Hive启动时遇到的连接拒绝问题（HDFS9000端口）和配置警告（如HBase未找到、metastore警告）。根据之前的诊断，核心问题是HDFS的NameNode未运行导致端口9000无监听，以及可能的配置不一致。解决步骤将...
hbase shell 操作失败查看日志遇到的一个bug
one city one love
07-24 2181
File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). 这是由于 hadoop 的hdfs系统 中datanode没有与 namenode 连接页产生的，所以需要在namenode的 50070web页面上查看是否有datanode连接。...
异常处理 could only be replicated to 0 nodes, instead of 1
04-17
NULL 博文链接：https://xq0804200134.iteye.com/blog/1814655
hbase错误记录一; File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplica
热门推荐
ZeroTeam_麒麟
03-26 1万+
hbase启动提示org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.错误原因
org.apache.hadoop.ipc.RemoteException(java.io.IOException) File hbaseMast
ddzzz_的博客
04-26 2169
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/Mast Hbase启动时，master或regionserver启动不成功，查看logs目录下.log文件的报错内容 报错内容：org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/MasterProcWALs/state-00000000000000000001.log c
HBase启动之后HMaster自动关闭的问题处理
望月思灵
02-10 5710
查看HBase的日志有报错 2018-02-10 03:42:00,570 WARN [Thread-72] hdfs.DFSClient: DataStreamer Exception org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only ...
Linux系统使用过程中的错误（hadoop，habse）
wang2821的博客
06-21 1660
1.error：jps:command not found 2.error：hdfs:未找到命令 3.error：syntax errer,unexpected tIDENTIFIER ** 解决问题：语法错误，hbase中常见的情况是前后代码中多或少打了一个‘ ** 4.error：ping请求超时 5.error：name node is in safe mode. ** 解决问题：退出安全模式。 进入：hdfs dfsadmin -safemode enter 退出：.
hadoop文件操作错误---org.apache.hadoop.ipc.RemoteException(java.io.IOException)
沐雨金鳞
01-25 4033
使用hadoop dfsadmin -report命令查看磁盘使用情况 如果是下面的情况 Configured Capacity: 0 (0 B) Present Capacity: 0 (0 B) DFS Remaining: 0 (0 B) DFS Used: 0 (0 B) DFS Used%: NaN% Under replicated blocks: 0
HDFS超租约异常总结（org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException）
weixin_30731287的博客
09-12 550
异常信息： 13/09/11 12:12:06 INFO hdfs.DFSClient: SMALL_BUFFER_SIZE is 512 org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No leas e on /tmp/put_dir/...
2025-03-18 14:41:38,515 WARN hdfs.DataStreamer: Exception in createBlockOutputStream blk_1073742742_1918 java.io.IOException: Invalid token in javax.security.sasl.qop: DI at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessage(DataTransferSaslUtil.java:220) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessage(DataTransferSaslUtil.java:237) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:544) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getSaslStreams(SaslDataTransferClient.java:446) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:289) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:236) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.socketSend(SaslDataTransferClient.java:203) at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.socketSend(SaslDataTransferClient.java:193) at org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1780) at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1728) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:713) 2025-03-18 14:41:38,518 WARN hdfs.DataStreamer: Abandoning BP-1654643170-10.4.136.105-1741764777416:blk_1073742742_1918 2025-03-18 14:41:38,523 WARN hdfs.DataStreamer: Excluding datanode DatanodeInfoWithStorage[10.4.136.115:1026,DS-b87f7d43-952f-484b-a735-11c2dfc16dc8,DISK] 2025-03-18 14:41:38,540 WARN hdfs.DataStreamer: Exception in createBlockOutputStream blk_1073742743_1919 java.io.IOException: Invalid token in javax.security.sasl.qop: DI at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.rea
03-19
<think>嗯，用户遇到了Hadoop HDFS中的SASL握手失败问题，具体错误是IOException 'Invalid token in javax.security.sasl.qop'。我需要先理解这个错误的常见原因。根据经验，SASL相关的问题通常与Kerberos配置有关，或者QoP设置不一致。 首先，用户提到可能由于ZooKeeper中的残留文件导致的问题，比如version-2目录。之前引用[4]提到删除version-2并重启后解决了HMaster的问题，可能需要检查ZooKeeper的数据一致性。不过SASL握手失败更可能直接关联到Hadoop和Kerberos的配置。 接下来，检查Kerberos是否配置正确。引用[5]提到Hadoop使用Kerberos进行身份验证，所以需要确认所有节点的Kerberos票据是否有效，时间是否同步，以及jaas.conf文件是否正确。引用[1]和[2]都提到了设置JVMFLAGS指定jaas.conf路径，这可能与SASL配置有关，需要确认这些配置是否存在且路径正确。 然后，QoP（质量保护）设置。错误信息中提到了javax.security.sasl.qop无效，说明客户端和服务端的QoP配置不一致。Hadoop中的配置项如hadoop.rpc.protection、dfs.data.transfer.protection需要检查是否一致，是否在服务端和客户端有相同的值。 另外，SASL相关配置文件的正确性也很重要，比如core-site.xml、hdfs-site.xml中的安全配置是否正确，是否有遗漏的参数。同时，查看日志文件，如HDFS的NameNode和DataNode日志，以及客户端的错误日志，可以帮助定位具体问题。 用户可能遗漏的步骤包括更新配置后未重启服务，或者Kerberos票据过期。需要提醒用户检查kinit是否成功，keytab文件是否正确，并重启相关服务。此外，ZooKeeper的残留数据可能影响协调服务，但在此错误中可能不是主要原因，不过如果问题持续，清理ZooKeeper数据并重启也是一个可能的解决步骤。 最后，综合以上可能原因，分步骤给出解决方案，从检查Kerberos配置、QoP设置、配置文件、日志分析等方面逐一排查，并提醒用户注意配置一致性和服务重启的重要性。</think>针对Hadoop HDFS中出现的`SASL handshake failure IOException 'Invalid token in javax.security.sasl.qop'`问题，以下是系统性解决方案： --- ### **问题原因分析** 该错误通常由以下原因导致： 1. **Kerberos配置不一致**：Hadoop集群未正确配置Kerberos认证，或票据过期[^5]。 2. **QoP（Quality of Protection）参数不匹配**：客户端与服务端的`hadoop.rpc.protection`或`dfs.data.transfer.protection`参数配置不一致。 3. **ZooKeeper残留数据**：若集群依赖ZooKeeper（如HBase），残留的`version-2`目录可能导致协调服务异常[^4]。 --- ### **解决步骤** #### 1. 检查Kerberos配置 - **验证票据有效性**： 执行`klist`检查Kerberos票据是否过期，若过期需重新获取： ```bash kinit -kt <keytab_path> <principal> ``` - **检查`jaas.conf`文件**： 确保所有节点的`jaas.conf`路径正确，并通过环境变量`JVMFLAGS`指定（如引用[1][^1][^2]）： ```bash export JVMFLAGS="-Djava.security.auth.login.config=/path/to/jaas.conf" ``` #### 2. 统一QoP参数 - **修改Hadoop配置文件**： 在`core-site.xml`中设置一致的QoP参数： ```xml <property> <name>hadoop.rpc.protection</name> <value>privacy</value> <!-- 可选值：authentication, integrity, privacy --> </property> ``` 在`hdfs-site.xml`中配置数据传输保护： ```xml <property> <name>dfs.data.transfer.protection</name> <value>privacy</value> </property> ``` - **同步配置到所有节点**： 确保客户端与服务端配置完全一致，并重启HDFS服务。 #### 3. 清理ZooKeeper残留数据 - 停止ZooKeeper服务后，删除`dataDir`目录下的`version-2`和`zookeeper_server.pid`文件（参考引用[4]）： ```bash rm -rf /opt/zookeeper/data/version-2 rm -f /opt/zookeeper/data/zookeeper_server.pid ``` - 重启ZooKeeper服务，生成新数据目录。 #### 4. 验证与日志分析 - **检查HDFS服务日志**： 查看NameNode和DataNode的日志（`logs/hadoop-*-namenode-*.log`），定位具体握手失败的位置。 - **网络与时间同步**： 使用`ntpdate`确保集群节点时间同步，避免Kerberos因时间偏差拒绝认证。 --- ### **相关配置示例** ```bash # 启动HDFS前加载Kerberos配置 export HADOOP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf" hdfs namenode -format start-dfs.sh ``` ---
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值