【排错】Could not reseek StoreFileScanner[HFileScanner for reader reader=hdfs://dmp-nn-1:8020/hbase/...

最新推荐文章于 2025-10-09 10:46:28 发布
原创 最新推荐文章于 2025-10-09 10:46:28 发布 · 3.6k 阅读 · 2 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

在执行HBase查询时遇到'Could not reseek StoreFileScanner'异常，这通常与硬盘故障有关。错误表明在读取HFileBlock时校验失败。解决方法包括关闭有问题的datanode和regionserver，检测硬盘发现坏块，移除并替换故障硬盘，更新配置文件，重新初始化和启动服务。

0: jdbc:phoenix:localhost> select count(*) from bigdata;
+------------+
|  COUNT(1)  |
+------------+
java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=35, exceptions:
Mon Jul 13 14:14:03 CST 2015, org.apache.hadoop.hbase.client.RpcRetryingCaller@238f5df3, java.io.IOException: java.io.IOException: Could not reseek StoreFileScanner[HFileScanner for reader reader=hdfs://dmp-nn-1:8020/hbase/data/default/BIGDATA/07af823fa35b2d34add48e8e54ea8897/0/0b48d700f68d42ddad1939af39a993c4_SeqId_2_, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false][prefetchOnOpen=false], firstKey=\x01\x80\x00\x00\x00\x00\x00\x00\x0A/0:ACCEPT/1436523054347/DeleteColumn, lastKey=\x01\x80\x00\x00\x00\x05\xF5\xE0\xFE/0:_0/1436523054347/Put, avgKeyLen=32, avgValueLen=6, entries=329957958, length=6552478020, cur=\x01\x80\x00\x00\x00\x01E$\xBE/0:ACCT_INPUT_PACKET/1436523054347/Put/vlen=1/mvcc=0] to key \x01\x80\x00\x00\x00\x01E$\xBE//OLDEST_TIMESTAMP/Minimum/vlen=0/mvcc=0
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:184)
 at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.seekToNextRow(StoreScanner.java:672)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:529)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:140)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3900)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3980)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3858)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3849)
 at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:228)
 at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:134)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1203)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1517)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1592)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1556)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1198)
 at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3173)
 at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587)
 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
 at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
 at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Failed to read compressed block at 1409556126, onDiskSizeWithoutHeader=27178, preReadHeaderSize=33, header.length=33, header bytes: IDXLEAF2\x00\x02\x00:\x00\x02\x00\x16\x00\x00\x00\x00O\xA4\xF3\xD4\x01\x00\x00@\x00\x00\x02\x007
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1449)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1312)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:387)
 at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:253)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:524)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:572)
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 ... 25 more
Caused by: java.io.IOException: Invalid HFile block magic: \x00\x00\x00\x00\x00\x00\x00\x00
 at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
 at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:165)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:239)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1446)
 ... 32 more

Mon Jul 13 14:14:16 CST 2015, org.apache.hadoop.hbase.client.RpcRetryingCaller@238f5df3, java.io.IOException: java.io.IOException: Could not reseek StoreFileScanner[HFileScanner for reader reader=hdfs://dmp-nn-1:8020/hbase/data/default/BIGDATA/07af823fa35b2d34add48e8e54ea8897/0/0b48d700f68d42ddad1939af39a993c4_SeqId_2_, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false][prefetchOnOpen=false], firstKey=\x01\x80\x00\x00\x00\x00\x00\x00\x0A/0:ACCEPT/1436523054347/DeleteColumn, lastKey=\x01\x80\x00\x00\x00\x05\xF5\xE0\xFE/0:_0/1436523054347/Put, avgKeyLen=32, avgValueLen=6, entries=329957958, length=6552478020, cur=\x01\x80\x00\x00\x00\x01E$\xBE/0:ACCT_INPUT_PACKET/1436523054347/Put/vlen=1/mvcc=0] to key \x01\x80\x00\x00\x00\x01E$\xBE//OLDEST_TIMESTAMP/Minimum/vlen=0/mvcc=0
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:184)
 at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.seekToNextRow(StoreScanner.java:672)
 at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:529)
 at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:140)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3900)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3980)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3858)
 at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3849)
 at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:228)
 at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:134)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1203)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1517)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1592)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1556)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1198)
 at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3173)
 at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587)
 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
 at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
 at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Failed to read compressed block at 1409556126, onDiskSizeWithoutHeader=27178, preReadHeaderSize=33, header.length=33, header bytes: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1449)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1312)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:387)
 at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:253)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:524)
 at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:572)
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 ... 25 more
Caused by: java.io.IOException: Invalid HFile block magic: \x00\x00\x00\x00\x00\x00\x00\x00
 at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
 at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:165)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:239)
 at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1446)
 ... 32 more

 

Hadoop系统为了保证数据的一致性，会对文件生成相应的校验文件，并在读写的时候进行校验，确保数据的准确性。

所以说此类的问题，只有一种情况会导致，就是硬盘故障，就是写进去的文件（无所谓完整不完整，生成的校验文件实际上是根据写进去的文件生成的，所以说校验文件和文件块是一一对应，并保持一致的），只要写进去的和读出来的内容保持一致了，校验才可以通过，这个块有3份，写到这个机器的1份如果校验不过去，实际上，hbase也不会去读其他对应冗余块，所以说，处理的方法，不是把此节点上的regionserver，而是把datanode，regionserver通通关闭，然后集群自动检测这个节点不存在后，让他自己再去补齐这个块的冗余；

我把此节点datanode，regionserver关闭后，开始对此节点的硬盘进行检测；

检测后，发现有一块硬盘，存在坏块；摘掉此硬盘，恢复hbase的校验参数，服务正常；

hadoop集群可以做到机器直接宕机的容错，但是实际上没有做到磁盘的故障的容错，看来磁盘的监控十分有必要去做；


解决过程记录：
1.停下hbasse、zookeeper相关服务
2.badblocks -s -v -o /root/badblocks.log /dev/sdc 检测磁盘
3.关闭存在坏道的机器节点 service hadoop-yarn-nodemanager stop
4.namenode上删除坏道节点
5.bin/start-balancer.sh
6.配置/etc/zookeeper/conf.dist/zoo.cfg,移除存在坏道的节点（如果此节点有作为zookeeper节点的话）
7.配置/usr/local/hbase-0.98.6-cdh5.2.0/conf/hbase-site.xml的hbase.zookeeper.quorum属性，移除存在坏道的节点（如果此节点有作为zookeeper节点的话）
8.配置/usr/local/hbase-0.98.6-cdh5.2.0/conf/regionservers，移除存在坏道的节点。
9.初始化、启动zookeeper,启动hbase

 

 

conda创建环境失败：CondaHTTPError: HTTP 000 CONNECTION FAILED for url https://repo.anaconda.com...如何解决？
**My Coding Family**
11-23 1777
🏆本文收录于 《全栈Bug调优(实战版)》 专栏，该专栏专注于分享我在真实项目开发中遇到的各类疑难Bug及其深层成因，并系统提供高效、可复现的解决思路和实操方案。无论你是刚入行的新手开发者，还是拥有多年项目经验的资深工程师，本专栏都将为你提供一条系统化、高质量的问题排查与优化路径，助力你加速成长，攻克技术壁垒，迈向技术价值最大化与职业发展的更高峰🚀！
如何解决 java.lang.NoClassDefFoundError: org/apache/ibatis/annotations/Mapper...问题 | 亲测有效
**My Coding Family**
05-25 3692
记录:java.lang.NoClassDefFoundError: org/apache/ibatis/annotations/Mapper...【解决方案】
hbase Could not seek StoreFileScanner[HFileScanner for reader异常
wflh323的专栏
03-10 2140
最近hbase查询出现异常，hbase监控都正常，异常如下： hbase(main):003:0> get 'w:t','xxxx' COLUMN CELL ...
Caused by: java.io.IOException: On-disk size without header provided is 6, but block header contains
qq_37288262的博客
06-30 1147
Caused by: java.io.IOException: On-disk size without header provided is 6前言问题的来源：问题分析问题解决： 前言 本片报错信息博主粘的比较详细，还请耐心查看。 问题的来源： 本人写了一个spark程序，调用hbase的API接口进行scan获取数据。起先可以正常导出按批数据，可是当进行了某某批次的时候，就会爆出如下的错误： WARN TaskSetManager: Lost task 8.0 in stage 79.0 (TID 3
巨量引擎 DMP人群管理 -数据源文件上传(java 实现)
QQ734664264的博客
10-09 2095
本文介绍了使用Protocol Buffer（protobuf）的完整流程：1）下载protobuf-2.5.0和protoc编译工具；2）编译生成Java库文件；3）配置protoc.exe环境；4）创建.proto协议文件；5）使用protoc生成Java/Python代码；6）将生成的Java文件导入项目；7）引入protobuf-java依赖；8）编写示例代码（以SHA256加密手机号为例）；9）最终验证通过。整个过程详细说明了从环境搭建到代码生成的关键步骤，适用于需要处理结构化数据的开发场景。
hbase Invalid HFile block magic on hdfs system
大胡子的垃圾堆
09-25 2584
昨天hbase突然出错，前台的应用访问hbase链接堆死，
java.io.IOException: Could not obtain block: blk
豹先生
08-11 4165
联系两个晚上跑数据出现这个异常，具体内容如下： 2011-08-10 05:51:21,823 ERRORorg.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction/Splitfailed for
Hive执行异常org.apache.hadoop.hdfs.BlockMissingException
月月大王的博客
10-26 2903
Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-2040810143-192.168.144.145-1612269795515:blk_1077591653_3851069 file=/hbase/data/default/cycle_middle_data/c4cc4a321e4779c75d810ba0698079c3/info/395d0f3895cf407b88921f9cbc8
MacOS M1 docker守护进程无法启动排错整理-剑指“Cannot connect to the Docker daemon at unix:///var/run/docker.sock.”
Code Captain的专栏
03-20 2598
在macOS上，出现错误“Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?”通常是因为Docker Desktop没有正确运行，或者Docker守护进程没有启动。
记录:Exception in thread “main“ java.lang.NoClassDefFoundError: freemarker/template...【解决方案】
**My Coding Family**
05-16 3025
记录:Exception in thread “main“ java.lang.NoClassDefFoundError: freemarker/template...详细，实测有效。
ch3/visualizeGeometry编译报错：/usr/bin/ld: 找不到 -lepoxy::epoxy /usr/bin/ld: 找不到 -lOpenGL::EGL
weixin_43764716的博客
06-06 1696
《视觉SLAM十四讲》ch3/visualizeGeometry编译报错：/usr/bin/ld: 找不到 -lepoxy::epoxy /usr/bin/ld: 找不到 -lOpenGL::EGL
java 线程池 阻塞超时_Java多线程处理某个线程超时的问题
weixin_39859061的博客
02-13 344
Mon Apr 29 18:12:01 CST 2019, org.apache.hadoop.hbase.client.RpcRetryingCaller@34f04eb2, java.io.IOException: java.io.IOException: Could not seek StoreFileScanner[HFileScanner for reader reader=hdfs:/...
解决Caused by: java.io.IOException: 设备上没有空间
热门推荐
般若
02-20 1万+
文件名称 版本号 作者 qq 组件版本 标题 v1.0.0 学生宫布 8416837 SpringBoot 2.2.6SpringCloud Hoxton.SR4 文章目录报错全称代码疑似产生的原因弯路、坑分析解决方案领悟关于 报错全称 CENT OS环境，运行jar包时，出现 Caused by: java.io.IOException: 设备上没有空间 执行df -h 发现/dev/mapper/centos-root 100% 代码 疑似产生的原因 空间已占用100% 弯路、坑 删...
Hbase错误持续汇总
weixin_30335353的博客
01-15 196
1、当一个客户端对Hbase进行大量操作时，我对这个表进行count时，发现到2000多万行时， Hbase 报java.io.IOException: Could not seekStoreFileScanner，……Caused by: java.io.IOException: Could notobtain block…… 网上资料说修改 hadoop的配置文件 ...
phoenix查询奇怪的异常
lingzhiwangcn的专栏
03-24 2977
selectSIWEB.DNSDELAY from CPSPHOENIXTESTDEMO where provcity='8320800' andDNSDELAY='-1';[this is OK] selectSIWEB.DNSDELAY from CPSPHOENIXTESTDEMO where provcity like '8320%' and DNSDELAY='-1';
org.apache.phoenix.exception.PhoenixIOException: SYSTEM:CATALOG
zzy7075的专栏
02-22 964
Error: SYSTEM:CATALOG (state=08000,code=101) org.apache.phoenix.exception.PhoenixIOException: SYSTEM:CATALOG at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:113) at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCop
HBase Scanner
迷糊的佳佳不迷糊
11-19 4087
一个region有一个regionscanner，一个regionscanner维持一个 PriorityQueue，包含storescanner； 一个storescanner维持一个 PriorityQueue，包含一个memstorescanner和多个storefilescannerstore是一个family。 regionscanner扫描一个region, 包含多个store
[root@master server]# [root@master server]# [root@master server]# [root@master server]# [root@master server]# [root@master server]# vi /export/server/hadoop/etc/hadoop/hadoop-env.sh [root@master server]# ls -ld /export/server/tmp/datanode drwx------. 2 root root 6 11月 6 20:45 /export/server/tmp/datanode [root@master server]# chown -R root:root /export/server/tmp/datanode [root@master server]# chmod -R 755 /export/server/tmp/datanode [root@master server]# hdfs namenode -format -bash: hdfs: 未找到命令 [root@master server]# # 停止可能残留进程 [root@master server]# /export/server/hadoop/sbin/stop-dfs.sh Stopping namenodes on [master] 上一次登录：四 11月 6 20:46:16 CST 2025pts/2 上 master: Unable to get valid context for root Stopping datanodes 上一次登录：四 11月 6 20:49:54 CST 2025pts/2 上 localhost: Unable to get valid context for root Stopping secondary namenodes [master] 上一次登录：四 11月 6 20:49:55 CST 2025pts/2 上 master: Unable to get valid context for root [root@master server]# [root@master server]# # 清理 NameNode 和 DataNode 目录 [root@master server]# rm -rf /export/server/tmp/namenode/* [root@master server]# rm -rf /export/server/tmp/datanode/* [root@master server]# /export/server/hadoop/bin/hdfs namenode -format 2025-11-06 20:50:08,919 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = master/192.168.10.130 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 3.1.3 STARTUP_MSG: classpath = /export/server/hadoop/etc/hadoop:/export/server/hadoop/share/hadoop/common/lib/accessors-smart-1.2.jar:/export/server/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/export/server/hadoop/share/hadoop/common/lib/asm-5.0.4.jar:/export/server/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/export/server/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/export/server/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/export/server/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/export/server/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/export/server/hadoop/share/hadoop/common/lib/commons-codec-1.11.jar:/export/server/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/export/server/hadoop/share/hadoop/common/lib/commons-compress-1.18.jar:/export/server/hadoop/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/export/server/hadoop/share/hadoop/common/lib/commons-io-2.5.jar:/export/server/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/export/server/hadoop/share/hadoop/common/lib/commons-lang3-3.4.jar:/export/server/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/export/server/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/export/server/hadoop/share/hadoop/common/lib/commons-net-3.6.jar:/export/server/hadoop/share/hadoop/common/lib/curator-client-2.13.0.jar:/export/server/hadoop/share/hadoop/common/lib/curator-framework-2.13.0.jar:/export/server/hadoop/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/export/server/hadoop/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/export/server/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/export/server/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/export/server/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/export/server/hadoop/share/hadoop/common/lib/hadoop-annotations-3.1.3.jar:/export/server/hadoop/share/hadoop/common/lib/hadoop-auth-3.1.3.jar:/export/server/hadoop/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/export/server/hadoop/share/hadoop/common/lib/httpclient-4.5.2.jar:/export/server/hadoop/share/hadoop/common/lib/httpcore-4.4.4.jar:/export/server/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-core-2.7.8.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/export/server/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/export/server/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/export/server/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/export/server/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/export/server/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/export/server/hadoop/share/hadoop/common/lib/jersey-core-1.19.jar:/export/server/hadoop/share/hadoop/common/lib/jersey-json-1.19.jar:/export/server/hadoop/share/hadoop/common/lib/jersey-server-1.19.jar:/export/server/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.jar:/export/server/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/common/lib/jsch-0.1.54.jar:/export/server/hadoop/share/hadoop/common/lib/json-smart-2.3.jar:/export/server/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/export/server/hadoop/share/hadoop/common/lib/jsr305-3.0.0.jar:/export/server/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/export/server/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/export/server/hadoop/share/hadoop/common/lib/netty-3.10.5.Final.jar:/export/server/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/export/server/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/export/server/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/export/server/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/export/server/hadoop/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/export/server/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/export/server/hadoop/share/hadoop/common/lib/snappy-java-1.0.5.jar:/export/server/hadoop/share/hadoop/common/lib/stax2-api-3.1.4.jar:/export/server/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/export/server/hadoop/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/export/server/hadoop/share/hadoop/common/lib/zookeeper-3.4.13.jar:/export/server/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/export/server/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/export/server/hadoop/share/hadoop/common/hadoop-common-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/common/hadoop-common-3.1.3.jar:/export/server/hadoop/share/hadoop/common/hadoop-nfs-3.1.3.jar:/export/server/hadoop/share/hadoop/common/hadoop-kms-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs:/export/server/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/export/server/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/export/server/hadoop/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/export/server/hadoop/share/hadoop/hdfs/lib/okio-1.6.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/export/server/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/export/server/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/export/server/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/export/server/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/json-smart-2.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/export/server/hadoop/share/hadoop/hdfs/lib/asm-5.0.4.jar:/export/server/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/export/server/hadoop/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/export/server/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/export/server/hadoop/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-io-2.5.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/export/server/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/export/server/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-net-3.6.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jettison-1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/export/server/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/export/server/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/export/server/hadoop/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/export/server/hadoop/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/export/server/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/export/server/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/gson-2.2.4.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/export/server/hadoop/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/export/server/hadoop/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/export/server/hadoop/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/export/server/hadoop/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/export/server/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.3.jar:/export/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.3-tests.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.3.jar:/export/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn:/export/server/hadoop/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/export/server/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/export/server/hadoop/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/export/server/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/export/server/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/export/server/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/export/server/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/export/server/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/export/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/export/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/export/server/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/export/server/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/export/server/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/export/server/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.jar:/export/server/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/export/server/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/export/server/hadoop/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/export/server/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/export/server/hadoop/share/hadoop/yarn/lib/objenesis-1.0.jar:/export/server/hadoop/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/export/server/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.1.3.jar:/export/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.1.3.jar STARTUP_MSG: build = https://gitbox.apache.org/repos/asf/hadoop.git -r ba631c436b806728f8ec2f54ab1e289526c90579; compiled by 'ztang' on 2019-09-12T02:47Z STARTUP_MSG: java = 1.8.0_391 ************************************************************/ 2025-11-06 20:50:08,929 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2025-11-06 20:50:09,070 INFO namenode.NameNode: createNameNode [-format] 2025-11-06 20:50:09,760 INFO common.Util: Assuming 'file' scheme for path /export/server/tmp/namenode in configuration. 2025-11-06 20:50:09,761 INFO common.Util: Assuming 'file' scheme for path /export/server/tmp/namenode in configuration. Formatting using clusterid: CID-39a7664e-8e3d-4fba-bf06-024957a80acf 2025-11-06 20:50:09,802 INFO namenode.FSEditLog: Edit logging is async:true 2025-11-06 20:50:09,821 INFO namenode.FSNamesystem: KeyProvider: null 2025-11-06 20:50:09,822 INFO namenode.FSNamesystem: fsLock is fair: true 2025-11-06 20:50:09,822 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false 2025-11-06 20:50:09,835 INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE) 2025-11-06 20:50:09,835 INFO namenode.FSNamesystem: supergroup = supergroup 2025-11-06 20:50:09,835 INFO namenode.FSNamesystem: isPermissionEnabled = false 2025-11-06 20:50:09,835 INFO namenode.FSNamesystem: HA Enabled: false 2025-11-06 20:50:09,882 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-11-06 20:50:09,889 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000 2025-11-06 20:50:09,889 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true 2025-11-06 20:50:09,893 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000 2025-11-06 20:50:09,894 INFO blockmanagement.BlockManager: The block deletion will start around 2025 十一月 06 20:50:09 2025-11-06 20:50:09,895 INFO util.GSet: Computing capacity for map BlocksMap 2025-11-06 20:50:09,895 INFO util.GSet: VM type = 64-bit 2025-11-06 20:50:09,896 INFO util.GSet: 2.0% max memory 235.9 MB = 4.7 MB 2025-11-06 20:50:09,896 INFO util.GSet: capacity = 2^19 = 524288 entries 2025-11-06 20:50:09,908 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false 2025-11-06 20:50:09,924 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS 2025-11-06 20:50:09,924 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2025-11-06 20:50:09,924 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0 2025-11-06 20:50:09,924 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: defaultReplication = 1 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: maxReplication = 512 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: minReplication = 1 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: encryptDataTransfer = false 2025-11-06 20:50:09,925 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000 2025-11-06 20:50:09,943 INFO namenode.FSDirectory: GLOBAL serial map: bits=24 maxEntries=16777215 2025-11-06 20:50:09,959 INFO util.GSet: Computing capacity for map INodeMap 2025-11-06 20:50:09,963 INFO util.GSet: VM type = 64-bit 2025-11-06 20:50:09,964 INFO util.GSet: 1.0% max memory 235.9 MB = 2.4 MB 2025-11-06 20:50:09,964 INFO util.GSet: capacity = 2^18 = 262144 entries 2025-11-06 20:50:09,965 INFO namenode.FSDirectory: ACLs enabled? false 2025-11-06 20:50:09,965 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true 2025-11-06 20:50:09,965 INFO namenode.FSDirectory: XAttrs enabled? true 2025-11-06 20:50:09,965 INFO namenode.NameNode: Caching file names occurring more than 10 times 2025-11-06 20:50:09,975 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536 2025-11-06 20:50:09,978 INFO snapshot.SnapshotManager: SkipList is disabled 2025-11-06 20:50:09,982 INFO util.GSet: Computing capacity for map cachedBlocks 2025-11-06 20:50:09,982 INFO util.GSet: VM type = 64-bit 2025-11-06 20:50:09,982 INFO util.GSet: 0.25% max memory 235.9 MB = 603.8 KB 2025-11-06 20:50:09,982 INFO util.GSet: capacity = 2^16 = 65536 entries 2025-11-06 20:50:09,996 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10 2025-11-06 20:50:09,996 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10 2025-11-06 20:50:09,996 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25 2025-11-06 20:50:09,999 INFO namenode.FSNamesystem: Retry cache on namenode is enabled 2025-11-06 20:50:09,999 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 2025-11-06 20:50:10,001 INFO util.GSet: Computing capacity for map NameNodeRetryCache 2025-11-06 20:50:10,001 INFO util.GSet: VM type = 64-bit 2025-11-06 20:50:10,001 INFO util.GSet: 0.029999999329447746% max memory 235.9 MB = 72.5 KB 2025-11-06 20:50:10,001 INFO util.GSet: capacity = 2^13 = 8192 entries 2025-11-06 20:50:10,055 INFO namenode.FSImage: Allocated new BlockPoolId: BP-267218780-192.168.10.130-1762433410047 2025-11-06 20:50:10,117 INFO common.Storage: Storage directory /export/server/tmp/namenode has been successfully formatted. 2025-11-06 20:50:10,209 INFO namenode.FSImageFormatProtobuf: Saving image file /export/server/tmp/namenode/current/fsimage.ckpt_0000000000000000000 using no compression 2025-11-06 20:50:10,309 INFO namenode.FSImageFormatProtobuf: Image file /export/server/tmp/namenode/current/fsimage.ckpt_0000000000000000000 of size 391 bytes saved in 0 seconds . 2025-11-06 20:50:10,322 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 2025-11-06 20:50:10,335 INFO namenode.FSImage: FSImageSaver clean checkpoint: txid = 0 when meet shutdown. 2025-11-06 20:50:10,336 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at master/192.168.10.130 ************************************************************/ [root@master server]# /export/server/hadoop/sbin/start-dfs.sh Starting namenodes on [master] 上一次登录：四 11月 6 20:49:56 CST 2025pts/2 上 master: Unable to get valid context for root Starting datanodes 上一次登录：四 11月 6 20:50:14 CST 2025pts/2 上 localhost: Unable to get valid context for root Starting secondary namenodes [master] 上一次登录：四 11月 6 20:50:14 CST 2025pts/2 上 master: Unable to get valid context for root [root@master server]# jps 87714 DataNode 92283 Jps [root@master server]# lsof -i :9866 -bash: lsof: 未找到命令 [root@master server]# # 或使用 netstat [root@master server]# netstat -tuln | grep 9866 -bash: netstat: 未找到命令 [root@master server]# systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:firewalld(1) 10月 31 16:34:13 master systemd[1]: Starting firewalld - dynamic fi.... 10月 31 16:34:13 master systemd[1]: Started firewalld - dynamic fir.... 11月 06 20:43:16 master systemd[1]: Stopping firewalld - dynamic fi.... 11月 06 20:43:17 master systemd[1]: Stopped firewalld - dynamic fir.... Hint: Some lines were ellipsized, use -l to show in full. [root@master server]# systemctl stop firewalld [root@master server]# systemctl disable firewalld [root@master server]# jps 87714 DataNode 92955 Jps [root@master server]#
最新发布
11-07
# 操作重述 您已完成 Hadoop 的配置、格式化 NameNode 并启动 `start-dfs.sh`，**`jps` 显示 `DataNode` 已运行（PID 存在）**，但： - **`NameNode` 和 `SecondaryNameNode` 未出现** - Web UI `http://192.168.10.130:9870` 无法访问 - 执行 `lsof` 和 `netstat` 提示命令未安装 - 启动时提示：`Unable to get valid context for root` 说明：**NameNode 实际上并未成功启动**，仅 DataNode 起来了，这是典型的主从节点不一致或 SSH 认证异常导致的问题。 --- ## ✅ 根本原因分析 ### ❌ 关键错误信息： ```bash Starting namenodes on [master] master: Unable to get valid context for root ``` 这个错误通常出现在 **SSH 登录上下文失败**，尤其是在使用 SELinux 或某些安全策略的系统中。虽然您已关闭防火墙，但 `start-dfs.sh` 脚本默认会通过 SSH 连接本地主机来启动各组件（即使在同一台机器），如果 SSH 上下文验证失败，则 NameNode 无法拉起。 > 即使 `ssh master exit` 成功，Hadoop 内部调用仍可能因安全策略失败。 --- ## ✅ 解决方案：强制本地启动 + 确保命令可用 ### ✔ 步骤 1：手动启动 NameNode（绕过 SSH） 由于远程启动失败，直接在本地执行： ```bash /export/server/hadoop/bin/hdfs --daemon start namenode ``` ### ✔ 步骤 2：手动启动 SecondaryNameNode ```bash /export/server/hadoop/bin/hdfs --daemon start secondarynamenode ``` ### ✔ 验证进程是否起来 ```bash jps ``` ✅ 正常输出应包含： ``` NameNode DataNode SecondaryNameNode Jps ``` --- ### ✔ 步骤 3：安装必要的网络工具（用于后续排错） ```bash yum install -y net-tools yum install -y lsof ``` 然后验证端口监听情况： ```bash netstat -tuln | grep 9870 # NameNode Web UI netstat -tuln | grep 9000 # fs.defaultFS 通信端口 netstat -tuln | grep 9866 # DataNode 数据传输端口 ``` 或使用 `lsof`： ```bash lsof -i :9870 lsof -i :9000 ``` --- ### ✔ 步骤 4：确认 `core-site.xml` 配置正确 ```bash vi /export/server/hadoop/etc/hadoop/core-site.xml ``` 确保内容为： ```xml <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> </configuration> ``` > 如果写成 `localhost` 或 `127.0.0.1`，WebUI 将无法从外部访问。 --- ### ✔ 步骤 5：确认 `/etc/hosts` 映射无误 ```bash cat /etc/hosts ``` 必须包含： ```text 192.168.10.130 master ``` 避免解析到 `127.0.0.1` 导致绑定错误地址。 --- ### ✔ 步骤 6：检查 NameNode 日志确认启动状态 ```bash tail -100f /export/server/hadoop/logs/hadoop-root-namenode-master.log ``` 查找关键词： - `Web server started` - `NameNode up` - `Address already in use` → 端口被占 - `Cannot create directory` → 权限问题 --- ### ✔ 步骤 7：允许 Hadoop 绑定高权限端口（可选） 若日志报错 “Cannot assign requested address”，可在 `hdfs-site.xml` 中调整绑定地址为 `0.0.0.0`： ```xml <property> <name>dfs.namenode.http-address</name> <value>0.0.0.0:9870</value> </property> <property> <name>dfs.namenode.rpc-address</name> <value>master:9000</value> </property> ``` 这样可以从任意 IP 访问 Web UI。 --- ### ✔ 步骤 8：Windows 主机测试访问 在 Windows 浏览器打开： ``` http://192.168.10.130:9870 ``` 如果仍未响应，请检查： 1. VMware 网络模式是否为 **NAT 或桥接** 2. Windows 是否能 `ping 192.168.10.130` 3. 虚拟机是否禁用了网络？执行 `ifconfig` 查看 IP 是否正常 --- ## 🔍 知识点（共三项） **1. --daemon 启动方式作用** 绕过SSH远程调用，直接在本地启动Hadoop守护进程，适用于单机调试和SSH异常场景。 **2. NameNode Web UI端口机制** `9870`是Hadoop 3.x默认HTTP端口，由`dfs.namenode.http-address`控制绑定地址与端口。 **3. hosts映射对分布式系统影响** 错误的主机解析会导致服务绑定到localhost，外部无法访问，必须确保host名→IP正确对应。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值