在HDFS中,提供了fsck命令,用于检查HDFS上文件和目录的健康状态、获取文件的block信息和位置信息等。
fsck命令必须由HDFS超级用户来执行,普通用户无权限。
可通过hdfs fsck来查看该命令的帮助文档,如下图所示:
1.手工修复 hdfs debug
1)造一份数据上传到hdfs
[hadoop@hadoop001 data]$ hadoop fs -put test.txt /blockrecover
2)通过hdfs fsck定位该文件块所在位置
[hadoop@hadoop001 data]$ hdfs fsck /blockrecover/test.txt -files -blocks -locations
19/04/04 12:38:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop001:50070
FSCK started by hadoop (auth:SIMPLE) from /192.168.0.108 for path /blockrecover/test.txt at Thu Apr 04 12:38:48 PDT 2019
/blockrecover/test.txt 2 bytes, 1 block(s): Under replicated BP-1279338148-192.168.199.200-1491145566665:blk_1073742362_1538. Target Replicas is 3 but found 1 replica(s).
0. BP-1279338148-192.168.199.200-1491145566665:blk_1073742362_1538 len=2 Live_repl=1 [DatanodeInfoWithStorage[192.168.0.108:50010,DS-9841e6c3-9a74-4bc8-869e-3543dcf7de90,DISK]]
Status: HEALTHY
Total size: 2 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 2 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt b