【Hadoop】Hadoop/Yarn中hdfs与mapreduce相关问题汇总

最新推荐文章于 2023-03-14 12:23:04 发布
原创 最新推荐文章于 2023-03-14 12:23:04 发布 · 5.5k 阅读 · 5 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#Hadoop #HDFS #namenode #MapReduce #Yarn

本文总结了Hadoop在启动和运行过程中遇到的HDFS及MapReduce相关问题，包括HDFS健康状态问题、NameNode安全模式、权限错误以及MapReduce任务复制失败等，并提供了相应的解决方案，帮助用户解决Hadoop集群中的常见故障。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

在hadoop启动的时候，会出现各种各样的问题，NameNode，JobTracker等各个模块都会有莫名奇妙的问题出现，在这里对hdfs的问题进行一下简单的记录。

注：问题无先后次序，可能这次没遇到下次又遇到，所以统一记录在次

一：HDFS问题

1：HDFS initialized but not 'healthy' yet, waiting...

这个日志会在启动hadoop的时候在JobTracker的log日志文件中出现，在这里就是hdfs出现问题，导致DataNode无法启动，这里唯一的解决方式就是把所有的NameNode管理的路径下的文件删除然后重新执行namenode -format，而删除的地方主要有存放临时数据的tmp路径，存放数据的data路径还有name路径，全部删除之后重新format次问题就解决了

2：在执行hadoop程序的时候出现Name node is in safe mode

这个异常一般就直接会在IDE的控制台输出，这个错误的主要导致原因是，datanode不停在丢失数据，所以此时namenode就强制本身进入safe mode模式，在该模式下对数据只可以进行读操作而不能进行写操作。解决此异常很简单，直接执行命令让namenode离开次模式就可以了。./hadoop dfsadmin-safemode leave


3：原来hadoop一直可以正常启动，有天在启动之后查看namenode的log发现如下in_use.lock (Permission denied)错误日志：

INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
>> 1INFO metrics.FSNamesystemMetrics: Initializing
>> FSNamesystemMetrics using context
>> object:org.apache.hadoop.metrics.spi.NoEmitMetricsContext
ERROR namenode.FSNamesystem: FSNamesystem initialization failed. java.io.FileNotFoundException:  /var/lib/hadoop-0.20/cache/hadoop/dfs/name/in_use.lock (Permission denied)
>> at java.io.RandomAccessFile.open(Native Method)
>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>> at
>> org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.tryLock(Storage.java:614)
>> at
>> org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:591)
>> at
>> org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:449)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:304)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:372)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:335)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:467)
>> at
>

最低0.47元/天 解锁文章

200万优质内容无限畅学
大数据Hadoop入门——HDFS、Yarn、MapReduce
weixin_44090845的博客
11-16 2634
Hadoop框架详细个人总结——Hadoop入门篇大数据是什么？Hadoop基本概念介绍一、什么是Hadoop？二、Hadoop优势——四高三、Hadoop组成（重点）四、HDFS架构五、YARN架构六、MapReduce架构七、HDFS、YARN、MapReduce 三者关系Hadoop运行环境搭建虚拟机的准备一、安装虚拟机二、虚拟机配置及环境准备Hadoop运行模式一、本地运行模式二、伪分布式模式三、完全分布式模式**1.虚拟机准备****2.准备一个集群分发脚本****3.SSH 无密登录配置****
【图解大数据技术】Hadoop、HDFS、MapReduce、Yarn
weixin_43889578的博客
06-29 2726
MapReduce是一个分布式离线计算框架，专门用于处理大数据场景中与实时性无关的一些离线计算任务。MapReduce的数据输入一般是HDFS，然后经过InputFormat进行输入格式化，变成格式；然后执行用户实现的Mapper类型的map方法，进行数据映射，映射处理的结果也是格式；然后执行一个shuffle过程，对映射结果进行按key进行分组分区，把同一区域的所有KV发送到同一个Reducer，由一个节点进行；Reducer对同一个key分组下的所有value进行聚合操作；
【Hadoop】在Linux中的Hadoop部署与yarn HDFS MapReduce 的配置中常见的问题？你解决了吗？
wyz0516071128的博客
06-10 677
（1） 浏览器问题 当你配置都完成了，你输入主机名+端口号去访问时，不能访问，但是你得确定前面的配置的哪些步骤都没有错，这可能是你浏览器的问题，我推荐用Chrome或者火狐，这两个基本上是可以的。（2） 域名问题 当你输入IP地址+端口号能访问时，这时你就得看看你的Windows上的映射是否配置成功，可能是在你在配置Windows中的映射的时候配置错了或者是你没有配置。C:...
ERROR hdfs.DFSClient: Failed to close file解决方法
weixin_30699235的博客
04-11 2526
14/04/11 17:59:44 ERROR hdfs.DFSClient: Failed to close file /wlan_out/_temporary/_attempt_local_0001_r_000000_0/part-r-00000org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.nameno...
下一代Apache Hadoop MapReduce框架的架构
caodaoxi的专栏
10-23 161
背景   随着集群规模和负载增加，MapReduce JobTracker在内存消耗，线程模型和扩展性/可靠性/性能方面暴露出了缺点，为此需要对它进行大整修。 需求 当我们对Hadoop MapReduce框架进行改进时，需要时刻谨记的一个重要原则是用户的需求。近几年来，从Hadoop用户那里总结出MapReduce框架当前最紧迫的需求有： （1）可靠性（Reliability）...
Hadoop YARN 常见问题【二】
不负青春、不负卿
06-14 308
YARN** 1.yarn与mapreduce的关系？ 答： （1）yarn是资源调度框架，mapreduce是分布式计算框架； （2）yarn将jobTracker的资源管理和任务调度划分开了，通过ResourceManager进行资源的统一管理和分配， ApplicationManager进行解析mapreduce程序然后变成一个个小任务,需要多少资源向ResourceManager请求， 然...
Hadoop笔记11之 Hadoop异常 hdfs.DFSClient(hdfs 不能上传文件)
似水流年
09-12 2123
问题: 无法将问文件上传到HDFS集群中,报错如图: 异常分析: 上图所示,说明Hadoop集群中没有datanode结点即是即是slave中datanode没有被启动, 解决方式: 当打datanode无法正常启动时,修改master  中 vim /home/zkpk/hadoopdata/dfs/name/current/VERSION (不同虚拟机路径可能不
[root@master logs]# cat hadoop-root-datanode-master.log 2025-08-23 21:26:53,162 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = master/192.168.88.8 STARTUP_MSG: args = [] STARTUP_MSG: version = 3.3.6 STARTUP_MSG: classpath = /server/hadoop/etc/hadoop:/server/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/server/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-text-1.10.0.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/server/hadoop/share/hadoop/common/lib/jetty-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/hadoop-annotations-3.3.6.jar:/server/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/server/hadoop/share/hadoop/common/lib/jersey-json-1.20.jar:/server/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-9.8.1.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/snappy-java-1.1.8.2.jar:/server/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/gson-2.9.0.jar:/server/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-redis-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-http2-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jetty-util-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-util-ajax-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/zookeeper-3.6.3.jar:/server/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/server/hadoop/share/hadoop/common/lib/httpcore-4.4.13.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-rxtx-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/curator-client-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/netty-buffer-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/server/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-xml-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/commons-net-3.9.0.jar:/server/hadoop/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/server/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/server/hadoop/share/hadoop/common/lib/jettison-1.5.4.jar:/server/hadoop/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/jsr305-3.0.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/server/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-socks-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-io-2.8.0.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/server/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/commons-codec-1.15.jar:/server/hadoop/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-stomp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/zookeeper-jute-3.6.3.jar:/server/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/server/hadoop/share/hadoop/common/lib/stax2-api-4.2.1.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-http-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/server/hadoop/share/hadoop/common/lib/hadoop-shaded-guava-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-core-2.12.7.jar:/server/hadoop/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/server/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/server/hadoop/share/hadoop/common/lib/jetty-security-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/reload4j-1.2.22.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-haproxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/curator-framework-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/jsch-0.1.55.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-memcache-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/jetty-http-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-proxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jetty-webapp-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/jersey-server-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-sctp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/server/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-io-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-udt-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-xml-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/server/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/server/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/server/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-smtp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/server/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-mqtt-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/httpclient-4.5.13.jar:/server/hadoop/share/hadoop/common/lib/netty-all-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/server/hadoop/share/hadoop/common/lib/jersey-core-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/server/hadoop/share/hadoop/common/lib/hadoop-auth-3.3.6.jar:/server/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/commons-compress-1.21.jar:/server/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/server/hadoop/share/hadoop/common/hadoop-registry-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-common-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-kms-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-nfs-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-common-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs:/server/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/server/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.8.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar:/server/hadoop/share/hadoop/hdfs/lib/okio-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/gson-2.9.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-1.4.10.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-redis-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-http2-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-util-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/zookeeper-3.6.3.jar:/server/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/server/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-buffer-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-xml-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/server/hadoop/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/server/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-socks-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-io-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/server/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/zookeeper-jute-3.6.3.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/server/hadoop/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-common-1.4.10.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-http-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/server/hadoop/share/hadoop/hdfs/lib/okhttp-4.9.3.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-security-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-http-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/HikariCP-java7-2.4.12.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-io-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-udt-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-all-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-compress-1.21.jar:/server/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6-tests.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6-tests.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6.jar:/server/hadoop/share/hadoop/yarn:/server/hadoop/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/server/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.68.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-client-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/server/hadoop/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jline-3.9.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/server/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/server/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-annotations-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-jndi-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/server/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/server/hadoop/share/hadoop/yarn/lib/objenesis-2.6.jar:/server/hadoop/share/hadoop/yarn/lib/asm-tree-9.4.jar:/server/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-common-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-plus-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jna-5.2.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-api-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-client-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.68.jar:/server/hadoop/share/hadoop/yarn/lib/asm-commons-9.4.jar:/server/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/server/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/server/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/server/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/server/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.6.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1be78238728da9266a4f88195058f08fd012bf9c; compiled by 'ubuntu' on 2023-06-18T08:22Z STARTUP_MSG: java = 1.8.0_401 ************************************************************/ 2025-08-23 21:26:53,175 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2025-08-23 21:26:53,573 INFO org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/server/hadoop/data/dn 2025-08-23 21:26:53,762 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties 2025-08-23 21:26:53,865 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-08-23 21:26:53,865 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2025-08-23 21:26:54,362 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-08-23 21:26:54,408 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576 2025-08-23 21:26:54,417 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is master 2025-08-23 21:26:54,417 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-08-23 21:26:54,439 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0 2025-08-23 21:26:54,484 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:9866 2025-08-23 21:26:54,486 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwidth is 104857600 bytes/s 2025-08-23 21:26:54,486 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 100 2025-08-23 21:26:54,598 INFO org.eclipse.jetty.util.log: Logging initialized @2208ms to org.eclipse.jetty.util.log.Slf4jLog 2025-08-23 21:26:54,928 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /root/hadoop-http-auth-signature-secret 2025-08-23 21:26:54,939 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined 2025-08-23 21:26:54,950 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2025-08-23 21:26:54,955 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2025-08-23 21:26:54,955 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2025-08-23 21:26:54,955 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2025-08-23 21:26:55,012 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 43231 2025-08-23 21:26:55,014 INFO org.eclipse.jetty.server.Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 1.8.0_401-b10 2025-08-23 21:26:55,087 INFO org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0 2025-08-23 21:26:55,087 INFO org.eclipse.jetty.server.session: No SessionScavenger set, using defaults 2025-08-23 21:26:55,090 INFO org.eclipse.jetty.server.session: node0 Scavenging every 600000ms 2025-08-23 21:26:55,121 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4fad9bb2{logs,/logs,file:///server/hadoop/logs/,AVAILABLE} 2025-08-23 21:26:55,122 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@1941a8ff{static,/static,file:///server/hadoop/share/hadoop/hdfs/webapps/static/,AVAILABLE} 2025-08-23 21:26:55,320 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@35390ee3{datanode,/,file:///server/hadoop/share/hadoop/hdfs/webapps/datanode/,AVAILABLE}{file:/server/hadoop/share/hadoop/hdfs/webapps/datanode} 2025-08-23 21:26:55,337 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@66565121{HTTP/1.1, (http/1.1)}{localhost:43231} 2025-08-23 21:26:55,337 INFO org.eclipse.jetty.server.Server: Started @2947ms 2025-08-23 21:26:55,504 WARN org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Got null for restCsrfPreventionFilter - will not do any filtering. 2025-08-23 21:26:55,620 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:9864 2025-08-23 21:26:55,636 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root 2025-08-23 21:26:55,637 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup 2025-08-23 21:26:55,638 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor 2025-08-23 21:26:55,857 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false. 2025-08-23 21:26:55,878 INFO org.apache.hadoop.ipc.Server: Listener at 0.0.0.0:9867 2025-08-23 21:26:55,883 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9867 2025-08-23 21:26:56,440 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:9867 2025-08-23 21:26:56,495 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2025-08-23 21:26:56,507 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Unable to get NameNode addresses. java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address.[] or dfs.namenode.rpc-address.[] is not configured. at org.apache.hadoop.hdfs.DFSUtil.getNNServiceRpcAddressesForCluster(DFSUtil.java:656) at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:157) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1755) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:564) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:3148) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:3054) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:3098) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:3242) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:3266) 2025-08-23 21:26:56,522 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.w.WebAppContext@35390ee3{datanode,/,null,STOPPED}{file:/server/hadoop/share/hadoop/hdfs/webapps/datanode} 2025-08-23 21:26:56,527 INFO org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@66565121{HTTP/1.1, (http/1.1)}{localhost:0} 2025-08-23 21:26:56,527 INFO org.eclipse.jetty.server.session: node0 Stopped scavenging 2025-08-23 21:26:56,527 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@1941a8ff{static,/static,file:///server/hadoop/share/hadoop/hdfs/webapps/static/,STOPPED} 2025-08-23 21:26:56,528 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@4fad9bb2{logs,/logs,file:///server/hadoop/logs/,STOPPED} 2025-08-23 21:26:56,537 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Waiting up to 30 seconds for transfer threads to complete 2025-08-23 21:26:56,537 INFO org.apache.hadoop.ipc.Server: Stopping server on 9867 2025-08-23 21:26:56,542 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping DataNode metrics system... 2025-08-23 21:26:56,542 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system stopped. 2025-08-23 21:26:56,543 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system shutdown complete. 2025-08-23 21:26:56,548 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Shutdown complete. 2025-08-23 21:26:56,548 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain java.io.IOException: No services to connect, missing NameNode address. at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:165) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1755) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:564) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:3148) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:3054) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:3098) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:3242) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:3266) 2025-08-23 21:26:56,550 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.io.IOException: No services to connect, missing NameNode address. 2025-08-23 21:26:56,558 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at master/192.168.88.8 ************************************************************/ 2025-08-23 21:29:44,258 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = master/192.168.88.8 STARTUP_MSG: args = [] STARTUP_MSG: version = 3.3.6 STARTUP_MSG: classpath = /server/hadoop/etc/hadoop:/server/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/server/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-text-1.10.0.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/server/hadoop/share/hadoop/common/lib/jetty-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/hadoop-annotations-3.3.6.jar:/server/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/server/hadoop/share/hadoop/common/lib/jersey-json-1.20.jar:/server/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-9.8.1.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/snappy-java-1.1.8.2.jar:/server/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/gson-2.9.0.jar:/server/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-redis-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-http2-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jetty-util-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-util-ajax-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/zookeeper-3.6.3.jar:/server/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/server/hadoop/share/hadoop/common/lib/httpcore-4.4.13.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-rxtx-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/curator-client-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/netty-buffer-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/server/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-xml-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/commons-net-3.9.0.jar:/server/hadoop/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/server/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/server/hadoop/share/hadoop/common/lib/jettison-1.5.4.jar:/server/hadoop/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/jsr305-3.0.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/server/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-socks-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-io-2.8.0.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/server/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/commons-codec-1.15.jar:/server/hadoop/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-stomp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/zookeeper-jute-3.6.3.jar:/server/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/server/hadoop/share/hadoop/common/lib/stax2-api-4.2.1.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-http-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/server/hadoop/share/hadoop/common/lib/hadoop-shaded-guava-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/jackson-core-2.12.7.jar:/server/hadoop/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/server/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/server/hadoop/share/hadoop/common/lib/jetty-security-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/reload4j-1.2.22.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-haproxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/curator-framework-5.2.0.jar:/server/hadoop/share/hadoop/common/lib/jsch-0.1.55.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-memcache-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/jetty-http-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-handler-proxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/server/hadoop/share/hadoop/common/lib/jetty-webapp-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/jersey-server-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-sctp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/netty-resolver-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/server/hadoop/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/server/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-io-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/netty-transport-udt-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/jetty-xml-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/server/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/server/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/server/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-smtp-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/server/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/server/hadoop/share/hadoop/common/lib/netty-codec-mqtt-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/server/hadoop/share/hadoop/common/lib/httpclient-4.5.13.jar:/server/hadoop/share/hadoop/common/lib/netty-all-4.1.89.Final.jar:/server/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/server/hadoop/share/hadoop/common/lib/jersey-core-1.19.4.jar:/server/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/server/hadoop/share/hadoop/common/lib/hadoop-auth-3.3.6.jar:/server/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/server/hadoop/share/hadoop/common/lib/commons-compress-1.21.jar:/server/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/server/hadoop/share/hadoop/common/hadoop-registry-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-common-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-kms-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-nfs-3.3.6.jar:/server/hadoop/share/hadoop/common/hadoop-common-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs:/server/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/server/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.8.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar:/server/hadoop/share/hadoop/hdfs/lib/okio-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/gson-2.9.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-1.4.10.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-redis-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-http2-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-util-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/zookeeper-3.6.3.jar:/server/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/server/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-buffer-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-xml-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/server/hadoop/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/server/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-socks-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-io-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/server/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/zookeeper-jute-3.6.3.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/server/hadoop/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/server/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-common-1.4.10.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-http-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/server/hadoop/share/hadoop/hdfs/lib/okhttp-4.9.3.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-security-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/server/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-http-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/HikariCP-java7-2.4.12.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-io-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-transport-udt-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/server/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/server/hadoop/share/hadoop/hdfs/lib/netty-all-4.1.89.Final.jar:/server/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/server/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/server/hadoop/share/hadoop/hdfs/lib/commons-compress-1.21.jar:/server/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6-tests.jar:/server/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6-tests.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6-tests.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.6.jar:/server/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6.jar:/server/hadoop/share/hadoop/yarn:/server/hadoop/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/server/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.68.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-client-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/server/hadoop/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jline-3.9.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/server/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/server/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-annotations-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-jndi-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/server/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/server/hadoop/share/hadoop/yarn/lib/objenesis-2.6.jar:/server/hadoop/share/hadoop/yarn/lib/asm-tree-9.4.jar:/server/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-common-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jetty-plus-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jna-5.2.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/server/hadoop/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-api-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-client-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/server/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.68.jar:/server/hadoop/share/hadoop/yarn/lib/asm-commons-9.4.jar:/server/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-servlet-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/websocket-server-9.4.51.v20230217.jar:/server/hadoop/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/server/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/server/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/server/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/server/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.3.6.jar:/server/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.6.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1be78238728da9266a4f88195058f08fd012bf9c; compiled by 'ubuntu' on 2023-06-18T08:22Z STARTUP_MSG: java = 1.8.0_401 ************************************************************/ 2025-08-23 21:29:44,272 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2025-08-23 21:29:44,656 INFO org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/server/hadoop/data/dn 2025-08-23 21:29:44,792 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties 2025-08-23 21:29:44,866 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-08-23 21:29:44,866 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2025-08-23 21:29:45,210 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-08-23 21:29:45,231 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576 2025-08-23 21:29:45,236 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is master 2025-08-23 21:29:45,236 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-08-23 21:29:45,241 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0 2025-08-23 21:29:45,264 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:9866 2025-08-23 21:29:45,266 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwidth is 104857600 bytes/s 2025-08-23 21:29:45,266 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 100 2025-08-23 21:29:45,304 INFO org.eclipse.jetty.util.log: Logging initialized @1787ms to org.eclipse.jetty.util.log.Slf4jLog 2025-08-23 21:29:45,503 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /root/hadoop-http-auth-signature-secret 2025-08-23 21:29:45,513 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined 2025-08-23 21:29:45,524 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2025-08-23 21:29:45,527 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2025-08-23 21:29:45,527 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2025-08-23 21:29:45,528 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2025-08-23 21:29:45,572 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 41547 2025-08-23 21:29:45,573 INFO org.eclipse.jetty.server.Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 1.8.0_401-b10 2025-08-23 21:29:45,614 INFO org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0 2025-08-23 21:29:45,614 INFO org.eclipse.jetty.server.session: No SessionScavenger set, using defaults 2025-08-23 21:29:45,615 INFO org.eclipse.jetty.server.session: node0 Scavenging every 600000ms 2025-08-23 21:29:45,632 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4fad9bb2{logs,/logs,file:///server/hadoop/logs/,AVAILABLE} 2025-08-23 21:29:45,633 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@1941a8ff{static,/static,file:///server/hadoop/share/hadoop/hdfs/webapps/static/,AVAILABLE} 2025-08-23 21:29:45,708 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@35390ee3{datanode,/,file:///server/hadoop/share/hadoop/hdfs/webapps/datanode/,AVAILABLE}{file:/server/hadoop/share/hadoop/hdfs/webapps/datanode} 2025-08-23 21:29:45,720 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@66565121{HTTP/1.1, (http/1.1)}{localhost:41547} 2025-08-23 21:29:45,720 INFO org.eclipse.jetty.server.Server: Started @2203ms 2025-08-23 21:29:45,796 WARN org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Got null for restCsrfPreventionFilter - will not do any filtering. 2025-08-23 21:29:45,892 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:9864 2025-08-23 21:29:45,904 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root 2025-08-23 21:29:45,905 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup 2025-08-23 21:29:45,914 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor 2025-08-23 21:29:46,034 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false. 2025-08-23 21:29:46,052 INFO org.apache.hadoop.ipc.Server: Listener at 0.0.0.0:9867 2025-08-23 21:29:46,055 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9867 2025-08-23 21:29:46,463 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:9867 2025-08-23 21:29:46,501 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2025-08-23 21:29:46,517 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Unable to get NameNode addresses. java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address.[] or dfs.namenode.rpc-address.[] is not configured. at org.apache.hadoop.hdfs.DFSUtil.getNNServiceRpcAddressesForCluster(DFSUtil.java:656) at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:157) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1755) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:564) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:3148) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:3054) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:3098) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:3242) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:3266) 2025-08-23 21:29:46,558 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.w.WebAppContext@35390ee3{datanode,/,null,STOPPED}{file:/server/hadoop/share/hadoop/hdfs/webapps/datanode} 2025-08-23 21:29:46,565 INFO org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@66565121{HTTP/1.1, (http/1.1)}{localhost:0} 2025-08-23 21:29:46,565 INFO org.eclipse.jetty.server.session: node0 Stopped scavenging 2025-08-23 21:29:46,565 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@1941a8ff{static,/static,file:///server/hadoop/share/hadoop/hdfs/webapps/static/,STOPPED} 2025-08-23 21:29:46,565 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@4fad9bb2{logs,/logs,file:///server/hadoop/logs/,STOPPED} 2025-08-23 21:29:46,573 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Waiting up to 30 seconds for transfer threads to complete 2025-08-23 21:29:46,574 INFO org.apache.hadoop.ipc.Server: Stopping server on 9867 2025-08-23 21:29:46,590 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping DataNode metrics system... 2025-08-23 21:29:46,590 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system stopped. 2025-08-23 21:29:46,590 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system shutdown complete. 2025-08-23 21:29:46,596 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Shutdown complete. 2025-08-23 21:29:46,596 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain java.io.IOException: No services to connect, missing NameNode address. at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:165) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1755) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:564) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:3148) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:3054) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:3098) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:3242) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:3266) 2025-08-23 21:29:46,598 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.io.IOException: No services to connect, missing NameNode address. 2025-08-23 21:29:46,634 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at master/192.168.88.8 ************************************************************/
最新发布
08-24
在 Hadoop 的 `hdfs-site.xml` 文件中，必须为每个 NameNode 配置以下属性： ```xml <name>dfs.namenode.rpc-address</name> <value>namenode-hostname:8020</value> </property> ``` 其中 `namenode-hostname`...
<name>yarn.application.classpath</name> <value>/opt/hadoop-3.4.0/etc/hadoop:/opt/hadoop-3.4.0/share/hadoop/common/lib/*:/opt/hadoop-3.4.0/share/hadoop/common/*:/opt/hadoop-3.4.0/share/hadoop/hdfs:/opt/hadoop-3.4.0/share/hadoop/hdfs/lib/*:/opt/hadoop-3.4.0/share/hadoop/hdfs/*:/opt/hadoop-3.4.0/share/hadoop/mapreduce/*:/opt/hadoop-3.4.0/share/hadoop/yarn:/opt/hadoop-3.4.0/share/hadoop/yarn/lib/*:/opt/hadoop-3.4.0/share/hadoop/yarn/*
03-16
$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/...
Hadoop 配置yarn常见错误
刘洋_heaven的博客
10-18 2218
1.报错时环境及配置 hadoop环境为一主三从，在namenode上启用yarn，配置yarn-site.xml，三个从节点yarn-site.xml配置文件为空。 yarn-site.xml的内容如下 <configuration> <property> <name>yarn.nodemanager.a...
HADOOP及YARN常见错误
innersense的专栏
05-09 585
1 edits_log越积越多的问题 HADOOP 2.6.X版本，在非HA模式下，仍然使用的的是secondarynamenode合并editslog的方法 。 当secondarynamenode不能正确合并的时候，有可能是secondaryname的堆太小了，如果edtislog的文件太多， 就会导致合并失败。 可以调整 # The maximum amount of heap to use, in MB. Default is 1000. export HADOOP_HEAPSIZE=2
File does not exist: hdfs://xxx:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framewo
前方的路在刚开始
08-06 1196
先用cdh安装yarn的mapredce的jar,然后在看看是不是时间问题，原因是0.5秒同步一次心跳，我其中一台节点，是在其他地方，所以网速不好。
Hadoop常见错误及解决办法汇总
热门推荐
Jacob
01-12 3万+
错误一：java.io.IOException: Incompatible clusterIDs 时常出现在namenode重新格式化之后 ' V0 h# C5 a8 o+ ^1 n * w- t  k( ]$ }+ p6 d  G2014-04-29 14:32:53,877 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Ini
Hadoop2.X中Mapreduce，YARN等配置，启动和常见错误解决办法
ShiLuoHeroKing的博客
03-21 1755
本篇文章主要介绍一下 hadoop2.X 的启动的相关配置和启动过程常见错误的解决办法，废话不多少，下面开始按照官网的描述步骤开始介绍。 1.在opt目录下建立两个目录，分别为 modules,software ,建立这两个目录的目的是为了方便管理软件的安装和卸载 2.安装jdk ,由于本文中搭建的是hadoop2.X的环境，其中使用YARN，所以对jdk的版本有要求，最低为1
YARN调试过程中的常见问题
weixin_33722405的博客
08-07 1439
执行操作： hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.0.jar wordcount /user/today/input /user/today/output 报错： 错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.a...
hadoop异常
weixin_33982670的博客
03-13 284
1. org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan t...
Hbase安装常见问题总结
小程故事多
03-21 821
    最近这段时间因为工作需要安装了hadoop+hbase集群，在安装的过程中总结了以下一些常见的问题，供大家参考。   一、问题1 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.  1.       删除/tmp下hadoop 有关系的东...
解决开启hadoop时,NameNode起不来的记录
m0_66693152的博客
03-14 1493
当时是开启hadoop集群的时候，不小心关机了，然后在开启的时候就启动不起来了。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值