在yarn集群上面跑mapReduce的wordcount方法

hadoop jar /export/servers/hadoop3.3.5/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.5.jar wordcount    /bigdata(要统计的文件所在的文件夹)     /test/output(输出文件夹)

出现报错

2024-12-27 19:59:33,518 INFO mapreduce.Job:  map 0% reduce 0%
2024-12-27 19:59:33,532 INFO mapreduce.Job: Job job_1735300724312_0001 failed with state FAILED to AM Container for appattempt_1735300724312_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2024-12-27 19:59:32.704]Exception from container-launch.
Container id: container_1735300724312_0001_02_000001
Exit code: 1

[2024-12-27 19:59:32.731]Container exited with a non-zero exit code 1. Error file: prelaunch.er
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster


[2024-12-27 19:59:32.732]Container exited with a non-zero exit code 1. Error file: prelaunch.er
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster

解决方法

在配置文件yarn-site.xml中添加Hadoop classpath。

首先输入hadoop classpath 查看,然后把输出的这些路径全都复制上

打开yarn-site.xml添加classpath如下 (vi /../hadoop3.3.5/etc/hadoop/yarn-site.xml)

<property>
        <name>yarn.application.classpath</name>
	<value>/export/servers/hadoop3.3.5/etc/hadoop:/export/servers/hadoop3.3.5/share/hadoop/common/lib/*:/export/servers/hadoop3.3.5/share/hadoop/common/*:/export/servers/hadoop3.3.5/share/hadoop/hdfs:/export/servers/hadoop3.3.5/share/hadoop/hdfs/lib/*:/export/servers/hadoop3.3.5/share/hadoop/hdfs/*:/export/servers/hadoop3.3.5/share/hadoop/mapreduce/*:/export/servers/hadoop3.3.5/share/hadoop/yarn:/export/servers/hadoop3.3.5/share/hadoop/yarn/lib/*:/export/servers/hadoop3.3.5/share/hadoop/yarn/* </value>
   </property>

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值