hadoop运行mapreduce报错Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRApp

原创 于 2022-12-20 20:09:32 发布 · 923 阅读 · 8 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#hadoop #mapreduce #大数据 #yarn #报错

在运行Hadoop的Wordcount示例时遇到Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster。错误信息提示检查mapred-site.xml配置。解决方案是使用`hadoop classpath`命令获取classpath，然后将该路径添加到mapred-site.xml的yarn.app.mapreduce.am.env，mapreduce.map.env和mapreduce.reduce.env的value中。确保配置的value与命令输出的classpath一致。

异常问题：

今天运行word count单词统计示例，hadoop运行mapreduce报错Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

详细错误信息如下：
  1. Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
  3. Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
  4. <property>
  5. <name>yarn.app.mapreduce.am.env</name>
  6. <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
  7. </property>
  8. <property>
  9. <name>mapreduce.map.env</name>
  10. <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
  11. </property>
  12. <property>
  13. <name>mapreduce.reduce.env</name>
  14. <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
  15. </property>

解决办法：

先手hadoop classpath命令得到classpath:
hadoop classpath
如下：
  1. [root@hd1 mapreduce]# hadoop classpath
  2. /home/hadoop-3.1.4/etc/hadoop:/home/hadoop-3.1.4/share/hadoop/common/lib/*:/home/hadoop-3.1.4/share/hadoop/common/*:/home/hadoop-3.1.4/share/hadoop/hdfs:/home/hadoop-3.1.4/share/hadoop/hdfs/lib/*:/home/hadoop-3.1.4/share/hadoop/hdfs/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/lib/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/*:/home/hadoop-3.1.4/share/hadoop/yarn:/home/hadoop-3.1.4/share/hadoop/yarn/lib/*:/home/hadoop-3.1.4/share/hadoop/yarn/*

再修改mapred-site.xml配置，增加如下：
  1. <property>
  2. <name>yarn.app.mapreduce.am.env</name> <value>HADOOP_MAPRED_HOME=/home/hadoop-3.1.4/etc/hadoop:/home/hadoop-3.1.4/share/hadoop/common/lib/*:/home/hadoop-3.1.4/share/hadoop/common/*:/home/hadoop-3.1.4/share/hadoop/hdfs:/home/hadoop-3.1.4/share/hadoop/hdfs/lib/*:/home/hadoop-3.1.4/share/hadoop/hdfs/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/lib/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/*:/home/hadoop-3.1.4/share/hadoop/yarn:/home/hadoop-3.1.4/share/hadoop/yarn/lib/*:/home/hadoop-3.1.4/share/hadoop/yarn/*</value>
  3. </property>
  4. <property>
  5. <name>mapreduce.map.env</name>
  6. <value>HADOOP_MAPRED_HOME=/home/hadoop-3.1.4/etc/hadoop:/home/hadoop-3.1.4/share/hadoop/common/lib/*:/home/hadoop-3.1.4/share/hadoop/common/*:/home/hadoop-3.1.4/share/hadoop/hdfs:/home/hadoop-3.1.4/share/hadoop/hdfs/lib/*:/home/hadoop-3.1.4/share/hadoop/hdfs/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/lib/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/*:/home/hadoop-3.1.4/share/hadoop/yarn:/home/hadoop-3.1.4/share/hadoop/yarn/lib/*:/home/hadoop-3.1.4/share/hadoop/yarn/*</value>
  7. </property>
  8. <property>
  9. <name>mapreduce.reduce.env</name>
  10. <value>HADOOP_MAPRED_HOME=/home/hadoop-3.1.4/etc/hadoop:/home/hadoop-3.1.4/share/hadoop/common/lib/*:/home/hadoop-3.1.4/share/hadoop/common/*:/home/hadoop-3.1.4/share/hadoop/hdfs:/home/hadoop-3.1.4/share/hadoop/hdfs/lib/*:/home/hadoop-3.1.4/share/hadoop/hdfs/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/lib/*:/home/hadoop-3.1.4/share/hadoop/mapreduce/*:/home/hadoop-3.1.4/share/hadoop/yarn:/home/hadoop-3.1.4/share/hadoop/yarn/lib/*:/home/hadoop-3.1.4/share/hadoop/yarn/*</value>
  11. </property>

注意：上面的配置的value都是上面命令得到的值，复制到value中即可

评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
表情包 代码片
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值