执行mr时候报的一个错

最新推荐文章于 2023-05-24 16:17:09 发布

原创最新推荐文章于 2023-05-24 16:17:09 发布 · 222 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#hadoop

hadoop 专栏收录该内容

22 篇文章

订阅专栏

hadoop版本:hadoop-2.2.0-cdh5.0.0-beta-1

今天在执行sqoop往hbase里导数的时候，DN节点报了下面的错:


Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnRuntimeException
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2521)
	at java.lang.Class.getMethod0(Class.java:2764)
	at java.lang.Class.getMethod(Class.java:1653)
	at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 6 more

看异常是相应的jar没找到

为了执行MR,在yarn-site.xml中有这样一个的配置：


        <property>
                <description>Classpath for typical applications.</description>
                <name>yarn.application.classpath</name>
                <value>$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,
                        $HADOOP_COMMON_HOME/share/hadoop/common/lib/*,
                        $HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,
                        $YARN_HOME/share/hadoop/yarn/*,$YARN_HOME/share/hadoop/yarn/lib/*,
                        $YARN_HOME/share/hadoop/mapreduce/*,$YARN_HOME/share/hadoop/mapreduce/lib/*</value>
        </property>

应该是这里的变量没有读到，导致这个原因应该是相应的环境变量没设置，
vi ~/.bashrc

增加：


export HADOOP_MAPRED_HOME=${HADOOP_HOME}
export HADOOP_COMMON_HOME=${HADOOP_HOME}
export HADOOP_HDFS_HOME=${HADOOP_HOME}
export YARN_HOME=${HADOOP_HOME}
export HADOOP_YARN_HOME=${HADOOP_HOME}
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HDFS_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export YARN_CONF_DIR=${HADOOP_HOME}/etc/hadoop

这样设置以后，yarn-site.xml配置的变量就能读到相应的值，上述错误消失。