Hadoop学习-错误记录

1.NameNode线程无法启动
解决办法：删除/usr/local/Hadoop/hadoop-1.1.2/libexec/../logs/目录下的所有log文件，然后删除hadoop.tmp.dir路径下的所有文件，最后重新格式化hdfshadoop namenode -format。在master和slave节点上分别执行一遍

2.对HDFS进行格式化操作，在shutdown时报UnkonwHostException.
原因：master节点和slave节点的hostname配置错误，导致找不到主机。
解决办法：修改hostname或hosts中的DNS使其相匹配。

3.在eclipse中的plugins中添加hadoop-plugins.jar后，eclipse读取hadoop插件失败。
原因：在安装新的插件以前你启动过 eclipse ，在 org.eclipse.update 文件夹下记录了插件的历史更新情况，它只记忆了以前的插件更新情况，而你新安装的插件它并不记录
解决办法：（1）删掉该文件夹即可，再次启动eclipse时，会扫描所有插件，导致启动较慢。
（2）在 eclipse\configuration 目录下的 config.ini 文件中加入一行 : osgi.checkConfiguration=true
这样它就会寻找并安装插件 , 找到插件后可以把该行注释掉 ( 去掉 ), 这样以后每次启动就不会因为寻找插件而显得慢了

4.在window上利用eclipse访问hdfs时，报错AccessControlException。
解决办法：修改 /hadoop_home/conf 下的hdfs-site.xml，增加dfs.permissions为false

5.集群环境启动datanode失败。
查看日志信息如下：

很明显master和slave的namespaceID不匹配

解决办法：修改master或slave节点的namespaceID使之匹配，
进入该目录：

编辑VERSION：

6.运行mapper时出现如下错误：Failed to set permission to path

解决办法：
下载hadoop1.1.2源码，修改org.apache.hadoop.fs.FileUtil.java，注释掉checkReturnValue方法中的方法体，同时，在本地开发环境中创建org.apache.hadoop.fs包，并将该类方法该包中即可完美运行。

7.Hadoop2.6.0启动报错，错误如下：

VM: ssh: Could not resolve hostname VM: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
Java: ssh: Could not resolve hostname Java: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
to: ssh: Could not resolve hostname to: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name

解决办法：
在profile文件中添加如下两行：

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

8.Hadoop Iterable只能遍历一次
解决办法，复制内容到List中。