apache的hadoop升级到CDH hadoop2.0时遇到的问题及解决

1：引入的jar包

1.X版本有hadoop-core包；而2.x没有

如果你需要hdfs就引入\share\hadoop\common\lib + 

hadoop-common-2.0.0-cdh4.6.0.jar +

hadoop-hdfs-2.0.0-cdh4.6.0.jar +

\share\hadoop\yarn\*

 

如果你要MR则加上

\share\hadoop\mapreduce1\* 或

\share\hadoop\mapreduce2\* 

 

2：遇到权限问题的时候最好是把权限赋给当前的执行用户

chown hadoop.hadoop ./logs

 

3：No FileSystem for scheme: hdfs,No FileSystem for scheme: file

http://blog.newitfarmer.com/big_data/big-data-platform/hadoop/13953/repost-no-filesystem-for-scheme-hdfsno-filesystem-for-scheme-file

我的办法是加上

Configuration config = new Configuration();
config.set("fs.hdfs.impl",org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());