本文介绍了在Hadoop集群中集成HBase时遇到的问题及解决方案。作者尝试使用-libjars选项提供所需的JAR文件但未成功。最终通过修改HADOOP_CLASSPATH并设置HADOOP_USER_CLASSPATH_FIRST为true解决了找不到HBaseConfiguration类的问题。
As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath...
So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.
I got stuck at this...
Then a long weekend.
Today is monday. I continue work on this. I should mention that I have read the articles like these:
[url]http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/[/url]
[url]http://grepalex.com/2013/02/25/hadoop-libjars/[/url]
But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!
So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.
I got stuck at this...
Then a long weekend.
Today is monday. I continue work on this. I should mention that I have read the articles like these:
[url]http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/[/url]
[url]http://grepalex.com/2013/02/25/hadoop-libjars/[/url]
But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
