Oozie: sqoop 1.4.4 sharelib

最新推荐文章于 2019-12-29 18:37:02 发布

原创最新推荐文章于 2019-12-29 18:37:02 发布 · 180 阅读

CC 4.0 BY-SA版权

文章标签：

8 篇文章

订阅专栏

本文介绍了解决Oozie 4.0.1在Hadoop 2.3.0环境中运行Sqoop示例任务时遇到的问题。由于默认的Sqoop共享库未针对Hadoop 2.x进行适配，作者手动更新了Oozie中的Sqoop共享库，并提供了详细的步骤说明。

ENV : oozie4.0.1 hadoop2.3.0 sqoop1.4.4

When run sqoop example with command

oozie job --oozie http://localhost:11000/oozie --config examples/apps/sqoop/job.properties -run

some errors produced due to the default sqoop sharelib is not targeted to hadoop2.x when I compile oozie

with commands

mvn clean package assembly:single -Dhadoop.version=2.3.0 \
-DjavaVersion=1.7 -DtargetJavaVersion=1.7 -DskipTests -DgenerateDocs

So, I update the sharelib for sqoop maually by following steps:

a. download sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz and untar it

b. bakeup jar in oozie-4.0.1/share/lib/sqoop to sqoopjar

c. delete jars in oozie-4.0.1/share/lib/sqoop

d. cp all jars in sqoop-1.4.4.bin__hadoop-2.0.4-alpha/lib to oozie-4.0.1/share/lib/sqoop

e. cp jar sqoop-1.4.4.bin__hadoop-2.0.4-alpha/sqoop-1.4.4.jar to oozie-4.0.1/share/lib/sqoop

f. cp jar sqoopjar/oozie-sharelib-sqoop-4.0.1.jar to oozie-4.0.1/share/lib/sqoop

g. delete share/lib/sqoop in hdfs and update it by

oozie-setup.sh sharelib upgrade -fs hdfs://192.168.122.1:2014 -locallib share/

If you should use mysql jdbc driver, don't forget to update mysql jdbc driver to sharelib