Pig: using hive metastore in oozie

When we run a pig job which using hive metastore table through hue. We need to locate all related jars to oozie sharelib

 

Prepare

a. compile hive-0.12.0 and hive-0.13.0 to against hadoop2.3.0

b. compile pig-0.12.0 to against hadoop2.3.0

 

Update local pig's sharelib

a. backups all jars in share/lib/pig

b. copy pig-0.12.0.jar  pig-0.12.0-withouthadoop.jar to  share/lib/pig

c. copy oozie-sharelib-pig-4.0.1.jar from backuped jars in share/lib/pig

d.  copy all jars in hive-0.13.0-bin/hcatalog/share/hcatalog  to share/lib/pig

e. copy all jars in hive-0.13.0-bin/lib to share/lib/pig

f.  copy mysql jdbc driver to share/lib/pig

Update hdfs's pig sharelib

a. delete share/lib/pig in hdfs

b. update sharelib in hdfs using

#oozie-setup.sh sharelib upgrade -fs hdfs://192.168.122.1:2014 -locallib share/   

 

Integrate with Hue

a.init hive metastore schema to 0.13.0 using

#schematool -dbType mysql -initSchemaTo 0.13.0

b.set hue configuration to use hive-0.12.0(not hive-0.13.0)