How to load different version of Spark into Oozie

This article explains the steps needed to load Spark2 into Oozie under CDH5.9.x which comes with Spark1.6. Although this was tested under CDH5.9.0, it should be similar for earlier releases. Please follow the steps below: Locate the current shared-lib directory by running: oozie admin -oozie http://<oozie-server-host>:11000/oozie -sharelibupdate you will get …

Sqoop Action with –query fails on oozie using tag

Yesterday I have discovered an Oozie bug that it does not handle the –query parameter for sqoop action. See my example sqoop action XML below: <action name="sqoop-ed7d" cred="hive2"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="${nameNode}/user/eric/sqoop-import"> </delete></prepare> <command></command>import –connect jdbc:mysql://node6.lab.cloudera.com/test –username root –password cloudera –target-dir hdfs://node5.lab.cloudera.com:8020/user/eric/sqoop-import –query "SELECT * FROM test …

Oozie “Multiple “ok to” Transitions To The Same Node Are Not Allowed

I have been working with Oozie for quite a few weeks, and the experience so far has been quite positive. It is quite easy to learn, given that you understands XML and Hadoop ecosystem. However, there is one limitation that is quite annoying, although I understand the purpose behind the …