“Client cannot authenticate via:[TOKEN, KERBEROS]” error in Oozie DistCp Action

When copying data across clusters using DistCp action through Oozie in a secured environment, due to a current Oozie bug, the job will fail with below error message: java.io.IOException: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "xxxxxxxxxx/xxx.xxx.xxx.xxx"; destination host is: …

Oozie LB Connection Failed With GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Error

It is quite common from lots of Hadoop clusters that after enabling Load Balancer for certain services that have Kerberos already, the connection to those services via Load Balancer will fail. However, the direct connection to them will always successful. The typical errors returned looks like below: GSSException: No valid …

Oozie SSH Action Does Not Support Chained Commands – OOZIE-1974

I have seen quite a few CDH users who try to run chained Linux command via Oozie’s SSH Action. Example is like below: <action name="sshTest"> <ssh xmlns="uri:oozie:ssh-action:0.1"> <host>${sshUserHost}</host> <command>kinit test.keytab test@TEST.COM ; python ….</command> <capture-output/> </ssh> <ok to="nextActino"/> <error to="kill"/> </action> We can see that the command to run on …

Spark Job SASL Authentication Error

Spark has an internal mechanism that authenticates executors with the driver controlling a given application. This can be controlled by setting “spark.authenticate” to “true”, as part of spark-submit’s parameters, like below: spark-submit –master yarn-cluster –conf spark.authenticate=true –conf spark.dynamicAllocation.enabled=true …. This setting is required if you have “spark.authenticate.enableSaslEncryption” enabled, or called …