CDH users commonly use YARN setting mapreduce.job.acl-view-job to control which users have access to view YARN application logs through Resource Manager or JobHistory Server web UI. If you have below setting under CM > YARN > Configuration It will allow “user2” to be able to view all MapReduce Job logs …
When copying data across clusters using DistCp action through Oozie in a secured environment, due to a current Oozie bug, the job will fail with below error message: This is a known issue and reported in the upstream JIRA of OOZIE-2900 This JIRA has been fixed in CDH6 and above. …
It is quite common from lots of Hadoop clusters that after enabling Load Balancer for certain services that have Kerberos already, the connection to those services via Load Balancer will fail. However, the direct connection to them will always successful. The typical errors returned looks like below: In case of …
Oozie is an orchestration system for managing Hadoop jobs. Currently it supports various actions, including, but not limited to, Spark1, Spark2, Shell, SSH, Hive, Sqoop and Java etc. However, due to certain business requirements, sometimes we want to disable some of the actions so that we can control how users …
I have seen quite a few CDH users who try to run chained Linux command via Oozie’s SSH Action. Example is like below: We can see that the command to run on remote host is as below: This is OK if both commands can finish successfully very quickly. However it …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!