2017-06-27 17:04:08,836 INFO org.apache.hive.spark.client.SparkClientImpl: [stderr-redir-1]: 17/06/27 17:04:08 WARN security.UserGroupInformation: PriviledgedActionException as:testuser (auth:PROXY) via hive/example.hadoop.com@REALM.COM (auth:KERBEROS) cause:org.apache.hadoop.security.AccessControlException: testuser tries to renew a token with renewer hiveIf you do some googling, you should be able to locate the corresponding upstream Hive JIRA for this issue: HIVE-15485. And from this JIRA, you should also be able to identify that the issue was introduced by HIVE-14383. This is due to the fact that Spark needs the principal/keytab passed in via –principal and –keytab options, and does the renewal by copying the keytab to the cluster and handling login to kerberos inside the application. But the option –principal and –keytab could not work with –proxy-user in spark-submit.sh, so at this moment we could support either the token renewal or the impersonation, but not both. The only way to avoid such issue is to upgrade CDH to the version that has the fix for HIVE-15485, which has been fixed in the following releases:
CDH5.8.5 CDH5.9.2 CDH5.10.1, CDH5.10.2 CDH5.11.0, CDH5.11.1Since HIVE-14383 was introduced in the following CDH:
CDH5.8.3, CDH5.8.4, CDH5.8.5 CDH5.9.1, CDH5.9.2 CDH5.10.0, CDH5.10.1, CDH5.10.2 CDH5.11.0, CDH5.11.1This makes the following CDH currently will have such issues:
CDH5.8.3, CDH5.8.4, CDH5.9.1, CDH5.10.0Please deploy the latest maintenance release for your major version to avoid such issue in Hive on Spark. Hope above helps.