[jira] [Created] (SPARK-6918) Secure HBase with Kerberos does not work over YARN

["Dean Chen (JIRA)" <ji...@apache.org>]

Dean Chen created SPARK-6918:
--------------------------------

             Summary: Secure HBase with Kerberos does not work over YARN
                 Key: SPARK-6918
                 URL: https://issues.apache.org/jira/browse/SPARK-6918
             Project: Spark
          Issue Type: New Feature
          Components: YARN
    Affects Versions: 1.3.0, 1.2.1, 1.3.1
            Reporter: Dean Chen


Attempts to access HBase from Spark executors will fail at the auth to the metastore with:

GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)

This is because HBase Kerberos auth token is not send to the executor. Will need to have something similar to obtainTokensForNamenodes(used for HDFS) in yarn/Client.scala. Storm also needed something similar: https://github.com/apache/storm/pull/226

I've created a patch for this that required an HBase dependency in the YARN module that we've been using successfully at eBay but am working on a version that does not require the HBase dependency by calling the class loader. Should be ready in a few days.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org