[jira] [Created] (SPARK-25428) Support plain Kerberos Authentication with Spark

["Shruti Gumma (JIRA)" <ji...@apache.org>]

Shruti Gumma created SPARK-25428:
------------------------------------

             Summary: Support plain Kerberos Authentication with Spark
                 Key: SPARK-25428
                 URL: https://issues.apache.org/jira/browse/SPARK-25428
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.3.1, 2.3.0, 2.2.2, 2.2.1, 2.2.0, 2.1.3, 2.1.2, 2.1.1
            Reporter: Shruti Gumma


Spark should work with plain Kerberos authentication. Currently, Spark can work with Hadoop delegation tokens, but not plain Kerberos. Hadoop's UserGroupInformation(UGI) class is responsible for handling security authentication in Spark. This UserGroupInformation(UGI) has support for Kerberos authentication, as well as Token authentication. Since Spark does not work correctly with the Kerberos auth method, it leads to a gap in fully supporting all the security authentication mechanisms.
 
 If Kerberos is used to login in UserGroupInformation(UGI) using keytabs at the startup of drivers and executors, then Spark does not allow this logged-in UserGroupInformation(UGI) user to correctly propagate. The exception arises from the implementation of the runAsSparkUser method in SparkHadoopUtil.
 
 The runAsSparkUser method in SparkHadoopUtil creates a new UGI based on the current static UGI and then transfers credentials from this current static UGI to the new UGI. This works well with other auth methods, except Kerberos. Transfer credentials implementation is not conducive for Kerberos auth model since it does not transfer all the required internal state of UGI( such as isKeytab and isKrbTkt). For Kerberos, the UGI has to be created from UGI.loginUserFromKeytab method only and not simply by doing a transfer credentials from the previous UGI to the new UGI. 
 
 Ideally, the CoarseGrainedExecutorBackend should login using keytab, similar to MesosCoarseGrainedExecutorBackend.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org