[jira] [Created] (SPARK-25329) Support passing Kerberos configuration information

["Dale Richardson (JIRA)" <ji...@apache.org>]

Dale Richardson created SPARK-25329:
---------------------------------------

             Summary: Support passing Kerberos configuration information
                 Key: SPARK-25329
                 URL: https://issues.apache.org/jira/browse/SPARK-25329
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 2.3.1
            Reporter: Dale Richardson


The current V2 Datasource API provides support for querying a portion of the SparkConfig namespace (spark.datasource.*) via the SessionConfigSupport API.  This was designed with the assumption that all configuration information for v2 data sources should be separate from each other.

Unfortunately, there are some cross-cutting concerns such as authentication that touch multiple data sources - this means that common configuration items need to be shared amongst multiple data sources.

In particular, Kerberos setup can use the following configuration items:
 # userPrincipal, spark configuration:: spark.yarn.principal
 # userKeytabPath spark configuration: spark.yarn.keytab
 # krb5ConfPath:  java.security.krb5.conf
 # kerberos debugging flag: sun.security.krb5.debug 
 # spark.security.credentials.${service}.enabled
 # JAAS config: java.security.auth.login.config ??
 # ZKServerPrincipal ??

So potential solutions to pass this information to various data sources are:
 # Pass the entire SparkContext object to data sources (not likely)
 # Pass the entire SparkConfig Map object to data sources
 # Pass all required configuration via environment variables
 # Extend SessionConfigSupport to support passing specific white-listed configuration values
 # Add a specific data source v2 API "SupportsKerberos" so that a data source can indicate that it supports Kerberos and also provide the means to pass needed configuration info.
 # Expand out all Kerberos configuration items to be in each data source config namespace that needs it.

If the data source requires TLS support then we also need to support passing all the  configuration values under  "spark.ssl.*"

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org