You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:44:23 UTC
[jira] [Resolved] (SPARK-25329) Support passing Kerberos
configuration information
[ https://issues.apache.org/jira/browse/SPARK-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-25329.
----------------------------------
Resolution: Incomplete
> Support passing Kerberos configuration information
> --------------------------------------------------
>
> Key: SPARK-25329
> URL: https://issues.apache.org/jira/browse/SPARK-25329
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Dale Richardson
> Priority: Major
> Labels: bulk-closed
>
> The current V2 Datasource API provides support for querying a portion of the SparkConfig namespace (spark.datasource.*) via the SessionConfigSupport API. This was designed with the assumption that all configuration information for v2 data sources should be separate from each other.
> Unfortunately, there are some cross-cutting concerns such as authentication that touch multiple data sources - this means that common configuration items need to be shared amongst multiple data sources.
> In particular, Kerberos setup can use the following configuration items:
> # userPrincipal, spark configuration:: spark.yarn.principal
> # userKeytabPath spark configuration: spark.yarn.keytab
> # krb5ConfPath: java.security.krb5.conf
> # kerberos debugging flag: sun.security.krb5.debug
> # spark.security.credentials.${service}.enabled
> # JAAS config: java.security.auth.login.config ??
> # ZKServerPrincipal ??
> So potential solutions to pass this information to various data sources are:
> # Pass the entire SparkContext object to data sources (not likely)
> # Pass the entire SparkConfig Map object to data sources
> # Pass all required configuration via environment variables
> # Extend SessionConfigSupport to support passing specific white-listed configuration values
> # Add a specific data source v2 API "SupportsKerberos" so that a data source can indicate that it supports Kerberos and also provide the means to pass needed configuration info.
> # Expand out all Kerberos configuration items to be in each data source config namespace that needs it.
> If the data source requires TLS support then we also need to support passing all the configuration values under "spark.ssl.*"
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org