You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2017/02/13 09:34:41 UTC
[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on
Mesos
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863371#comment-15863371 ]
Saisai Shao edited comment on SPARK-16742 at 2/13/17 9:34 AM:
--------------------------------------------------------------
The proposed solution is quite different from what existed in Spark on YARN. IIUC this solution looks doesn't honor delegation token, and wraps every HDFS operation with {{executeSecure}}, I simply doubt that this approach requires other components, like sql, streaming, should also know the existence of such APIs and try to wrap them. Also if newly added codes ignore this wrapper, this will lead to error. From my understanding it is quite intrusive.
Also how do you handle principal and keytab for driver/executors, do you need to ship keytab to every nodes and who is responsible for this?
And looks from your PR what you mainly focused is user impersonation, this is slightly different from what this JIRA mentioned about, also your main requirement is dynamic proxy user change, I would suggest to use another JIRA to track this, since this is a little different from support Kerberos in Mesos.
was (Author: jerryshao):
The proposed solution is quite different from what existed in Spark on YARN. IIUC this solution looks doesn't honor delegation token, and wraps every HDFS operation with {{executeSecure}}, I simply doubt that this approach requires other components, like sql, streaming, should also know the existence of such APIs and try to wrap them. Also if newly added codes ignore this wrapper, this will lead to error. From my understanding it is quite intrusive.
> Kerberos support for Spark on Mesos
> -----------------------------------
>
> Key: SPARK-16742
> URL: https://issues.apache.org/jira/browse/SPARK-16742
> Project: Spark
> Issue Type: New Feature
> Components: Mesos
> Reporter: Michael Gummelt
>
> We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be contributing it to Apache Spark soon.
> Mesosphere design doc: https://docs.google.com/document/d/1xyzICg7SIaugCEcB4w1vBWp24UDkyJ1Pyt2jtnREFqc/edit#heading=h.tdnq7wilqrj6
> Mesosphere code: https://github.com/mesosphere/spark/commit/73ba2ab8d97510d5475ef9a48c673ce34f7173fa
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org