You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by matd <ma...@gmail.com> on 2017/05/04 09:13:54 UTC
Kerberos impersonation of a Spark Context at runtime
Hi folks,
I have a Spark application executing various jobs for different users
simultaneously, via several Spark sessions on several threads.
My customer would like to kerberize his hadoop cluster. I wonder if there is
a way to configure impersonation such as each of these jobs would be ran
with the different proxy users. From what I see in spark conf and code, it's
not possible to do that at runtime for a specific context, but I'm not
familiar with Kerberos nor with this part of Spark.
Anyone can confirm/infirm this ?
Mathieu
(also on S.O
http://stackoverflow.com/questions/43765044/kerberos-impersonation-of-a-spark-context-at-runtime)
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kerberos-impersonation-of-a-Spark-Context-at-runtime-tp28651.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Kerberos impersonation of a Spark Context at runtime
Posted by Abel Rincón <ga...@gmail.com>.
Hi Mathieu,
Stratio is working on it, we have a solution running which accomplish our
use case, could you share your use case with us?
Here you have video and slides of our work on this topic
https://spark-summit.org/east-2017/events/kerberizing-spark/
Regards
Abel.
2017-05-04 15:01 GMT+02:00 Saisai Shao <sa...@gmail.com>:
> Current Spark doesn't support impersonate different users at run-time.
> Current Spark's proxy user is application level, which means when setting
> through --proxy-user the whole application will be running with that user.
>
> On Thu, May 4, 2017 at 5:13 PM, matd <ma...@gmail.com> wrote:
>
>> Hi folks,
>>
>> I have a Spark application executing various jobs for different users
>> simultaneously, via several Spark sessions on several threads.
>>
>> My customer would like to kerberize his hadoop cluster. I wonder if there
>> is
>> a way to configure impersonation such as each of these jobs would be ran
>> with the different proxy users. From what I see in spark conf and code,
>> it's
>> not possible to do that at runtime for a specific context, but I'm not
>> familiar with Kerberos nor with this part of Spark.
>>
>> Anyone can confirm/infirm this ?
>>
>> Mathieu
>>
>> (also on S.O
>> http://stackoverflow.com/questions/43765044/kerberos-imperso
>> nation-of-a-spark-context-at-runtime)
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-user-list.
>> 1001560.n3.nabble.com/Kerberos-impersonation-of-a-Spark-
>> Context-at-runtime-tp28651.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>
Re: Kerberos impersonation of a Spark Context at runtime
Posted by Saisai Shao <sa...@gmail.com>.
Current Spark doesn't support impersonate different users at run-time.
Current Spark's proxy user is application level, which means when setting
through --proxy-user the whole application will be running with that user.
On Thu, May 4, 2017 at 5:13 PM, matd <ma...@gmail.com> wrote:
> Hi folks,
>
> I have a Spark application executing various jobs for different users
> simultaneously, via several Spark sessions on several threads.
>
> My customer would like to kerberize his hadoop cluster. I wonder if there
> is
> a way to configure impersonation such as each of these jobs would be ran
> with the different proxy users. From what I see in spark conf and code,
> it's
> not possible to do that at runtime for a specific context, but I'm not
> familiar with Kerberos nor with this part of Spark.
>
> Anyone can confirm/infirm this ?
>
> Mathieu
>
> (also on S.O
> http://stackoverflow.com/questions/43765044/kerberos-
> impersonation-of-a-spark-context-at-runtime)
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Kerberos-impersonation-of-a-
> Spark-Context-at-runtime-tp28651.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>