You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Georg Heiler <ge...@gmail.com> on 2017/08/01 10:15:48 UTC
multiple users per flink deployment
Hi,
flink currently only seems to support a single kerberos ticket for
deployment. Are there plans to support different users per each job?
regards,
Georg
Re: multiple users per flink deployment
Posted by Eron Wright <er...@gmail.com>.
One of the key challenges is isolation, eg. ensuring that one job cannot
access the credentials of another. The easiest solution today is to use
the YARN deployment mode, with a separate app per job. Meanwhile,
improvements being made under the FLIP-6 banner for 1.4+ are lying
groundwork for a multiuser experience.
Hope this helps!
On Aug 2, 2017 8:29 AM, "Georg Heiler" <ge...@gmail.com> wrote:
> Thanks for the overview.
> Currently a single flink cluster seems to run all tasks with the same
> user. I would want to be able to run each flink job as a separate user
> instead.
>
> The update for separate read/write users is nice though.
> Tzu-Li (Gordon) Tai <tz...@apache.org> schrieb am Mi. 2. Aug. 2017 um
> 10:59:
>
>> Hi,
>>
>> There’s been quite a few requests on this recently on the mailing lists
>> and also mentioned by some users offline, so I think we may need to start
>> with plans to probably support this.
>> I’m CC’ing Eron to this thread to see if he has any thoughts on this, as
>> he was among the first authors driving the Kerberos support in Flink.
>> I’m not really sure if such a feature support makes sense, given that all
>> jobs of a single Flink deployment have full privileges and therefore no
>> isolation in between.
>>
>> Related question: what external service are you trying to authenticate to
>> with different users?
>> If it is Kafka and perhaps you have different users for the consumer /
>> producer, that will be very soon available in 1.3.2, which includes a
>> version bump to Kafka 0.10 that allows multiple independent users within
>> the same JVM through dynamic JAAS configuration.
>> See this mail thread [1] for more detail on that.
>>
>> Cheers,
>> Gordon
>>
>> [1] http://apache-flink-user-mailing-list-archive.2336050.
>> n4.nabble.com/Kafka-0-10-jaas-multiple-clients-td12831.html#a13317
>>
>> On 1 August 2017 at 6:16:08 PM, Georg Heiler (georg.kf.heiler@gmail.com)
>> wrote:
>>
>> Hi,
>>
>> flink currently only seems to support a single kerberos ticket for
>> deployment. Are there plans to support different users per each job?
>>
>> regards,
>> Georg
>>
>>
Re: multiple users per flink deployment
Posted by Georg Heiler <ge...@gmail.com>.
Thanks for the overview.
Currently a single flink cluster seems to run all tasks with the same user.
I would want to be able to run each flink job as a separate user instead.
The update for separate read/write users is nice though.
Tzu-Li (Gordon) Tai <tz...@apache.org> schrieb am Mi. 2. Aug. 2017 um
10:59:
> Hi,
>
> There’s been quite a few requests on this recently on the mailing lists
> and also mentioned by some users offline, so I think we may need to start
> with plans to probably support this.
> I’m CC’ing Eron to this thread to see if he has any thoughts on this, as
> he was among the first authors driving the Kerberos support in Flink.
> I’m not really sure if such a feature support makes sense, given that all
> jobs of a single Flink deployment have full privileges and therefore no
> isolation in between.
>
> Related question: what external service are you trying to authenticate to
> with different users?
> If it is Kafka and perhaps you have different users for the consumer /
> producer, that will be very soon available in 1.3.2, which includes a
> version bump to Kafka 0.10 that allows multiple independent users within
> the same JVM through dynamic JAAS configuration.
> See this mail thread [1] for more detail on that.
>
> Cheers,
> Gordon
>
> [1]
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-0-10-jaas-multiple-clients-td12831.html#a13317
>
> On 1 August 2017 at 6:16:08 PM, Georg Heiler (georg.kf.heiler@gmail.com)
> wrote:
>
> Hi,
>
> flink currently only seems to support a single kerberos ticket for
> deployment. Are there plans to support different users per each job?
>
> regards,
> Georg
>
>
Re: multiple users per flink deployment
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi,
There’s been quite a few requests on this recently on the mailing lists and also mentioned by some users offline, so I think we may need to start with plans to probably support this.
I’m CC’ing Eron to this thread to see if he has any thoughts on this, as he was among the first authors driving the Kerberos support in Flink.
I’m not really sure if such a feature support makes sense, given that all jobs of a single Flink deployment have full privileges and therefore no isolation in between.
Related question: what external service are you trying to authenticate to with different users?
If it is Kafka and perhaps you have different users for the consumer / producer, that will be very soon available in 1.3.2, which includes a version bump to Kafka 0.10 that allows multiple independent users within the same JVM through dynamic JAAS configuration.
See this mail thread [1] for more detail on that.
Cheers,
Gordon
[1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-0-10-jaas-multiple-clients-td12831.html#a13317
On 1 August 2017 at 6:16:08 PM, Georg Heiler (georg.kf.heiler@gmail.com) wrote:
Hi,
flink currently only seems to support a single kerberos ticket for deployment. Are there plans to support different users per each job?
regards,
Georg