Connecting to a service (Eg: Kafka, Cassandra) with different keytabs from the same cluster

["Kumar Bolar, Harshith" <hk...@arity.com>]

Hi all,
We have a central Flink cluster which will be used by multiple different teams (Data Science, Engineering etc). Each team has their own user and keytab to connect to services like Kafka, Cassandra etc. How should the jobs be configured such that different jobs use different keytabs and principals to connect to Kafka?
Right now with a single user, we have the following entry in the jaas.conf file.
KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useTicketCache=false
   renewTicket=true
   useKeyTab=true
   keyTab="/opt/certs/flink-user.keytab"
   serviceName="kafka"
   principal="flink-user@TEST.ABC.COM";
};
How can I add more users like this and reference them in the jobs separately?
Thanks,
Harshith

Re: Connecting to a service (Eg: Kafka, Cassandra) with different keytabs from the same cluster

[Dawid Wysakowicz <dw...@apache.org>]

Hi Harshith,

I haven't tried it, but for Kafka you should be able to use the dynamic
sasl configuration of the underlying KafkaConsumer. Try setting the
`sasl.jaas.config` parameter for the FlinkKafkaConsumer as per the Kafka
documentation.

As far as I know if you use a Flink's specific way of distributing
security contexts[1], you can provide only a single credentials for a
single Flink cluster.

Best,

Dawid


[1]
https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html#kerberos-authentication-setup-and-configuration

On 20/05/2019 10:15, Kumar Bolar, Harshith wrote:
>
> Hi all,
>
> We have a central Flink cluster which will be used by multiple
> different teams (Data Science, Engineering etc). Each team has their
> own user and keytab to connect to services like Kafka, Cassandra etc.
> How should the jobs be configured such that different jobs use
> different keytabs and principals to connect to Kafka?
>
> Right now with a single user, we have the following entry in the
> jaas.conffile.
>
> KafkaClient {
>
>    com.sun.security.auth.module.Krb5LoginModule required
>
>    useTicketCache=false
>
>    renewTicket=true
>
>    useKeyTab=true
>
>    keyTab="/opt/certs/flink-user.keytab"
>
>    serviceName="kafka"
>
>    principal="flink-user@TEST.ABC.COM";
>
> };
>
> How can I add more users like this and reference them in the jobs
> separately?
>
> Thanks,
>
> Harshith
>