You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@apex.apache.org by Thomas Weise <th...@apache.org> on 2017/07/07 01:05:29 UTC

Kafka operators with Kerberos authentication

Hi,

Has anyone run the Apex Kafka consumer or producer with security enabled?

I got authentication working in embedded mode and looking to deploy to the
cluster. It will require

* keytab
* JAAS config with the KafkaClient settings.
* JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
* config properties:

  <property>

<name>apex.operator.kafkaOutput.prop.properties(security.protocol)</name>
    <value>SASL_SSL</value>
  </property>

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(
sasl.kerberos.service.name)</name>
    <value>kafka</value>
  </property>

I guess the JAAS conf and keytab can be pushed with the FILES argument. Any
other ideas how to set this up?

Thanks,
Thomas

Re: Kafka operators with Kerberos authentication

Posted by Thomas Weise <th...@apache.org>.

The user's keytab needs to be deployed by the application or another user
owned process, just like what would need to occur for YARN.

Thomas

On Fri, Jul 7, 2017 at 3:31 PM, Pramod Immaneni <pr...@datatorrent.com>
wrote:

> Wouldn't the kafka jaas.conf and keytab be already present on the nodes if
> managing the kafka deployment through the distro?
>
> Thanks
>
> On Thu, Jul 6, 2017 at 6:05 PM, Thomas Weise <th...@apache.org> wrote:
>
>> Hi,
>>
>> Has anyone run the Apex Kafka consumer or producer with security enabled?
>>
>> I got authentication working in embedded mode and looking to deploy to
>> the cluster. It will require
>>
>> * keytab
>> * JAAS config with the KafkaClient settings.
>> * JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
>> * config properties:
>>
>>   <property>
>>     <name>apex.operator.kafkaOutput.prop.properties(security.
>> protocol)</name>
>>     <value>SASL_SSL</value>
>>   </property>
>>
>>   <property>
>>     <name>apex.operator.kafkaOutput.prop.properties(sasl.
>> kerberos.service.name)</name>
>>     <value>kafka</value>
>>   </property>
>>
>> I guess the JAAS conf and keytab can be pushed with the FILES argument.
>> Any other ideas how to set this up?
>>
>> Thanks,
>> Thomas
>>
>>
>

Re: Kafka operators with Kerberos authentication

Posted by Pramod Immaneni <pr...@datatorrent.com>.

Wouldn't the kafka jaas.conf and keytab be already present on the nodes if
managing the kafka deployment through the distro?

Thanks

On Thu, Jul 6, 2017 at 6:05 PM, Thomas Weise <th...@apache.org> wrote:

> Hi,
>
> Has anyone run the Apex Kafka consumer or producer with security enabled?
>
> I got authentication working in embedded mode and looking to deploy to the
> cluster. It will require
>
> * keytab
> * JAAS config with the KafkaClient settings.
> * JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
> * config properties:
>
>   <property>
>     <name>apex.operator.kafkaOutput.prop.properties(
> security.protocol)</name>
>     <value>SASL_SSL</value>
>   </property>
>
>   <property>
>     <name>apex.operator.kafkaOutput.prop.properties(sa
> sl.kerberos.service.name)</name>
>     <value>kafka</value>
>   </property>
>
> I guess the JAAS conf and keytab can be pushed with the FILES argument.
> Any other ideas how to set this up?
>
> Thanks,
> Thomas
>
>