Re: [EXTERNAL] kafka

["hsy541@gmail.com" <hs...@gmail.com>]

Hey Raja,

The setup for secure kafka input operator is not easy. You can follow these
steps.
1. Follow kafka document to setup your brokers properly (
https://kafka.apache.org/090/documentation.html#security_overview)
2. You have to manually create the client JAAS file (
https://kafka.apache.org/090/documentation.html#security_overview)
   a sample file would look like this:
     KafkaClient {

        com.sun.security.auth.module.Krb5LoginModule required
        useKeyTab=true
        storeKey=true
        keyTab="/etc/security/keytabs/kafka_client.keytab"
        principal="kafka-client-1@EXAMPLE.COM";
    };

3. On the operator you have to set the attribute JVM_OPTS
 -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf

4. On the operator you have to set the consumerProperties, for example

set dt.operator.$your_operator_name.{consumerProps.security.protocol}  to
 SASL or other security function you use
set dt.operator.$your_operator_name.{consumerPrope.
sasl.kerberos.service.name} to kafka


Hope this would help you!


Regards,
Siyuan



On Sun, Oct 30, 2016 at 10:56 PM, Raja.Aravapalli <
Raja.Aravapalli@target.com> wrote:

>
>
> Hi Team,
>
>
>
> Can someone pls help me with below requested information ?
>
>
>
> Does apache apex have any inbuilt kafka input operator to read from Kafka
> 0.9 secure kafka topics?
>
>
>
> Thanks a lot.
>
>
>
> Regards,
>
> Raja.
>
>
>
> *From: *"Raja.Aravapalli" <Ra...@target.com>
> *Reply-To: *"users@apex.apache.org" <us...@apex.apache.org>
> *Date: *Monday, October 24, 2016 at 2:29 PM
> *To: *"users@apex.apache.org" <us...@apex.apache.org>
> *Subject: *[EXTERNAL] kafka
>
>
>
>
>
> Hi,
>
>
>
> Do we have any kaka operator readily available to consume messages from
> secure kafka topics in kafka 0.9 clusters?
>
>
>
> Thanks a lot.
>
>
>
>
>
> Regards,
>
> Raja.
>

Re: [EXTERNAL] kafka

[Thomas Weise <th...@apache.org>]

IMO this should be added here:

http://apex.apache.org/docs/malhar/operators/kafkaInputOperator/


On Tue, Nov 1, 2016 at 5:24 PM, hsy541@gmail.com <hs...@gmail.com> wrote:

> Hey Raja,
>
> The setup for secure kafka input operator is not easy. You can follow
> these steps.
> 1. Follow kafka document to setup your brokers properly (
> https://kafka.apache.org/090/documentation.html#security_overview)
> 2. You have to manually create the client JAAS file (
> https://kafka.apache.org/090/documentation.html#security_overview)
>    a sample file would look like this:
>      KafkaClient {
>
>         com.sun.security.auth.module.Krb5LoginModule required
>         useKeyTab=true
>         storeKey=true
>         keyTab="/etc/security/keytabs/kafka_client.keytab"
>         principal="kafka-client-1@EXAMPLE.COM";
>     };
>
> 3. On the operator you have to set the attribute JVM_OPTS
>  -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf
>
> 4. On the operator you have to set the consumerProperties, for example
>
> set dt.operator.$your_operator_name.{consumerProps.security.protocol}  to
>  SASL or other security function you use
> set dt.operator.$your_operator_name.{consumerPrope.sasl.
> kerberos.service.name} to kafka
>
>
> Hope this would help you!
>
>
> Regards,
> Siyuan
>
>
>
> On Sun, Oct 30, 2016 at 10:56 PM, Raja.Aravapalli <
> Raja.Aravapalli@target.com> wrote:
>
>>
>>
>> Hi Team,
>>
>>
>>
>> Can someone pls help me with below requested information ?
>>
>>
>>
>> Does apache apex have any inbuilt kafka input operator to read from Kafka
>> 0.9 secure kafka topics?
>>
>>
>>
>> Thanks a lot.
>>
>>
>>
>> Regards,
>>
>> Raja.
>>
>>
>>
>> *From: *"Raja.Aravapalli" <Ra...@target.com>
>> *Reply-To: *"users@apex.apache.org" <us...@apex.apache.org>
>> *Date: *Monday, October 24, 2016 at 2:29 PM
>> *To: *"users@apex.apache.org" <us...@apex.apache.org>
>> *Subject: *[EXTERNAL] kafka
>>
>>
>>
>>
>>
>> Hi,
>>
>>
>>
>> Do we have any kaka operator readily available to consume messages from
>> secure kafka topics in kafka 0.9 clusters?
>>
>>
>>
>> Thanks a lot.
>>
>>
>>
>>
>>
>> Regards,
>>
>> Raja.
>>
>
>