You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Cassa L <lc...@gmail.com> on 2015/08/28 20:00:18 UTC
SSL between Kafka and Spark Streaming API
Hi,
I was going through SSL setup of Kafka.
https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
However, I am also using Spark-Kafka streaming to read data from Kafka. Is
there a way to activate SSL for spark streaming API or not possible at all?
Thanks,
LCassa
Re: SSL between Kafka and Spark Streaming API
Posted by Gwen Shapira <gw...@confluent.io>.
SSL is also for 0.8.3... if you are on 0.8.2.1, the normal Spark
instructions will work for you.
On Fri, Aug 28, 2015 at 1:32 PM, Cassa L <lc...@gmail.com> wrote:
> Just to confirm, is this what you are mentioning about? Is there any
> example on how to set it? I believe it is for 0.8.3 version?
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers
>
>
> On Fri, Aug 28, 2015 at 12:52 PM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > You can configure PLAINTEXT listener as well with the broker and use that
> > port for spark.
> >
> > --
> > Harsha
> >
> >
> > On August 28, 2015 at 12:24:45 PM, Sourabh Chandak (
> sourabh3934@gmail.com)
> > wrote:
> >
> > Can we use the existing kafka spark streaming jar to connect to a kafka
> > server running in SSL mode?
> >
> > We are fine with non SSL consumer as our kafka cluster and spark cluster
> > are in the same network
> >
> >
> > Thanks,
> > Sourabh
> >
> > On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > I can't speak for the Spark Community, but checking their code,
> > DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
> >
> >
> >
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
> >
> >
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
> >
> > On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
> >
> > > Hi I am using below Spark jars with Direct Stream API.
> > > spark-streaming-kafka_2.10
> > >
> > > When I look at its pom.xml, Kafka libraries that its pulling in is
> > > <groupId>org.apache.kafka</groupId>
> > > <artifactId>kafka_${scala.binary.version}</artifactId>
> > > <version>0.8.2.1</version>
> > >
> > >
> > > I believe this DirectStream API uses SimpleConsumer API. Can someone
> from
> > > Spark community confirm too?
> > >
> > > Thanks,
> > > LCassa.
> > >
> > > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <
> > kafka@harsha.io>
> > > wrote:
> > >
> > > > SSL is supported for new producer and consumer api and old api
> (simple
> > > > consumer and high-level consumer) is not supported.
> > > > I think spark uses simple consumer? if so its not supported.
> > > >
> > > > Thanks,
> > > > Harsha
> > > >
> > > >
> > > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com)
> wrote:
> > > >
> > > > Hi,
> > > > I was going through SSL setup of Kafka.
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > > > However, I am also using Spark-Kafka streaming to read data from
> Kafka.
> > > Is
> > > > there a way to activate SSL for spark streaming API or not possible
> at
> > > > all?
> > > >
> > > > Thanks,
> > > > LCassa
> > > >
> > > >
> > >
> >
> >
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cassa L <lc...@gmail.com>.
Just to confirm, is this what you are mentioning about? Is there any
example on how to set it? I believe it is for 0.8.3 version?
https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers
On Fri, Aug 28, 2015 at 12:52 PM, Sriharsha Chintalapani <ka...@harsha.io>
wrote:
> You can configure PLAINTEXT listener as well with the broker and use that
> port for spark.
>
> --
> Harsha
>
>
> On August 28, 2015 at 12:24:45 PM, Sourabh Chandak (sourabh3934@gmail.com)
> wrote:
>
> Can we use the existing kafka spark streaming jar to connect to a kafka
> server running in SSL mode?
>
> We are fine with non SSL consumer as our kafka cluster and spark cluster
> are in the same network
>
>
> Thanks,
> Sourabh
>
> On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> I can't speak for the Spark Community, but checking their code,
> DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
>
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
>
> On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
>
> > Hi I am using below Spark jars with Direct Stream API.
> > spark-streaming-kafka_2.10
> >
> > When I look at its pom.xml, Kafka libraries that its pulling in is
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka_${scala.binary.version}</artifactId>
> > <version>0.8.2.1</version>
> >
> >
> > I believe this DirectStream API uses SimpleConsumer API. Can someone from
> > Spark community confirm too?
> >
> > Thanks,
> > LCassa.
> >
> > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <
> kafka@harsha.io>
> > wrote:
> >
> > > SSL is supported for new producer and consumer api and old api (simple
> > > consumer and high-level consumer) is not supported.
> > > I think spark uses simple consumer? if so its not supported.
> > >
> > > Thanks,
> > > Harsha
> > >
> > >
> > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> > >
> > > Hi,
> > > I was going through SSL setup of Kafka.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > > However, I am also using Spark-Kafka streaming to read data from Kafka.
> > Is
> > > there a way to activate SSL for spark streaming API or not possible at
> > > all?
> > >
> > > Thanks,
> > > LCassa
> > >
> > >
> >
>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cassa L <lc...@gmail.com>.
Just to confirm, is this what you are mentioning about? Is there any
example on how to set it? I believe it is for 0.8.3 version?
https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers
On Fri, Aug 28, 2015 at 12:52 PM, Sriharsha Chintalapani <ka...@harsha.io>
wrote:
> You can configure PLAINTEXT listener as well with the broker and use that
> port for spark.
>
> --
> Harsha
>
>
> On August 28, 2015 at 12:24:45 PM, Sourabh Chandak (sourabh3934@gmail.com)
> wrote:
>
> Can we use the existing kafka spark streaming jar to connect to a kafka
> server running in SSL mode?
>
> We are fine with non SSL consumer as our kafka cluster and spark cluster
> are in the same network
>
>
> Thanks,
> Sourabh
>
> On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> I can't speak for the Spark Community, but checking their code,
> DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
>
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
>
> On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
>
> > Hi I am using below Spark jars with Direct Stream API.
> > spark-streaming-kafka_2.10
> >
> > When I look at its pom.xml, Kafka libraries that its pulling in is
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka_${scala.binary.version}</artifactId>
> > <version>0.8.2.1</version>
> >
> >
> > I believe this DirectStream API uses SimpleConsumer API. Can someone from
> > Spark community confirm too?
> >
> > Thanks,
> > LCassa.
> >
> > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <
> kafka@harsha.io>
> > wrote:
> >
> > > SSL is supported for new producer and consumer api and old api (simple
> > > consumer and high-level consumer) is not supported.
> > > I think spark uses simple consumer? if so its not supported.
> > >
> > > Thanks,
> > > Harsha
> > >
> > >
> > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> > >
> > > Hi,
> > > I was going through SSL setup of Kafka.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > > However, I am also using Spark-Kafka streaming to read data from Kafka.
> > Is
> > > there a way to activate SSL for spark streaming API or not possible at
> > > all?
> > >
> > > Thanks,
> > > LCassa
> > >
> > >
> >
>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Sriharsha Chintalapani <ka...@harsha.io>.
You can configure PLAINTEXT listener as well with the broker and use that port for spark.
--
Harsha
On August 28, 2015 at 12:24:45 PM, Sourabh Chandak (sourabh3934@gmail.com) wrote:
Can we use the existing kafka spark streaming jar to connect to a kafka server running in SSL mode?
We are fine with non SSL consumer as our kafka cluster and spark cluster are in the same network
Thanks,
Sourabh
On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
I can't speak for the Spark Community, but checking their code,
DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
> Hi I am using below Spark jars with Direct Stream API.
> spark-streaming-kafka_2.10
>
> When I look at its pom.xml, Kafka libraries that its pulling in is
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_${scala.binary.version}</artifactId>
> <version>0.8.2.1</version>
>
>
> I believe this DirectStream API uses SimpleConsumer API. Can someone from
> Spark community confirm too?
>
> Thanks,
> LCassa.
>
> On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > SSL is supported for new producer and consumer api and old api (simple
> > consumer and high-level consumer) is not supported.
> > I think spark uses simple consumer? if so its not supported.
> >
> > Thanks,
> > Harsha
> >
> >
> > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> >
> > Hi,
> > I was going through SSL setup of Kafka.
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > However, I am also using Spark-Kafka streaming to read data from Kafka.
> Is
> > there a way to activate SSL for spark streaming API or not possible at
> > all?
> >
> > Thanks,
> > LCassa
> >
> >
>
Re: SSL between Kafka and Spark Streaming API
Posted by Sourabh Chandak <so...@gmail.com>.
Can we use the existing kafka spark streaming jar to connect to a kafka
server running in SSL mode?
We are fine with non SSL consumer as our kafka cluster and spark cluster
are in the same network
Thanks,
Sourabh
On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> I can't speak for the Spark Community, but checking their code,
> DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
>
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
>
> On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
>
> > Hi I am using below Spark jars with Direct Stream API.
> > spark-streaming-kafka_2.10
> >
> > When I look at its pom.xml, Kafka libraries that its pulling in is
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka_${scala.binary.version}</artifactId>
> > <version>0.8.2.1</version>
> >
> >
> > I believe this DirectStream API uses SimpleConsumer API. Can someone from
> > Spark community confirm too?
> >
> > Thanks,
> > LCassa.
> >
> > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <
> kafka@harsha.io>
> > wrote:
> >
> > > SSL is supported for new producer and consumer api and old api (simple
> > > consumer and high-level consumer) is not supported.
> > > I think spark uses simple consumer? if so its not supported.
> > >
> > > Thanks,
> > > Harsha
> > >
> > >
> > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> > >
> > > Hi,
> > > I was going through SSL setup of Kafka.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > > However, I am also using Spark-Kafka streaming to read data from Kafka.
> > Is
> > > there a way to activate SSL for spark streaming API or not possible at
> > > all?
> > >
> > > Thanks,
> > > LCassa
> > >
> > >
> >
>
Re: SSL between Kafka and Spark Streaming API
Posted by Sourabh Chandak <so...@gmail.com>.
Can we use the existing kafka spark streaming jar to connect to a kafka
server running in SSL mode?
We are fine with non SSL consumer as our kafka cluster and spark cluster
are in the same network
Thanks,
Sourabh
On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> I can't speak for the Spark Community, but checking their code,
> DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
>
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
>
> https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
>
> On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
>
> > Hi I am using below Spark jars with Direct Stream API.
> > spark-streaming-kafka_2.10
> >
> > When I look at its pom.xml, Kafka libraries that its pulling in is
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka_${scala.binary.version}</artifactId>
> > <version>0.8.2.1</version>
> >
> >
> > I believe this DirectStream API uses SimpleConsumer API. Can someone from
> > Spark community confirm too?
> >
> > Thanks,
> > LCassa.
> >
> > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <
> kafka@harsha.io>
> > wrote:
> >
> > > SSL is supported for new producer and consumer api and old api (simple
> > > consumer and high-level consumer) is not supported.
> > > I think spark uses simple consumer? if so its not supported.
> > >
> > > Thanks,
> > > Harsha
> > >
> > >
> > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> > >
> > > Hi,
> > > I was going through SSL setup of Kafka.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > > However, I am also using Spark-Kafka streaming to read data from Kafka.
> > Is
> > > there a way to activate SSL for spark streaming API or not possible at
> > > all?
> > >
> > > Thanks,
> > > LCassa
> > >
> > >
> >
>
Re: SSL between Kafka and Spark Streaming API
Posted by Gwen Shapira <gw...@confluent.io>.
I can't speak for the Spark Community, but checking their code,
DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lc...@gmail.com> wrote:
> Hi I am using below Spark jars with Direct Stream API.
> spark-streaming-kafka_2.10
>
> When I look at its pom.xml, Kafka libraries that its pulling in is
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_${scala.binary.version}</artifactId>
> <version>0.8.2.1</version>
>
>
> I believe this DirectStream API uses SimpleConsumer API. Can someone from
> Spark community confirm too?
>
> Thanks,
> LCassa.
>
> On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > SSL is supported for new producer and consumer api and old api (simple
> > consumer and high-level consumer) is not supported.
> > I think spark uses simple consumer? if so its not supported.
> >
> > Thanks,
> > Harsha
> >
> >
> > On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
> >
> > Hi,
> > I was going through SSL setup of Kafka.
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> > However, I am also using Spark-Kafka streaming to read data from Kafka.
> Is
> > there a way to activate SSL for spark streaming API or not possible at
> > all?
> >
> > Thanks,
> > LCassa
> >
> >
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cody Koeninger <co...@koeninger.org>.
Yeah, the direct api uses the simple consumer
On Fri, Aug 28, 2015 at 1:32 PM, Cassa L <lc...@gmail.com> wrote:
> Hi I am using below Spark jars with Direct Stream API.
> spark-streaming-kafka_2.10
>
> When I look at its pom.xml, Kafka libraries that its pulling in is
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_${scala.binary.version}</artifactId>
> <version>0.8.2.1</version>
>
>
> I believe this DirectStream API uses SimpleConsumer API. Can someone from
> Spark community confirm too?
>
> Thanks,
> LCassa.
>
> On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> SSL is supported for new producer and consumer api and old api (simple
>> consumer and high-level consumer) is not supported.
>> I think spark uses simple consumer? if so its not supported.
>>
>> Thanks,
>> Harsha
>>
>>
>> On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
>>
>> Hi,
>> I was going through SSL setup of Kafka.
>> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
>> However, I am also using Spark-Kafka streaming to read data from Kafka.
>> Is
>> there a way to activate SSL for spark streaming API or not possible at
>> all?
>>
>> Thanks,
>> LCassa
>>
>>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cody Koeninger <co...@koeninger.org>.
Yeah, the direct api uses the simple consumer
On Fri, Aug 28, 2015 at 1:32 PM, Cassa L <lc...@gmail.com> wrote:
> Hi I am using below Spark jars with Direct Stream API.
> spark-streaming-kafka_2.10
>
> When I look at its pom.xml, Kafka libraries that its pulling in is
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_${scala.binary.version}</artifactId>
> <version>0.8.2.1</version>
>
>
> I believe this DirectStream API uses SimpleConsumer API. Can someone from
> Spark community confirm too?
>
> Thanks,
> LCassa.
>
> On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> SSL is supported for new producer and consumer api and old api (simple
>> consumer and high-level consumer) is not supported.
>> I think spark uses simple consumer? if so its not supported.
>>
>> Thanks,
>> Harsha
>>
>>
>> On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
>>
>> Hi,
>> I was going through SSL setup of Kafka.
>> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
>> However, I am also using Spark-Kafka streaming to read data from Kafka.
>> Is
>> there a way to activate SSL for spark streaming API or not possible at
>> all?
>>
>> Thanks,
>> LCassa
>>
>>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cassa L <lc...@gmail.com>.
Hi I am using below Spark jars with Direct Stream API.
spark-streaming-kafka_2.10
When I look at its pom.xml, Kafka libraries that its pulling in is
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_${scala.binary.version}</artifactId>
<version>0.8.2.1</version>
I believe this DirectStream API uses SimpleConsumer API. Can someone from
Spark community confirm too?
Thanks,
LCassa.
On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
wrote:
> SSL is supported for new producer and consumer api and old api (simple
> consumer and high-level consumer) is not supported.
> I think spark uses simple consumer? if so its not supported.
>
> Thanks,
> Harsha
>
>
> On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
>
> Hi,
> I was going through SSL setup of Kafka.
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> However, I am also using Spark-Kafka streaming to read data from Kafka. Is
> there a way to activate SSL for spark streaming API or not possible at
> all?
>
> Thanks,
> LCassa
>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Cassa L <lc...@gmail.com>.
Hi I am using below Spark jars with Direct Stream API.
spark-streaming-kafka_2.10
When I look at its pom.xml, Kafka libraries that its pulling in is
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_${scala.binary.version}</artifactId>
<version>0.8.2.1</version>
I believe this DirectStream API uses SimpleConsumer API. Can someone from
Spark community confirm too?
Thanks,
LCassa.
On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io>
wrote:
> SSL is supported for new producer and consumer api and old api (simple
> consumer and high-level consumer) is not supported.
> I think spark uses simple consumer? if so its not supported.
>
> Thanks,
> Harsha
>
>
> On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
>
> Hi,
> I was going through SSL setup of Kafka.
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
> However, I am also using Spark-Kafka streaming to read data from Kafka. Is
> there a way to activate SSL for spark streaming API or not possible at
> all?
>
> Thanks,
> LCassa
>
>
Re: SSL between Kafka and Spark Streaming API
Posted by Sriharsha Chintalapani <ka...@harsha.io>.
SSL is supported for new producer and consumer api and old api (simple consumer and high-level consumer) is not supported.
I think spark uses simple consumer? if so its not supported.
Thanks,
Harsha
On August 28, 2015 at 11:00:30 AM, Cassa L (lcassa8@gmail.com) wrote:
Hi,
I was going through SSL setup of Kafka.
https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
However, I am also using Spark-Kafka streaming to read data from Kafka. Is
there a way to activate SSL for spark streaming API or not possible at all?
Thanks,
LCassa