Re: PySpark Direct Streaming : SASL Security Compatibility Issue

[Gabor Somogyi <ga...@gmail.com>]

Hi Muthuraman,

Previous to 0.9, Kafka had no built-in security features but DStreams
support only kafka 0.8.
Suggest to use structured streaming where 0.10+ (2.0.0 in spark 2.4)
support is available.

BR,
G


On Wed, Nov 28, 2018 at 4:15 AM Ramaswamy, Muthuraman <
Muthuraman.Ramaswamy@viasat.com> wrote:

> Hi All,
>
>
>
> I am using PySpark Direct Streaming to connect to a remote secured Kafka
> broker and is secured with Kerberos Authentication. The
> KafkaUtils.createDirectStream python call gives me the following error:
>
>
>
> 18/11/27 18:20:05 WARN VerifiableProperties: Property sasl.mechanism is
> not valid
>
> 18/11/27 18:20:05 WARN VerifiableProperties: Property security.protocol is
> not valid
>
>
>
> Does spark-streaming-kafka-0-8_2.11 library support SASL_PLAINTEXT
> security protocol? Is there any compatibility issue with 0.8 version of the
> package?
>
>
>
> Appreciate your help to resolve the security issue.
>
>
>
> Thank you,
>
>
>
> *Muthu R*
>
>
>