You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by "Ramaswamy, Muthuraman" <Mu...@viasat.com> on 2018/10/10 01:31:12 UTC

PySpark Streaming : Accessing the Remote Secured Kafka

All,

Currently, I am using PySpark Streaming (Classic Regular DStream Style and not Structured Streaming). Now, our remote Kafka is secured with Kerberos.

To enable PySpark Streaming to access the secured Kafka, what steps I should perform? Can I pass the principal/keytab and jaas config in the Spark Submit command? What are my options? Any pointers/links to access the secured Kafka broker from PySpark Streaming will be helpful.

Appreciate your time.

Thank you,

~Muthu