Unsubscribe

[serkan taş <se...@hotmail.com>]

Android için Outlook<https://aka.ms/ghei36> uygulamasini edinin

________________________________
From: Darshan Pandya <da...@gmail.com>
Sent: Saturday, October 14, 2017 11:55:06 AM
To: user
Subject: Spark Structured Streaming not connecting to Kafka using kerberos

Hello,

I'm using Spark 2.1.0 on CDH 5.8 with kafka 0.10.0.1 + kerberos

I am unable to connect to the kafka broker with the following message


17/10/14 14:29:10 WARN clients.NetworkClient: Bootstrap broker 10.197.19.25:9092<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F10.197.19.25%3A9092&data=02%7C01%7Cserkan_tas%40hotmail.com%7Cef009a347c5545fd5c1f08d512e15442%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636435681337925365&sdata=qGSmwm%2FcXZFRJhTn1ASW1U9Q4xN2zLy2joiAKd9ou64%3D&reserved=0> disconnected

and is unable to consume any messages.

And am using it as follows


jaas.conf

KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="./gandalf.keytab"
storeKey=true
useTicketCache=false
serviceName="kafka"
principal="gandalf@DOMAIN.COM<ma...@DOMAIN.COM>";
};

$SPARK_HOME/bin/spark-submit \
--master yarn \
--files jaas.conf,gandalf.keytab \
--driver-java-options "-Djava.security.auth.login.config=./jaas.conf -Dhdp.version=2.4.2.0-258" \
--conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./jaas.conf" \
--class com.example.ClassName uber-jar-with-deps-and-hive-site.jar

Thanks in advance.


--
Sincerely,
Darshan