[jira] [Created] (SPARK-30495) How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster

["act_coder (Jira)" <ji...@apache.org>]

act_coder created SPARK-30495:
---------------------------------

             Summary: How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster
                 Key: SPARK-30495
                 URL: https://issues.apache.org/jira/browse/SPARK-30495
             Project: Spark
          Issue Type: Question
          Components: Structured Streaming
    Affects Versions: 3.0.0
            Reporter: act_coder


Trying to read data from a secured Kafka cluster using spark structured
streaming. Also, using the below library to read the data -
+*"spark-sql-kafka-0-10_2.12":"3.0.0-preview"*+ since it has the feature to
specify our custom group id (instead of spark setting its own custom group
id)

+*Dependency used in code:*+

        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql-kafka-0-10_2.12</artifactId>
        <version>3.0.0-preview</version>

Getting the below error - even after specifying the required JAAS
configuration in spark options.

Caused by: java.lang.IllegalArgumentException: requirement failed:
Delegation token must exist for this connector. at
scala.Predef$.require(Predef.scala:281) at

org.apache.spark.kafka010.KafkaTokenUtil$.isConnectorUsingCurrentToken(KafkaTokenUtil.scala:299)
at
org.apache.spark.sql.kafka010.KafkaDataConsumer.getOrRetrieveConsumer(KafkaDataConsumer.scala:533)
at
org.apache.spark.sql.kafka010.KafkaDataConsumer.$anonfun$get$1(KafkaDataConsumer.scala:275)


Following document specifies that we can disable the feature of obtaining
delegation token -
[https://spark.apache.org/docs/3.0.0-preview/structured-streaming-kafka-integration.html]

Tried setting this property *spark.security.credentials.kafka.enabled to*
*false in spark config,* but it is still failing with the same error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org