[jira] [Commented] (KAFKA-13859) SCRAM authentication issues with kafka-clients 3.0.1

["Oliver Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530796#comment-17530796 ] 

Oliver Payne commented on KAFKA-13859:
--------------------------------------

Sorry for the late response. I see that this has already been marked resolved, but wanted to answer the questions I left hanging here.
[~dengziming] The following exception is coming from the client logs:

 
{code:java}
org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.
    at org.springframework.kafka.core.KafkaTemplate.lambda$buildCallback$6(KafkaTemplate.java:690) ~[spring-kafka-2.8.5.jar:2.8.5]
    at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer$1.onCompletion(DefaultKafkaProducerFactory.java:1001) ~[spring-kafka-2.8.5.jar:2.8.5]
    at org.apache.kafka.clients.producer.KafkaProducer$InterceptorCallback.onCompletion(KafkaProducer.java:1350) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:273) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.ProducerBatch.abort(ProducerBatch.java:161) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:773) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.maybeAbortBatches(Sender.java:498) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:307) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:243) ~[kafka-clients-3.0.1.jar:na]
    at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed. {code}
Our broker version is 2.6.2

 

Here are our producer configs:
{code:java}
"security.protocol" -> "SASL_SSL""value.serializer" -> "org.apache.kafka.common.serialization.StringSerializer""sasl.mechanism" -> "SCRAM-SHA-512""sasl.jaas.config" -> "org.apache.kafka.common.security.scram.ScramLoginModule required username="redacted" password="redacted";""bootstrap.servers" -> "server-name-redacted1:9096, server-name-redacted2:9096, server-name-redacted3:9096""key.serializer" -> "org.apache.kafka.common.serialization.StringSerializer""ssl.endpoint.identification.algorithm" -> "https" {code}
 

 

I also added the enable.idempotence = false per your recommendation, and it seemed to resolve the issue. Thanks for the suggestion

 

> SCRAM authentication issues with kafka-clients 3.0.1
> ----------------------------------------------------
>
>                 Key: KAFKA-13859
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13859
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 3.0.1
>            Reporter: Oliver Payne
>            Assignee: dengziming
>            Priority: Major
>
> When attempting to produce records to Kafka using a client configured with SCRAM authentication, the authentication is being rejected, and the following exception is thrown:
> {{org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.}}
> I am seeing this happen with a Springboot service that was recently upgraded to 2.6.5. After looking into this, I learned that Springboot moved to kafka-clients 3.0.1 from 3.0.0 in that version. And sure enough, downgrading to kafka-clients resolved the issue, with no changes made to the configs.
> I have also attempted to connect to a separate server with kafka-clients 3.0.1, using plaintext authentication. That works fine. So the issue appears to be with SCRAM authentication.
> I will note that I am attempting to connect to an AWS MSK instance. We use SCRAM-SHA-512 as our sasl mechanism, using the basic {{ScramLoginModule.}} 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)