Custom auth broken in confluent-5.3/kafka-2.3 ?

[Joris Peeters <jo...@gmail.com>]

Hello,

We use custom Kafka authentication and authorisation, in a manner very
similar to https://github.com/navikt/kafka-plain-saslserver-2-ad, i.e. by
providing an implementation of
org.apache.kafka.common.security.auth.AuthenticateCallBackHandler and
kafka.security.auth.Authorizer - for plain user/pass. We inject the jar
into the cp-kafka docker images and activate it through environment
variables.

This works fine in versions up to cp-kafka:5.2.2 (corresponding to Kafka
2.2, I believe), but doesn't work in cp-kafka:5.3.1 (Kafka 2.3).

The logs do suggest that there is a problem, i.e.

[2019-11-25 14:00:03,854] DEBUG connections.max.reauth.ms for
mechanism=PLAIN: 0
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,854] DEBUG Set SASL server state to
HANDSHAKE_OR_VERSIONS_REQUEST during authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,854] DEBUG Handling Kafka request API_VERSIONS during
authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,854] DEBUG Set SASL server state to HANDSHAKE_REQUEST
during authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,855] DEBUG Handling Kafka request SASL_HANDSHAKE
during authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,855] DEBUG Using SASL mechanism 'PLAIN' provided by
client
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,856] DEBUG Set SASL server state to AUTHENTICATE
during authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,857] DEBUG Set SASL server state to FAILED during
authentication
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2019-11-25 14:00:03,857] INFO [SocketServer brokerId=1003] Failed
authentication with /xx.xx.xx.xx (Authentication failed: credentials for
user could not be verified) (org.apache.kafka.common.network.Selector)

Is anyone experiencing something similar? Is anyone using custom auth in
Kafka 2.3 without issues?

Happy to open a JIRA issue. The release notes for Kafka 2.3 don't
immediately reveal a standout suspect to me.

Thanks,
-Joris.