[jira] [Assigned] (KAFKA-9962) Admin client throws UnsupportedVersion exception when talking to old broker

["Jose Armando Garcia Sancio (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jose Armando Garcia Sancio reassigned KAFKA-9962:
-------------------------------------------------

    Assignee:     (was: Jose Armando Garcia Sancio)

> Admin client throws UnsupportedVersion exception when talking to old broker
> ---------------------------------------------------------------------------
>
>                 Key: KAFKA-9962
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9962
>             Project: Kafka
>          Issue Type: Task
>          Components: clients
>    Affects Versions: 2.3.1, 2.5.0, 2.4.1
>            Reporter: Jose Armando Garcia Sancio
>            Priority: Critical
>
> Users are getting this error when using a client version 2.5 against a 1.1.0 cluster/broker.
> {code:java}
> [2020-04-28 01:09:10,663] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:63)
> io.confluent.ksql.util.KsqlServerException: Could not get Kafka authorized operations!
>         at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:51)
>         at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.create(KsqlAuthorizationValidatorFactory.java:52)
>         at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:639)
>         at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:567)
>         at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:100)
>         at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:59)
> Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
>         at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
>         at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
>         at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
>         at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
>         at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:49)
>         ... 5 more
> Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
> {code}
> Looking at KIP-430, it mentions that the client is supposed to handle this case:
>  # Existing clients using older versions will not request authorized operations in Describe requests since the default is to disable this feature. This keeps older clients compatible with newer brokers.
>  # Newer clients connecting to older brokers will use the older protocol version and hence will not request authorized operations.
>  # When the AdminClient is talking to a broker which does not support KIP-430, it will fill in either null or UnsupportedVersionException for the returned ACL operations fields in objects. For example, `ConsumerGroupDescription#authorizedOperations` will be null if the broker did not supply this information. DescribeClusterResult#authorizedOperations will throw an `UnsupportedVersionException` if the broker did not supply this information.
>  # When new operations are added, newer brokers may return operations that are not known to older clients. AdminClient will ignore any bit that is set in authorized_operations that is not known to the client. The Set<AclOperation> created by the client from the bits returned by the broker will only include operations that the client client knows about.
> I assume that this deployment environment falls under case 2, we have this in the serialization code:
> {code:java}
>         if (_version >= 8) {
>             _writable.writeByte(includeClusterAuthorizedOperations ? (byte) 1 : (byte) 0);
>         } else {
>             if (includeClusterAuthorizedOperations) {
>                 throw new UnsupportedVersionException("Attempted to write a non-default includeClusterAuthorizedOperations at version " + _version);
>             }
>         }
> {code}
> I also looks like we blindly set the version independent of the Broker’s supported version:
> {code:java}
>             MetadataRequest.Builder createRequest(int timeoutMs) {
>                 // Since this only requests node information, it's safe to pass true for allowAutoTopicCreation (and it
>                 // simplifies communication with older brokers)
>                 return new MetadataRequest.Builder(new MetadataRequestData()
>                     .setTopics(Collections.emptyList())
>                     .setAllowAutoTopicCreation(true)
>                     .setIncludeClusterAuthorizedOperations(options.includeAuthorizedOperations()));
>             }
> {code}
> To implement 2. we need to make these properties ignorable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)