[jira] [Created] (KAFKA-14569) Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft

["Yash Mayya (Jira)" <ji...@apache.org>]

Yash Mayya created KAFKA-14569:
----------------------------------

             Summary: Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft
                 Key: KAFKA-14569
                 URL: https://issues.apache.org/jira/browse/KAFKA-14569
             Project: Kafka
          Issue Type: Task
          Components: KafkaConnect
            Reporter: Yash Mayya
            Assignee: Yash Mayya


ZooKeeper mode is going to be deprecated in Apache Kafka 4.0. Connect currently uses an [EmbeddedKafkaCluster|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L95] for integration tests that depends on an [EmbeddedZookeeper.|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L147] This should be migrated to remove the ZooKeeper dependency, instead working in the KRaft mode (probably with co-located brokers and controllers). We could potentially leverage the existing test kit for KRaft clusters ([https://github.com/apache/kafka/tree/b8ab09820cd96290176afd24cf7b03e7cda7f783/core/src/test/java/kafka/testkit)] which handles a bunch of stuff including the listeners configuration setup, formatting the metadata log directory, allowing usage of non-static random ports for `controller.quorum.voters`, initialization of the shared server(s), broker(s), and controller(s) etc.

 

One more thing to note is that some Connect integration tests currently use the `kafka.security.authorizer.AclAuthorizer` which requires ZooKeeper. These tests should be migrated to use the new authorizer from [KIP-801|https://cwiki.apache.org/confluence/display/KAFKA/KIP-801%3A+Implement+an+Authorizer+that+stores+metadata+in+__cluster_metadata] if we want to completely eliminate the dependency on ZooKeeper.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)