[jira] [Commented] (KAFKA-14569) Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft

["Ismael Juma (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-14569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654391#comment-17654391 ] 

Ismael Juma commented on KAFKA-14569:
-------------------------------------

Thanks for the JIRA. +1 for migrating the Connect integration tests to use kraft mode instead of zk mode.

> Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft
> ----------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-14569
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14569
>             Project: Kafka
>          Issue Type: Task
>          Components: KafkaConnect
>            Reporter: Yash Mayya
>            Assignee: Yash Mayya
>            Priority: Minor
>
> ZooKeeper mode is going to be deprecated in Apache Kafka 4.0. Connect currently uses an [EmbeddedKafkaCluster|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L95] (that depends on an [EmbeddedZookeeper)|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L147] for integration tests with the [EmbeddedConnectCluster|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedConnectCluster.java#L70]. This should be migrated to remove the ZooKeeper dependency, instead working in the KRaft mode (probably with co-located brokers and controllers). We could potentially leverage the [existing test kit for KRaft clusters|https://github.com/apache/kafka/tree/b8ab09820cd96290176afd24cf7b03e7cda7f783/core/src/test/java/kafka/testkit] which handles a bunch of stuff including the listeners configuration setup, formatting the metadata log directory, allowing usage of non-static random ports for `controller.quorum.voters`, initialization of the shared server(s), broker(s), and controller(s) etc.
>  
> One more thing to note is that some Connect integration tests currently use the `kafka.security.authorizer.AclAuthorizer` which requires ZooKeeper. These tests should be migrated to use the new authorizer from [KIP-801|https://cwiki.apache.org/confluence/display/KAFKA/KIP-801%3A+Implement+an+Authorizer+that+stores+metadata+in+__cluster_metadata] if we want to completely eliminate the dependency on ZooKeeper.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)