You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "hudeqi (Jira)" <ji...@apache.org> on 2023/06/25 12:08:00 UTC

[jira] [Updated] (KAFKA-15119) Support incremental synchronization of topicAcl in MirrorSourceConnector

     [ https://issues.apache.org/jira/browse/KAFKA-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

hudeqi updated KAFKA-15119:
---------------------------
    Description: 
In the “syncTopicAcls” thread of MirrorSourceConnector, full amount of "TopicAclBindings" related to the replicated topics of the source cluster will be regularly listed, and then fully updated to the target cluster. Therefore, a large number of repeated "TopicAclBindings" will be repeatedly sent  by calling "targetAdminClient". This action is redundant. In addition, if too many "TopicAclBindings" are updated at one time, it may also take a long time for the target cluster to handle processing the "createAcls" request, which will affect the accumulation of the request queue of the target cluster and further affect the processing delay of other type requests.

Therefore, "TopicAclBinding" can be like the variable “knownConsumerGroups” in MirrorCheckpointConnector, and only update the incremental added "TopicAclBinding" every time, which can solve the above-mentioned problems.

> Support incremental synchronization of topicAcl in MirrorSourceConnector
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-15119
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15119
>             Project: Kafka
>          Issue Type: Improvement
>          Components: mirrormaker
>    Affects Versions: 3.4.1
>            Reporter: hudeqi
>            Assignee: hudeqi
>            Priority: Major
>
> In the “syncTopicAcls” thread of MirrorSourceConnector, full amount of "TopicAclBindings" related to the replicated topics of the source cluster will be regularly listed, and then fully updated to the target cluster. Therefore, a large number of repeated "TopicAclBindings" will be repeatedly sent  by calling "targetAdminClient". This action is redundant. In addition, if too many "TopicAclBindings" are updated at one time, it may also take a long time for the target cluster to handle processing the "createAcls" request, which will affect the accumulation of the request queue of the target cluster and further affect the processing delay of other type requests.
> Therefore, "TopicAclBinding" can be like the variable “knownConsumerGroups” in MirrorCheckpointConnector, and only update the incremental added "TopicAclBinding" every time, which can solve the above-mentioned problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)