You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "JimmyWang6 (via GitHub)" <gi...@apache.org> on 2025/10/28 11:18:53 UTC

[PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

JimmyWang6 opened a new pull request, #20783:
URL: https://github.com/apache/kafka/pull/20783

   see: [KAFKA-19817](https://issues.apache.org/jira/browse/KAFKA-19817)
   This PR intended to move `DynamicTopicClusterQuotaPublisher`  to the metadata module .
   To prevent circular dependencies, several associated classes have been migrated as well,including `ClientSensors` , `ClientQuotaMetadataManager`, `ThrottleCallback` and `ThrottledChannel`.
   Additionally, a new interface `QuotaManagersProvider` has been introduced to avoid the need to move class `QuotaManagers` in  `QuotaFactory`, as doing so would have necessitated significantly more extensive changes.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3694460569

   @JimmyWang6 could you check the build error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3765528054

   @chia7712 Thanks for the review! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2685253565


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   My bad! I almost left ClientSensors out in the cold.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2686047113


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener
+    ) {
+        this.clusterId = clusterId;
+        this.nodeId = nodeId;
+        this.faultHandler = faultHandler;
+        this.nodeType = nodeType;
+        this.clientQuotaCallbackPlugin = clientQuotaCallbackPlugin;
+        this.quotaConfigChangeListener = quotaConfigChangeListener;
+    }
+
+    @Override
+    public String name() {
+        return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
+    }
+
+    @Override
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
+        try {
+            clientQuotaCallbackPlugin.ifPresent(plugin -> {
+                if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
+                    Cluster cluster = MetadataCache.toCluster(clusterId, newImage);
+                    if (plugin.get().updateClusterMetadata(cluster)) {
+                        quotaConfigChangeListener.onChange();
+                    }
+                }
+            });
+        } catch (Exception e) {

Review Comment:
   Could we use `Throwable` for consistency?



##########
server/src/main/java/org/apache/kafka/network/Session.java:
##########
@@ -21,14 +21,8 @@
 
 import java.net.InetAddress;
 
-public class Session {
-    public final KafkaPrincipal principal;
-    public final InetAddress clientAddress;
-    public final String sanitizedUser;
-
-    public Session(KafkaPrincipal principal, InetAddress clientAddress) {
-        this.principal = principal;
-        this.clientAddress = clientAddress;
-        this.sanitizedUser = Sanitizer.sanitize(principal.getName());
+public record Session(KafkaPrincipal principal, InetAddress clientAddress) {
+    public String sanitizedUser() {
+        return Sanitizer.sanitize(principal.getName());

Review Comment:
   On second thought, this might introduces a performance regression. Would you mind initializing it in the constructor 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #20783:
URL: https://github.com/apache/kafka/pull/20783


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2660861330


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   Makes sense, thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2478240866


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final Integer nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        Integer nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers
+    ) {
+        this.clusterId = clusterId;
+        this.nodeId = nodeId;
+        this.faultHandler = faultHandler;
+        this.nodeType = nodeType;
+        this.quotaManagersProvider = quotaManagers;
+    }
+
+    @Override
+    public String name() {
+        return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
+    }
+
+    @Override
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
+        onMetadataUpdate(delta, newImage);
+    }
+
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage) {

Review Comment:
   could you merge those two `onMetadataUpdate`?



##########
server-common/src/main/java/org/apache/kafka/server/network/Session.java:
##########
@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.kafka.network;
+package org.apache.kafka.server.network;
 
 import org.apache.kafka.common.security.auth.KafkaPrincipal;
 import org.apache.kafka.common.utils.Sanitizer;

Review Comment:
   Could you rewrite `Session` by `record`?



##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final Integer nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        Integer nodeId,

Review Comment:
   `int`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2659724602


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   It appears the benefit of this change is that we don't need to move `ClientQuotaManager` from the server module to the server-common module, right?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2689206763


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener
+    ) {
+        this.clusterId = clusterId;
+        this.nodeId = nodeId;
+        this.faultHandler = faultHandler;
+        this.nodeType = nodeType;
+        this.clientQuotaCallbackPlugin = clientQuotaCallbackPlugin;
+        this.quotaConfigChangeListener = quotaConfigChangeListener;
+    }
+
+    @Override
+    public String name() {
+        return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
+    }
+
+    @Override
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
+        try {
+            clientQuotaCallbackPlugin.ifPresent(plugin -> {
+                if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
+                    Cluster cluster = MetadataCache.toCluster(clusterId, newImage);
+                    if (plugin.get().updateClusterMetadata(cluster)) {
+                        quotaConfigChangeListener.onChange();
+                    }
+                }
+            });
+        } catch (Exception e) {

Review Comment:
   Done, thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3461306378

   ```
   > Task :server-common:checkstyleMain
   [ant:checkstyle] [ERROR] /home/chia7712/project/kafka/server-common/src/main/java/org/apache/kafka/server/quota/ClientQuotaManager.java:34:1: Disallowed import - org.apache.kafka.server.network.Session. [ImportControl]
   [ant:checkstyle] [ERROR] /home/chia7712/project/kafka/server-common/src/main/java/org/apache/kafka/server/quota/ClientQuotaManager.java:35:1: Disallowed import - org.apache.kafka.server.util.ShutdownableThread. [ImportControl]
   ```
   
   @JimmyWang6 please fix the build error


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3465984599

   @chia7712 Thanks for the comment, just fixed the build error, please take another look when you're free :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2659072619


##########
core/src/main/java/kafka/server/QuotaFactory.java:
##########
@@ -67,6 +68,17 @@ public void shutdown() {
             controllerMutation.shutdown();
             clientQuotaCallbackPlugin.ifPresent(plugin -> Utils.closeQuietly(plugin, "client quota callback plugin"));
         }
+
+        public void updateQuotaMetricsConfigs() {
+            fetch.updateQuotaMetricConfigs();
+            produce.updateQuotaMetricConfigs();
+            request.updateQuotaMetricConfigs();
+            controllerMutation.updateQuotaMetricConfigs();
+        }
+
+        public QuotaConfigChangeListener quotaConfigChangeListener() {

Review Comment:
   Done~



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2656498762


##########
core/src/main/java/kafka/server/QuotaFactory.java:
##########
@@ -67,6 +68,17 @@ public void shutdown() {
             controllerMutation.shutdown();
             clientQuotaCallbackPlugin.ifPresent(plugin -> Utils.closeQuietly(plugin, "client quota callback plugin"));
         }
+
+        public void updateQuotaMetricsConfigs() {
+            fetch.updateQuotaMetricConfigs();
+            produce.updateQuotaMetricConfigs();
+            request.updateQuotaMetricConfigs();
+            controllerMutation.updateQuotaMetricConfigs();
+        }
+
+        public QuotaConfigChangeListener quotaConfigChangeListener() {

Review Comment:
   ```java
           public QuotaConfigChangeListener quotaConfigChangeListener() {
               return () -> {
                   fetch.updateQuotaMetricConfigs();
                   produce.updateQuotaMetricConfigs();
                   request.updateQuotaMetricConfigs();
                   controllerMutation.updateQuotaMetricConfigs();
               };
           }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2480086265


##########
server-common/src/main/java/org/apache/kafka/server/network/Session.java:
##########
@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.kafka.network;
+package org.apache.kafka.server.network;
 
 import org.apache.kafka.common.security.auth.KafkaPrincipal;
 import org.apache.kafka.common.utils.Sanitizer;

Review Comment:
   Sure, done~



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2480744211


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers

Review Comment:
   Hi @chia7712 , 
   I just tried and found that `clientQuotaCallbackPlugin` also belongs to class `ClientQuotaManager`, so if we are going to de-coupe those components, the code will be shown like this:
   ```java
   public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
       private final FaultHandler faultHandler;
       private final String nodeType;
       private final Runnable updateQuotaMetricConfigs;
   
       public DynamicTopicClusterQuotaPublisher(
           FaultHandler faultHandler,
           String nodeType,
           Runnable updateQuotaMetricConfigs
       ) {
           this.faultHandler = faultHandler;
           this.nodeType = nodeType;
           this.updateQuotaMetricConfigs = updateQuotaMetricConfigs;
       }
   
       @Override
       public String name() {
           return "DynamicTopicClusterQuotaPublisher " + nodeType;
       }
   
       @Override
       public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
           try {
               if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
                   updateQuotaMetricConfigs.run();
               }
           } catch (Exception e) {
               String deltaName = "MetadataDelta up to " + newImage.highestOffsetAndEpoch().offset();
               faultHandler.handleFault("Uncaught exception while publishing dynamic topic or cluster changes from " + deltaName, e);
           }
       }
   }
   ``` 
   And the logic of the original `DynamicTopicClusterQuotaPublisher` will still remain in the new class `updateQuotaMetricConfigs` somewhere within the core module. However, such a modification might deviate from the original intent of this issue. What's your perspective on this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2689212142


##########
server/src/main/java/org/apache/kafka/network/Session.java:
##########
@@ -21,14 +21,8 @@
 
 import java.net.InetAddress;
 
-public class Session {
-    public final KafkaPrincipal principal;
-    public final InetAddress clientAddress;
-    public final String sanitizedUser;
-
-    public Session(KafkaPrincipal principal, InetAddress clientAddress) {
-        this.principal = principal;
-        this.clientAddress = clientAddress;
-        this.sanitizedUser = Sanitizer.sanitize(principal.getName());
+public record Session(KafkaPrincipal principal, InetAddress clientAddress) {
+    public String sanitizedUser() {
+        return Sanitizer.sanitize(principal.getName());

Review Comment:
   That's a great point, I've updated the code and thanks for catching that!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2654686564


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final Runnable updateQuotaMetricConfigs;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        Runnable updateQuotaMetricConfigs

Review Comment:
   Thanks for the review. I've refactored this logic into a interface to improve readability and semantics. Please take a look at the latest commit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2660092590


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   Yes, exactly. Other approaches would trigger a dependency chain requiring QuotaManager and ClientQuotaManager to be moved, which I want to avoid in this PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2480086011


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final Integer nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        Integer nodeId,

Review Comment:
   Done.



##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final Integer nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        Integer nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers
+    ) {
+        this.clusterId = clusterId;
+        this.nodeId = nodeId;
+        this.faultHandler = faultHandler;
+        this.nodeType = nodeType;
+        this.quotaManagersProvider = quotaManagers;
+    }
+
+    @Override
+    public String name() {
+        return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
+    }
+
+    @Override
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
+        onMetadataUpdate(delta, newImage);
+    }
+
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage) {

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2480518934


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers
+    ) {
+        this.clusterId = clusterId;
+        this.nodeId = nodeId;
+        this.faultHandler = faultHandler;
+        this.nodeType = nodeType;
+        this.quotaManagersProvider = quotaManagers;
+    }
+
+    @Override
+    public String name() {
+        return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
+    }
+
+    @Override
+    public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
+        try {
+            quotaManagersProvider.clientQuotaCallbackPlugin().ifPresent(plugin -> {
+                if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
+                    Cluster cluster = MetadataCache.toCluster(clusterId, newImage);
+                    if (plugin.get().updateClusterMetadata(cluster)) {

Review Comment:
   The necessity of this method is being discussed in KIP-1200. While I hope the vote succeeds, there is no harm in migrating the code though



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2480571901


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers

Review Comment:
   Could you consider de-coupling those components by using lambda function? for example:
   ```java
       public DynamicTopicClusterQuotaPublisher(
           String clusterId,
           int nodeId,
           FaultHandler faultHandler,
           String nodeType,
           Plugin<ClientQuotaCallback> clientQuotaCallbackPlugin,
           Runnable updateQuotaMetricConfigs
       ) {
           this.clusterId = clusterId;
           this.nodeId = nodeId;
           this.faultHandler = faultHandler;
           this.nodeType = nodeType;
           this.clientQuotaCallbackPlugin = clientQuotaCallbackPlugin;
           this.updateQuotaMetricConfigs = updateQuotaMetricConfigs;
       }
   
       @Override
       public String name() {
           return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
       }
   
       @Override
       public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
           try {
               if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
                   Cluster cluster = MetadataCache.toCluster(clusterId, newImage);
                   if (clientQuotaCallbackPlugin.get().updateClusterMetadata(cluster)) {
                       updateQuotaMetricConfigs.run();
                   }
               }
           } catch (Exception e) {
               String deltaName = "MetadataDelta up to " + newImage.highestOffsetAndEpoch().offset();
               faultHandler.handleFault("Uncaught exception while publishing dynamic topic or cluster changes from " + deltaName, e);
           }
       }
   ```
   1. we could create `DynamicTopicClusterQuotaPublisher` only if the `clientQuotaCallbackPlugin` is existent
   2. we wrap all callbacks in a single runnable object to avoid move many classes into `server-common` module
   
   @JimmyWang6 WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3698611041

   Hi @chia7712 ,
   I have fixed the errors, please take a look. :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2690407591


##########
checkstyle/import-control-server.xml:
##########
@@ -91,9 +91,11 @@
     <subpackage name="metrics">
       <allow class="org.apache.kafka.server.authorizer.AuthorizableRequestContext" />
       <allow class="org.apache.kafka.controller.QuorumFeatures" />
-      <allow class="org.apache.kafka.controller.metrics.ControllerMetadataMetrics" />
       <allow pkg="org.apache.kafka.server.telemetry" />
     </subpackage>
+    <subpackage name="quota">

Review Comment:
   we can say goodbye to this rule, right? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2683042113


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   Don't leave poor `ClientSensors` behind



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "JimmyWang6 (via GitHub)" <gi...@apache.org>.
JimmyWang6 commented on PR #20783:
URL: https://github.com/apache/kafka/pull/20783#issuecomment-3471205645

   @chia7712 Thanks for your comments. All of them have been addressed, and please take another look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2481823744


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final QuotaManagersProvider quotaManagersProvider;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        QuotaManagersProvider quotaManagers

Review Comment:
   yes but `DynamicTopicClusterQuotaPublisher` can generate the  `Cluster`  and check the result of `updateClusterMetadata`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2653429357


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final Runnable updateQuotaMetricConfigs;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        Runnable updateQuotaMetricConfigs

Review Comment:
   Could you add comments explaining why we use `Runnable` here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2679491169


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   `ClientQuotaManagerConfig`, `ClientSensors`, `Session`, `ThrottleCallback`, and `ThrottledChannel` - I believe their home is the server module, right?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20783:
URL: https://github.com/apache/kafka/pull/20783#discussion_r2660487284


##########
metadata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.publisher;
+
+import org.apache.kafka.common.Cluster;
+import org.apache.kafka.common.internals.Plugin;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.loader.LoaderManifest;
+import org.apache.kafka.image.publisher.MetadataPublisher;
+import org.apache.kafka.metadata.MetadataCache;
+import org.apache.kafka.server.fault.FaultHandler;
+import org.apache.kafka.server.quota.ClientQuotaCallback;
+
+import java.util.Optional;
+
+public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
+    private final String clusterId;
+    private final int nodeId;
+    private final FaultHandler faultHandler;
+    private final String nodeType;
+    private final Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin;
+    private final QuotaConfigChangeListener quotaConfigChangeListener;
+
+    public DynamicTopicClusterQuotaPublisher(
+        String clusterId,
+        int nodeId,
+        FaultHandler faultHandler,
+        String nodeType,
+        Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin,
+        QuotaConfigChangeListener quotaConfigChangeListener

Review Comment:
   It looks like `ClientQuotaManager` is still being moved. We should revert that, right? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org