You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "unknowntpo (via GitHub)" <gi...@apache.org> on 2026/03/19 10:13:01 UTC

[PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

unknowntpo opened a new pull request, #21818:
URL: https://github.com/apache/kafka/pull/21818

   This PR removes `CollectionUtils` entirely (KAFKA-20297) by replacing all usages with alternatives.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on PR #21818:
URL: https://github.com/apache/kafka/pull/21818#issuecomment-4101620684

   > @unknowntpo would you mind rebasing code?
   
   Ok, rebased.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r3006015856


##########
clients/src/main/java/org/apache/kafka/clients/consumer/StickyAssignor.java:
##########
@@ -230,7 +230,9 @@ protected MemberData memberData(Subscription subscription) {
     static ByteBuffer serializeTopicPartitionAssignment(MemberData memberData) {
         Struct struct = new Struct(STICKY_ASSIGNOR_USER_DATA_V1);
         List<Struct> topicAssignments = new ArrayList<>();
-        for (Map.Entry<String, List<Integer>> topicEntry : CollectionUtils.groupPartitionsByTopic(memberData.partitions).entrySet()) {
+        Map<String, List<Integer>> partitionsByTopic = memberData.partitions.stream()

Review Comment:
   ok fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2991980229


##########
clients/src/main/java/org/apache/kafka/clients/consumer/StickyAssignor.java:
##########
@@ -272,4 +274,9 @@ private static MemberData deserializeTopicPartitionAssignment(ByteBuffer buffer)
         Optional<Integer> generation = struct.hasField(GENERATION_KEY_NAME) ? Optional.of(struct.getInt(GENERATION_KEY_NAME)) : Optional.empty();
         return new MemberData(partitions, generation);
     }
+
+    private static Map<String, List<Integer>> groupPartitionsByTopic(Collection<TopicPartition> partitions) {

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #21818:
URL: https://github.com/apache/kafka/pull/21818#issuecomment-4091897269

   @unknowntpo would you mind rebasing code?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r3005591616


##########
clients/src/main/java/org/apache/kafka/clients/consumer/StickyAssignor.java:
##########
@@ -230,7 +230,9 @@ protected MemberData memberData(Subscription subscription) {
     static ByteBuffer serializeTopicPartitionAssignment(MemberData memberData) {
         Struct struct = new Struct(STICKY_ASSIGNOR_USER_DATA_V1);
         List<Struct> topicAssignments = new ArrayList<>();
-        for (Map.Entry<String, List<Integer>> topicEntry : CollectionUtils.groupPartitionsByTopic(memberData.partitions).entrySet()) {
+        Map<String, List<Integer>> partitionsByTopic = memberData.partitions.stream()

Review Comment:
   I'm a bit concerned about the performance. Would you mind using for loop instead?
   ```java
           var partitionsByTopic = new HashMap<String, List<Integer>>();
           for (TopicPartition tp : memberData.partitions) {
               partitionsByTopic.computeIfAbsent(tp.topic(), t -> new ArrayList<>()).add(tp.partition());
           }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on PR #21818:
URL: https://github.com/apache/kafka/pull/21818#issuecomment-4102847432

   and ci errors are fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2970789869


##########
clients/src/main/java/org/apache/kafka/common/security/oauthbearer/OAuthBearerExtensionsValidatorCallback.java:
##########
@@ -90,6 +89,12 @@ public Map<String, String> ignoredExtensions() {
         return Collections.unmodifiableMap(subtractMap(subtractMap(inputExtensions.map(), invalidExtensions), validatedExtensions));

Review Comment:
   I thought the `subtractMap` is more Functional Programming style, more declarative, and this path is unlikely a hot path, which is not suitable for repeating memory allocation, but your approach is okay for me, I'll change it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2991974934


##########
clients/src/test/java/org/apache/kafka/clients/admin/internals/DescribeProducersHandlerTest.java:
##########
@@ -327,4 +326,12 @@ private DescribeProducersResponse describeProducersResponse(
         return new DescribeProducersResponse(response);
     }
 
+    private static Map<String, Map<Integer, PartitionResponse>> groupPartitionDataByTopic(

Review Comment:
   Done, since caller only pass single `TopicPartition` and `PartitionResponse`, so the `Map` input was unnecessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2991974135


##########
clients/src/main/java/org/apache/kafka/common/security/oauthbearer/OAuthBearerExtensionsValidatorCallback.java:
##########
@@ -90,6 +89,12 @@ public Map<String, String> ignoredExtensions() {
         return Collections.unmodifiableMap(subtractMap(subtractMap(inputExtensions.map(), invalidExtensions), validatedExtensions));

Review Comment:
   done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2970039141


##########
clients/src/main/java/org/apache/kafka/clients/consumer/StickyAssignor.java:
##########
@@ -272,4 +274,9 @@ private static MemberData deserializeTopicPartitionAssignment(ByteBuffer buffer)
         Optional<Integer> generation = struct.hasField(GENERATION_KEY_NAME) ? Optional.of(struct.getInt(GENERATION_KEY_NAME)) : Optional.empty();
         return new MemberData(partitions, generation);
     }
+
+    private static Map<String, List<Integer>> groupPartitionsByTopic(Collection<TopicPartition> partitions) {

Review Comment:
   Could you inline it?



##########
clients/src/test/java/org/apache/kafka/clients/admin/internals/DescribeProducersHandlerTest.java:
##########
@@ -327,4 +326,12 @@ private DescribeProducersResponse describeProducersResponse(
         return new DescribeProducersResponse(response);
     }
 
+    private static Map<String, Map<Integer, PartitionResponse>> groupPartitionDataByTopic(

Review Comment:
   we don't use such complex structure in tests actually. Maybe we could use `describeProducersResponse(TopicPartition partition, PartitionResponse partitionResponse)` instead



##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignorTest.java:
##########
@@ -1522,8 +1527,8 @@ protected void verifyValidityAndBalance(Map<String, Subscription> subscriptions,
                 if (Math.abs(len - otherLen) <= 1)
                     continue;
 
-                Map<String, List<Integer>> map = CollectionUtils.groupPartitionsByTopic(partitions);
-                Map<String, List<Integer>> otherMap = CollectionUtils.groupPartitionsByTopic(otherPartitions);
+                Map<String, List<Integer>> map = groupPartitionsByTopic(partitions);

Review Comment:
   we could streamline the test.
   ```java
   Set<String> otherTopics = otherPartitions.stream()
       .map(TopicPartition::topic)
       .collect(Collectors.toSet());
   
   for (TopicPartition tp : partitions) {
       assertFalse(otherTopics.contains(tp.topic()), 
           "Error: Some partitions can be moved...");
   }
   ```



##########
clients/src/main/java/org/apache/kafka/common/security/oauthbearer/OAuthBearerExtensionsValidatorCallback.java:
##########
@@ -90,6 +89,12 @@ public Map<String, String> ignoredExtensions() {
         return Collections.unmodifiableMap(subtractMap(subtractMap(inputExtensions.map(), invalidExtensions), validatedExtensions));

Review Comment:
   The original logic will create map repeatedly. Could you use for-loop to streamline it?
   
   ```java
   public Map<String, String> ignoredExtensions() {
       Map<String, String> ignored = new HashMap<>();
       for (Map.Entry<String, String> entry : inputExtensions.map().entrySet()) {
           String key = entry.getKey();
           if (!invalidExtensions.containsKey(key) && !validatedExtensions.containsKey(key)) {
               ignored.put(key, entry.getValue());
           }
       }
       return Collections.unmodifiableMap(ignored);
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2991974934


##########
clients/src/test/java/org/apache/kafka/clients/admin/internals/DescribeProducersHandlerTest.java:
##########
@@ -327,4 +326,12 @@ private DescribeProducersResponse describeProducersResponse(
         return new DescribeProducersResponse(response);
     }
 
+    private static Map<String, Map<Integer, PartitionResponse>> groupPartitionDataByTopic(

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #21818:
URL: https://github.com/apache/kafka/pull/21818


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20297: Cleanup `org.apache.kafka.common.utils.CollectionUtils` [kafka]

Posted by "unknowntpo (via GitHub)" <gi...@apache.org>.
unknowntpo commented on code in PR #21818:
URL: https://github.com/apache/kafka/pull/21818#discussion_r2991979603


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignorTest.java:
##########
@@ -1522,8 +1527,8 @@ protected void verifyValidityAndBalance(Map<String, Subscription> subscriptions,
                 if (Math.abs(len - otherLen) <= 1)
                     continue;
 
-                Map<String, List<Integer>> map = CollectionUtils.groupPartitionsByTopic(partitions);
-                Map<String, List<Integer>> otherMap = CollectionUtils.groupPartitionsByTopic(otherPartitions);
+                Map<String, List<Integer>> map = groupPartitionsByTopic(partitions);

Review Comment:
   Okay, done, this is simple because we only use the topic name, lesson learned.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org