You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "lucliu1108 (via GitHub)" <gi...@apache.org> on 2026/02/23 21:00:59 UTC

[PR] KAKFA-1251: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

lucliu1108 opened a new pull request, #21557:
URL: https://github.com/apache/kafka/pull/21557

   # Summary
   This PR moves `assignedPartitions` out of `ModernConsumerMember` interface, add it as independent properties for `ShareGroupMember` and `ConsumerGroupMember`.
   
   ## Reason for the change
   In an upcoming PR, the structure of `ConsumerGroupMember#assignedPartitions` and `ConsumerGroupMember#partitionsPendingRevocation` will be changed to include epoch information as 
   ```
   Map<Uuid, Map<Integer, Integer>>
   ```
   This differs from the `ShareGroupMember#assignedPartitions` structure, which remains `Map<Uuid, Set<Integer>>`. Therefore, it is no longer appropriate to have this as a shared field in the base class.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20066: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

Posted by "lucliu1108 (via GitHub)" <gi...@apache.org>.
lucliu1108 commented on PR #21557:
URL: https://github.com/apache/kafka/pull/21557#issuecomment-3953272416

   Thanks for the review!
   @squah-confluent @lucasbru  I notice there's also change need in `AssignorBenchmarkUtils#createGroupSpec` method. In this PR since we are still keeping the same data structure for `assignedPartitions` for both `SharedGroupMember` and `ConsumerGroupMember`, instance-wise conversion could work. However, later on when the `ConsumerGroupMember#assignedPartition` is modified to include epochs, the current fix won't work.
   
   Considering that `createGroupSpec` is only used for empty member now, does it make sense to simply pass in an empty assignment in this case?
   ```
   memberSpecs.put(memberId, new MemberSubscriptionAndAssignmentImpl(
                   Optional.ofNullable(member.rackId()),
                   Optional.ofNullable(member.instanceId()),
                   new TopicIds(member.subscribedTopicNames(), topicResolver),
                   new Assignment(Assignment.EMPTY)
               ));
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAKFA-20066: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

Posted by "squah-confluent (via GitHub)" <gi...@apache.org>.
squah-confluent commented on code in PR #21557:
URL: https://github.com/apache/kafka/pull/21557#discussion_r2843139477


##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##########
@@ -2828,7 +2828,7 @@ private CoordinatorResult<Map.Entry<ShareGroupHeartbeatResponseData, Optional<In
         //    (subscribedTopicNames) to detect a full request as those must be set in a full request.
         // 2. The member's assignment has been updated.
         boolean isFullRequest = subscribedTopicNames != null;
-        if (memberEpoch == 0 || isFullRequest || hasAssignedPartitionsChanged(member, updatedMember)) {
+        if (memberEpoch == 0 || isFullRequest || ShareGroupMember.hasAssignedPartitionsChanged(member, updatedMember)) {

Review Comment:
   For symmetry, could we also qualify the `hasAssignedPartitionsChanged` call in `consumerGroupHeartbeat`?



##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/modern/share/ShareGroupMember.java:
##########
@@ -190,9 +190,31 @@ private ShareGroupMember(
             clientId,
             clientHost,
             subscribedTopicNames,
-            state,
-            assignedPartitions
+            state
         );
+        this.assignedPartitions = assignedPartitions;
+    }
+
+    /**
+     * The partitions assigned to this member.
+     */
+    private final Map<Uuid, Set<Integer>> assignedPartitions;

Review Comment:
   I think this declaration belongs in between the builder and constructor, like in `ConsumerGroupMember`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20066: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on PR #21557:
URL: https://github.com/apache/kafka/pull/21557#issuecomment-3951697815

   nit: @lucliu1108 if you want to prefill the Reviewers header, use the `committer-tools/reviewers.py` tool to generate it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20066: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

Posted by "squah-confluent (via GitHub)" <gi...@apache.org>.
squah-confluent commented on PR #21557:
URL: https://github.com/apache/kafka/pull/21557#issuecomment-3953602554

   @lucliu1108 That's unfortunate. I would probably duplicate `createGroupSpec` and have one version for consumer group members and another for share group members.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20066: Implement KIP-1251: Assignment epochs for consumer groups [1/N] [kafka]

Posted by "lucliu1108 (via GitHub)" <gi...@apache.org>.
lucliu1108 commented on PR #21557:
URL: https://github.com/apache/kafka/pull/21557#issuecomment-3953796466

   Thanks @squah-confluent 
   Right now i spitted them into 2 different methods and rename separately, since they have the same set of parameters and will be marked as name clash.
   In the next PR, since the consumerGroupMember assignedPartitions is restructured, I'll rename them back.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org