You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/10/16 15:35:59 UTC

[GitHub] [kafka] vvcephei commented on a change in pull request #9270: KAFKA-10284: Group membership update due to static member rejoin should be persisted

vvcephei commented on a change in pull request #9270:
URL: https://github.com/apache/kafka/pull/9270#discussion_r506551647



##########
File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala
##########
@@ -1037,24 +1037,52 @@ class GroupCoordinator(val brokerId: Int,
 
     val knownStaticMember = group.get(newMemberId)
     group.updateMember(knownStaticMember, protocols, responseCallback)
+    val oldProtocols = knownStaticMember.supportedProtocols
 
     group.currentState match {
       case Stable =>
-        info(s"Static member joins during Stable stage will not trigger rebalance.")
-        group.maybeInvokeJoinCallback(member, JoinGroupResult(
-          members = List.empty,
-          memberId = newMemberId,
-          generationId = group.generationId,
-          protocolType = group.protocolType,
-          protocolName = group.protocolName,
-          // We want to avoid current leader performing trivial assignment while the group
-          // is in stable stage, because the new assignment in leader's next sync call
-          // won't be broadcast by a stable group. This could be guaranteed by
-          // always returning the old leader id so that the current leader won't assume itself
-          // as a leader based on the returned message, since the new member.id won't match
-          // returned leader id, therefore no assignment will be performed.
-          leaderId = currentLeader,
-          error = Errors.NONE))
+        // check if group's selectedProtocol of next generation will change, if not, simply store group to persist the
+        // updated static member, if yes, rebalance should be triggered to let the group's assignment and selectProtocol consistent
+        val selectedProtocolOfNextGeneration = group.selectProtocol
+        if (group.protocolName.contains(selectedProtocolOfNextGeneration)) {
+          info(s"Static member which joins during Stable stage and doesn't affect selectProtocol will not trigger rebalance.")
+          val groupAssignment: Map[String, Array[Byte]] = group.allMemberMetadata.map(member => member.memberId -> member.assignment).toMap
+          groupManager.storeGroup(group, groupAssignment, error => {
+            if (error != Errors.NONE) {
+              warn(s"Failed to persist metadata for group ${group.groupId}: ${error.message}")
+
+              // Failed to persist member.id of the given static member, revert the update of the static member in the group.
+              group.updateMember(knownStaticMember, oldProtocols, null)
+              val oldMember = group.replaceGroupInstance(newMemberId, oldMemberId, groupInstanceId)
+              completeAndScheduleNextHeartbeatExpiration(group, oldMember)
+              responseCallback(JoinGroupResult(
+                List.empty,
+                memberId = JoinGroupRequest.UNKNOWN_MEMBER_ID,
+                generationId = group.generationId,
+                protocolType = group.protocolType,
+                protocolName = group.protocolName,
+                leaderId = currentLeader,
+                error = error
+              ))
+            }
+          })
+          group.maybeInvokeJoinCallback(member, JoinGroupResult(
+            members = List.empty,
+            memberId = newMemberId,
+            generationId = group.generationId,
+            protocolType = group.protocolType,
+            protocolName = group.protocolName,
+            // We want to avoid current leader performing trivial assignment while the group
+            // is in stable stage, because the new assignment in leader's next sync call
+            // won't be broadcast by a stable group. This could be guaranteed by
+            // always returning the old leader id so that the current leader won't assume itself
+            // as a leader based on the returned message, since the new member.id won't match
+            // returned leader id, therefore no assignment will be performed.
+            leaderId = currentLeader,
+            error = Errors.NONE))

Review comment:
       Hey @feyman2016 ,
   
   I'm still wondering if this block should be inside the callback of `storeGroup`. Otherwise, we would already have sent the response to the client before the `storeGroup` completes, and the client will never see the error response on L1058.
   
   Or did we specifically decide to make the `storeGroup` call best effort?
   
   (cc @abbccdda )




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org