[jira] [Updated] (KAFKA-12363) Simplify static group memberId update logic

["Jason Gustafson (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Gustafson updated KAFKA-12363:
------------------------------------
    Description: 
In KAFKA-10284, we amended the JoinGroup logic to ensure that the memberId of static members always gets persisted. The way this works is the following:

1. When the JoinGroup is received, we immediately replace the current memberId with the updated memberId.
2. We then send an append to the log to update group metadata
3. If the append succeeds, we return the new memberId in the JoinGroup response.
4. If the append fails, we revert to the old memberId and we return UNKNOWN_MEMBER_ID in the response for the new member.

I am not sure if there are any correctness problems with this logic, but it does seem strange. For example, we can end up fencing the old memberId after step 1 even if we end up reverting in step 3. I think it would be simpler to structure this as follows:

1. When the JoinGroup is received, send an append to the log to update group metadata
2. If the append succeeds, replace the existing memberId with the new committed memberId.
3. If the append fails, return UNKNOWN_MEMBER_ID to let the new member retry.

Basically we don't surface the effect of the member replacement until we know it has been committed to the log, which avoids the weird revert logic.

  was:
In KAFKA-10284, we amended the JoinGroup logic to ensure that the memberId of static members always gets persisted. The way this works is the following:

1. When the JoinGroup is received, we immediately replace the current memberId with the updated memberId.
2. We then send an append to the log to update group metadata
3. If the append is unsuccessful, we revert to the old memberId and we return UNKNOWN_MEMBER_ID in the response for the new member.
4. If the append is successful, we return the new memberId in the JoinGroup response.

I am not sure if there are any correctness problems with this logic, but it does seem strange. For example, we can end up fencing the old memberId after step 1 even if we end up reverting in step 3. I think it would be simpler to structure this as follows:

1. When the JoinGroup is received, send an append to the log to update group metadata
2. If the append succeeds, replace the existing memberId with the new committed memberId.
3. If the append fails, return UNKNOWN_MEMBER_ID to let the new member retry.

Basically we don't surface the effect of the member replacement until we know it has been committed to the log, which avoids the weird revert logic.


> Simplify static group memberId update logic
> -------------------------------------------
>
>                 Key: KAFKA-12363
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12363
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Jason Gustafson
>            Priority: Major
>
> In KAFKA-10284, we amended the JoinGroup logic to ensure that the memberId of static members always gets persisted. The way this works is the following:
> 1. When the JoinGroup is received, we immediately replace the current memberId with the updated memberId.
> 2. We then send an append to the log to update group metadata
> 3. If the append succeeds, we return the new memberId in the JoinGroup response.
> 4. If the append fails, we revert to the old memberId and we return UNKNOWN_MEMBER_ID in the response for the new member.
> I am not sure if there are any correctness problems with this logic, but it does seem strange. For example, we can end up fencing the old memberId after step 1 even if we end up reverting in step 3. I think it would be simpler to structure this as follows:
> 1. When the JoinGroup is received, send an append to the log to update group metadata
> 2. If the append succeeds, replace the existing memberId with the new committed memberId.
> 3. If the append fails, return UNKNOWN_MEMBER_ID to let the new member retry.
> Basically we don't surface the effect of the member replacement until we know it has been committed to the log, which avoids the weird revert logic.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)