You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/04/08 05:41:00 UTC
[jira] [Commented] (KAFKA-9823) Consumer should check equality of
the generation for coordinator requests
[ https://issues.apache.org/jira/browse/KAFKA-9823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077853#comment-17077853 ]
ASF GitHub Bot commented on KAFKA-9823:
---------------------------------------
guozhangwang commented on pull request #8445: KAFKA-9823: Remember the sent generation for the coordinator request
URL: https://github.com/apache/kafka/pull/8445
For join / sync / commit / heartbeat request, we would remember the sent generation in the created handler object, and then upon getting the error code, we could check whether the sent generation still matches the current generation. If not, it means that the member has already reset its generation or has participated in a new rebalance already. This means:
1. For join / sync-group request, we do not need to call reset-generation any more for illegal-generation / unknown-member. But we would still set the error since at a given time only one join/sync round-trip would be in flight, and hence we should not be participating in a new rebalance. Also for fenced instance error we still treat it as fatal since we should not be participating in a new rebalance, so this is still not expected.
2. For commit request, we do not set the corresponding error for illegal-generation / unknown-member / fenced-instance but raise rebalance-in-progress. For commit-sync it would be still thrown to user, while for commit-async it would be logged and swallowed.
3. For heartbeat request, we do not treat illegal-generation / unknown-member / fenced-instance errors and just consider it as succeeded since this should be a stale heartbeat which can be ignored.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Consumer should check equality of the generation for coordinator requests
> -------------------------------------------------------------------------
>
> Key: KAFKA-9823
> URL: https://issues.apache.org/jira/browse/KAFKA-9823
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Priority: Major
>
> In consumer's requests to group coordinator (heartbeat, join-group, sync-group, commit; leave-group is not considered since consumer do not check its responses anyways), we encoded the generation / member information and the response may indicate that the member.id is invalid or generation is stale, which would cause consumer to reset and re-join group.
> However, when the response is sent back it is possible that the consumer has already started re-join due to other channels indicating it out of the group, and hence resetting would introduce unnecessarily more re-joining operations.
> We should, instead, remember the generation information that was sent along with the request and upon getting the response compare that with the current generation information. If they do not match it means the responded error indicating stale information has been updated in other places and hence can be handled differently (for example, in heartbeat handling we can just ignore the error).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)