You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/09/06 17:52:47 UTC
[GitHub] [kafka] philipnee opened a new pull request, #12593: KAFKA-14196: Prevent fetching during the rebalancing
philipnee opened a new pull request, #12593:
URL: https://github.com/apache/kafka/pull/12593
```
1203 {"timestamp":1662486291187,"name":"records_consumed","count":2,"partitions":[{"topic":"test_topic","partition":0,"count":2,"minOffset":3566,"maxOffset":3567}]}
1204 {"timestamp":1662486291192,"name":"records_consumed","count":1,"partitions":[{"topic":"test_topic","partition":0,"count":1,"minOffset":3568,"maxOffset":3568}]}
1205 {"timestamp":1662486291198,"name":"partitions_revoked","partitions":[{"topic":"test_topic","partition":0}]}
1206 {"timestamp":1662486291209,"name":"partitions_assigned","partitions":[{"topic":"test_topic","partition":0}]}
1207 {"timestamp":1662486291218,"name":"records_consumed","count":12,"partitions":[{"topic":"test_topic","partition":0,"count":12,"minOffset":3569,"maxOffset":3580}]}
1208 {"timestamp":1662486291219,"name":"records_consumed","count":2,"partitions":[{"topic":"test_topic","partition":0,"count":2,"minOffset":3581,"maxOffset":3582}]}
1209 {"timestamp":1662486291220,"name":"records_consumed","count":1,"partitions":[{"topic":"test_topic","partition":0,"count":1,"minOffset":3583,"maxOffset":3583}]}
```
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji commented on a diff in pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
Posted by GitBox <gi...@apache.org>.
hachikuji commented on code in PR #12593:
URL: https://github.com/apache/kafka/pull/12593#discussion_r966321913
##########
clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java:
##########
@@ -1282,6 +1282,11 @@ private Fetch<K, V> pollForFetches(Timer timer) {
long pollTimeout = coordinator == null ? timer.remainingMs() :
Math.min(coordinator.timeToNextPoll(timer.currentTimeMs()), timer.remainingMs());
+ if(coordinator.isCommittingOffsetAsync()) {
Review Comment:
Hmm, do we want to do this in the general case? I think my expectation is that we would not continue fetching for partitions only when we have sent the offset commit and we are awaiting revocation as part of a rebalance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] philipnee closed pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
Posted by GitBox <gi...@apache.org>.
philipnee closed pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
URL: https://github.com/apache/kafka/pull/12593
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] philipnee commented on pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
Posted by GitBox <gi...@apache.org>.
philipnee commented on PR #12593:
URL: https://github.com/apache/kafka/pull/12593#issuecomment-1238583494
@dajac - The issue is actually quite complicated, but i'm seeing consumer consuming duplicated records because of the fetching in between the start of async commit and partition revoke (while the async commit in "in-flight"). The original behavior was using sync commit, but I think we are seeing coordinator getting stuck there.
Sorry... forgot to converted it into a draft.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
Posted by GitBox <gi...@apache.org>.
dajac commented on PR #12593:
URL: https://github.com/apache/kafka/pull/12593#issuecomment-1238523595
Hum… Isn’t fetching expected during a rebalance in the cooperative mode? Could you elaborate more on the issue that you are trying to solve?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] philipnee commented on pull request #12593: KAFKA-14196: Prevent fetching during the rebalancing
Posted by GitBox <gi...@apache.org>.
philipnee commented on PR #12593:
URL: https://github.com/apache/kafka/pull/12593#issuecomment-1238575066
sorry.. didn't mean to publish this for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org