[GitHub] [kafka] philipnee opened a new pull request, #12580: MINOR: Fix flaky OffsetValidationTest

[GitBox <gi...@apache.org>]

philipnee opened a new pull request, #12580:
URL: https://github.com/apache/kafka/pull/12580

   We are observing flakiness in the OffsetValidationTest, and I believe (evidently) it is due to not handling the uncommitted offset during the rebalancing.  Here the fix is very simple, commit the offset when the onPartitionRevoke is invoked.
   
   Here are the findings of this observation:
   
   **Error:**
   AssertionError: Total consumed records 3813 did not match consumed position 3809
   
   Of the 3 consumers it spawned, only 2 were actually consuming data (maybe it is just this specific run), and the largest offset was actually 3809.
   
   ```
   {"timestamp":1661999681665,"name":"records_consumed","count":1,"partitions":[{"topic":"test_topic","partition":0,"count":1,"minOffset":3808,"maxOffset":3808}]}
   {"timestamp":1661999681680,"name":"partitions_revoked","partitions":[{"topic":"test_topic","partition":0}]}
   {"timestamp":1661999681693,"name":"shutdown_complete"}
   ```
   Also, during the rebalancing, turned out, one of the consumer re-consume some of the already consumed reacord
   
   ```
   819 {"timestamp":1661999681453,"name":"records_consumed","count":1,"partitions":[{"topic":"test_topic","partition":0,"count":1,"minOffset":3703,"maxOffset":3703}]}
   820 {"timestamp":1661999681454,"name":"partitions_revoked","partitions":[{"topic":"test_topic","partition":0}]}
   821 {"timestamp":1661999681465,"name":"partitions_assigned","partitions":[{"topic":"test_topic","partition":0}]}
   822 {"timestamp":1661999681471,"name":"records_consumed","count":12,"partitions":[{"topic":"test_topic","partition":0,"count":12,"minOffset":3700,"maxOffset":3711}]}
   ```
   (sorry for the vim line number)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] philipnee closed pull request #12580: MINOR: Fix flaky OffsetValidationTest

[GitBox <gi...@apache.org>]

philipnee closed pull request #12580: MINOR: Fix flaky OffsetValidationTest
URL: https://github.com/apache/kafka/pull/12580


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org