[GitHub] [kafka] jsancio commented on pull request #12238: KIP-835: MetadataMaxIdleIntervalMs should be much bigger than brokerHeartbeatIntervalMs

[GitBox <gi...@apache.org>]

jsancio commented on PR #12238:
URL: https://github.com/apache/kafka/pull/12238#issuecomment-1145046468

   @dengziming Thanks for looking at this. Did you consider fixing the logic for when the controller allows the broker to unfence?
   
   The current implementation requires that `request.currentMetadataOffset() >= lastCommittedOffset`. As you point out this very difficult to achieve on a busy system. Either because of `NoOpRecord` or a lot of controller operations.
   
   For example, should we allow the broker to be behind by `A` records (or time) to unfence the broker? Another solution is to require the broker to only catch up to the last committed offset when they last sent the heartbeat. For example:
   
   1. Broker sends a hearbeat with current offset of `Y`. The last commit offset is `X`. The controller remember this last commit offset, call it `X'`
   2. Broker sends another hearbeat with current offset of `Z`. Unfence the broker if `Z >= X` or `Z >= X'`. What do you think?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org