[GitHub] [kafka] hachikuji commented on pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

[GitBox <gi...@apache.org>]

hachikuji commented on pull request #8479:
URL: https://github.com/apache/kafka/pull/8479#issuecomment-628204174


   @andrewchoi5 Just want to make sure I understand the problem. The scenario is that we lose the zk session while handling a LeaderAndIsr request. Current LeaderAndIsr handling works like this:
   
   1. Check epoch of each partition. If it is less than or equal to current epoch, ignore the update.
   2. Update the epoch for each partition.
   3. Make followers and leaders, which involves loading topic configs from zk.
   
   So the problem occurs when we hit an error loading the topic configs in step 3). This potentially causes us to miss the needed state changes from that request and also prevents us from being able to retry them because the epoch has already been updated. Is that right?
   
   I guess the fix here is only a partial fix. We would still be left with the one failed partition, right?
   
   Off the top of my head, I am wondering whether we should be retrying the request at a lower level. For example, maybe `getEntityConfigs` could catch the `ZooKeeperClientExpiredException` and retry. I assume there is a good reason we do not catch this exception in `retryRequestUntilConnected` already. Perhaps it is unsafe to assume that the requests are still valid after a session expiration. However, just for reading configurations, I do not see a problem retrying after a session expiration.
   
   cc @junrao 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org