[GitHub] [kafka] hachikuji edited a comment on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

[GitBox <gi...@apache.org>]

hachikuji edited a comment on pull request #9590:
URL: https://github.com/apache/kafka/pull/9590#issuecomment-790233818


   @jolshan The idea is to mimic during recovery the same operation that the log cleaner was trying to do. So if we were trying to replace segments [0, 10, 20] with [15] when the operation failed, then we'd want to do the same thing during recovery. The recovery logic currently assumes that a replacement like this is not possible, so we need to adjust it. For example, we might be left with the follower files on disk when the broker starts up:
   ```
   0.log
   10.log
   20.log
   30.log
   40.log
   15.swap
   30.swap
   ```
   We would want the recovery to generate two replacements: [0, 10, 20] -> 15 and [30, 40] -> 30. The tricky thing is that the recovery operation itself might fail, so we have to basically pick up the operation from the same point and continue with the same algorithm (preserving order and all)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org