[GitHub] [kafka] rajinisivaram commented on pull request #10930: KAFKA-12996; Return OFFSET_OUT_OF_RANGE for fetchOffset < startOffset even for diverging epochs

[GitBox <gi...@apache.org>]

rajinisivaram commented on pull request #10930:
URL: https://github.com/apache/kafka/pull/10930#issuecomment-869525608


   @rite2nikhil @guozhangwang @showuon Thanks for the reviews.
   
   @guozhangwang We throw `UnexpectedAppendOffsetException` only if we have data records with the wrong offset. In this case, ReplicaFetcherThread is requesting an offset that is lower than the start offset in the leader since leader's start offset was changed (e.g. fetcher thread is requesting offset 100, but leader's start offset is 200). We don't return any records since leader doesn't have records for the offset 100. Typically, leader would go through Log.read (https://github.com/apache/kafka/blob/bd1ee02b87ea508c1372af2d3982a8919e375b2d/core/src/main/scala/kafka/log/Log.scala#L1213) and throw OffsetOutOfRangeException that returns Errors.OFFSET_OUT_OF_RANGE. ReplicaFetcherThread already has special handling of this error (https://github.com/apache/kafka/blob/bd1ee02b87ea508c1372af2d3982a8919e375b2d/core/src/main/scala/kafka/server/AbstractFetcherThread.scala#L391). In case there was also a diverging epoch at the time, leader currently returns early with diverging epoch metadata 
 and Errors.NONE.   The change in this PR ensures that we throw OffsetOutOfRangeException in this case as well so that ReplicaFetcherThread applies the logic for OFFSET_OUT_OF_RANGE. Hope that makes sense.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org