[jira] [Assigned] (KAFKA-21317) Timestamp lookup can miss valid offsets after the log start offset advances

["Shantanu Kawale (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-21317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shantanu Kawale reassigned KAFKA-21317:
---------------------------------------

    Assignee: Shantanu Kawale

> Timestamp lookup can miss valid offsets after the log start offset advances
> ---------------------------------------------------------------------------
>
>                 Key: KAFKA-21317
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21317
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>            Reporter: Shantanu Kawale
>            Assignee: Shantanu Kawale
>            Priority: Major
>
> UnifiedLog.searchOffsetInLocalLog returns immediately from the first segment whose largestTimestamp meets the requested timestamp, even when that segment's findOffsetByTimestamp returns empty.
> This can incorrectly return no result for a timestamp lookup when the log start offset has advanced into a segment and a later segment contains a valid match.
> *Reproduction*
> Confirmed on Apache Kafka trunk commit:
> 8e46092edac3e837972ee13b40dcc149e7113ae7
> (4.5.0-SNAPSHOT, tested October 20, 2026)
> 1. Append offset 0 at timestamp T and offset 1 at timestamp T-1 into the first segment.
> 2. Roll the log and append offset 2 at timestamp T into the second segment.
> 3. Advance the high watermark to the log end.
> 4. Advance the log start offset to 1, as with DeleteRecords, leaving the first segment physically present.
> 5. Query timestamp T.
> Expected: timestamp T at offset 2.
> Actual: an empty result.
> *Root cause*
> The first segment remains eligible because its cached largest timestamp comes from offset 0. LogSegment correctly excludes that record because it is below the log start offset. UnifiedLog then returns the empty result without searching the second segment.
> *Proposed fix*
> Continue searching eligible segments until a lookup returns a result. Use in-memory segment base offsets to skip obsolete segments without reading segment tails.
> *Validation*
> - The new regression fails with trunk production code unchanged.
> - The same regression passes with the fix applied.
> - All 243 tests in UnifiedLogTest and LogSegmentTest pass.
> - Storage Spotless, Checkstyle, and SpotBugs checks pass.
> *Proposed patch:*
> https://github.com/shantanukawale-stripe/kafka/tree/fix-unified-log-timestamp-search
> Related issues
> KAFKA-10875 reports similar symptoms, but equivalence has not been established.
> KAFKA-21204 concerns the upper fetchable offset bound, not this cross-segment search failure.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)