[jira] [Created] (KAFKA-21203) recovery could fail when a preallocated segment hits offset overflow

["Jun Rao (Jira)" <ji...@apache.org>]

Jun Rao created KAFKA-21203:
-------------------------------

             Summary: recovery could fail when a preallocated segment hits offset overflow
                 Key: KAFKA-21203
                 URL: https://issues.apache.org/jira/browse/KAFKA-21203
             Project: Kafka
          Issue Type: Bug
            Reporter: Jun Rao


During log recovery, we discover offset overflow in a segment. If an overflow is detected, it stops the recovery and calls splitOverflowedSegment() to split the segment. However, since the segment hasn't been fully recovered, there could be untrimmed padding bytes in the segment if preallocation is enabled. This will cause an exception in splitOverflowedSegment() and fail log loading. The following call sequence shows how this can happen.

 
{code:java}
LogLoader.recoverLog()                                   LogLoader.java:456
  └─ retryOnOffsetOverflow(this::recoverLog)             :210  (outer) / :330 catch
       └─ recoverSegment(segment)                        :400
            └─ LogSegment.recover(...)                    LogSegment.java:482
                 for (RecordBatch batch : log.batches())  :490
                   batch.ensureValid()                    :491
                   ensureOffsetInRange(batch.lastOffset())  :492  ──► throws
                                                          LogSegmentOffsetOverflowException
  ── caught by retryOnOffsetOverflow ──                   LogLoader.java:330
       └─ LocalLog.splitOverflowedSegment(segment,...)    LocalLog.java:909
            while (position < sourceRecords.sizeInBytes())  :926  ◄── sizeInBytes() incl. padding
              sourceRecords.batchesFrom(position).iterator().next()  :927  ──► CorruptRecordException
              newSegment.appendFromFile(sourceRecords, position)     :930
                 appendChunkFromFile → records.batchesFrom(position).iterator()  LogSegment.java:294
{code}
 

Note that this issue likely won't happen in practice since the offset overflow case typically happens on existing rolled segment which has been trimmed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)