[GitHub] [kafka] dhruvilshah3 commented on a change in pull request #10388: KAFKA-12520: Ensure log loading does not truncate producer state unless required

[GitBox <gi...@apache.org>]

dhruvilshah3 commented on a change in pull request #10388:
URL: https://github.com/apache/kafka/pull/10388#discussion_r601865901



##########
File path: core/src/main/scala/kafka/log/LogSegment.scala
##########
@@ -322,17 +323,14 @@ class LogSegment private[log] (val log: FileRecords,
      offsetIndex.fetchUpperBoundOffset(startOffsetPosition, fetchSize).map(_.offset)
 
   /**
-   * Run recovery on the given segment. This will rebuild the index from the log file and lop off any invalid bytes
-   * from the end of the log and index.
+   * Ensure batches in the segment are valid and rebuild all corresponding indices.
    *
-   * @param producerStateManager Producer state corresponding to the segment's base offset. This is needed to recover
-   *                             the transaction index.
-   * @param leaderEpochCache Optionally a cache for updating the leader epoch during recovery.
-   * @return The number of bytes truncated from the log
+   * @param batchCallbackOpt Optional callback invoked for all valid batches in segment
+   * @return The number of invalid bytes at the end of the segment
    * @throws LogSegmentOffsetOverflowException if the log segment contains an offset that causes the index offset to overflow
    */
   @nonthreadsafe
-  def recover(producerStateManager: ProducerStateManager, leaderEpochCache: Option[LeaderEpochFileCache] = None): Int = {
+  def validateSegmentAndRebuildIndices(batchCallbackOpt: Option[FileChannelRecordBatch => Unit] = None) : Int = {
     offsetIndex.reset()
     timeIndex.reset()
     txnIndex.reset()

Review comment:
       There is another problem here in that we are not rebuilding the transaction index. The current logic seems pretty tied up with producer state maintenance. I will try to see if there's a way to separate it out.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org