[GitHub] [kafka] junrao commented on a change in pull request #10684: MINOR: Improve Log layer segment iteration logic and few other areas

[GitBox <gi...@apache.org>]

junrao commented on a change in pull request #10684:
URL: https://github.com/apache/kafka/pull/10684#discussion_r638164890



##########
File path: core/src/main/scala/kafka/log/LogSegments.scala
##########
@@ -210,11 +209,23 @@ class LogSegments(topicPartition: TopicPartition) {
    * @return the entry associated with the greatest offset, if it exists.
    */
   @threadsafe
-  def lastEntry: Option[Map.Entry[JLong, LogSegment]] = Option(segments.lastEntry)
+  def lastEntry: Option[Map.Entry[Long, LogSegment]] = Option(segments.lastEntry)
 
   /**
    * @return the log segment with the greatest offset, if it exists.
    */
   @threadsafe
   def lastSegment: Option[LogSegment] = lastEntry.map(_.getValue)
+
+  /**
+   * @return an iterable with log segments ordered from lowest base offset to highest,
+   *         each segment returned  has a base offset strictly greater than the provided baseOffset.
+   */
+  def higherSegments(baseOffset: Long): Iterable[LogSegment] = {
+    val view =
+      Option(segments.higherKey(baseOffset)).map {
+        higherOffset => segments.tailMap(higherOffset, true)
+      }.getOrElse(new ConcurrentSkipListMap[Long, LogSegment]())

Review comment:
       Could we return a constant empty map?

##########
File path: core/src/main/scala/kafka/log/Log.scala
##########
@@ -1217,10 +1213,8 @@ class Log(@volatile private var _dir: File,
           fetchDataInfo = segment.read(startOffset, maxLength, maxPosition, minOneMessage)
           if (fetchDataInfo != null) {
             if (includeAbortedTxns)
-              fetchDataInfo = addAbortedTransactions(startOffset, segmentEntry, fetchDataInfo)
-          } else segmentEntryOpt = segments.higherEntry(baseOffset)
-
-          done = fetchDataInfo != null || segmentEntryOpt.isEmpty
+              fetchDataInfo = addAbortedTransactions(startOffset, segment, fetchDataInfo)
+          } else segmentOpt = segmentsIterator.nextOption()

Review comment:
       The old logic supports skipping forward multiple segments to find the right data. The new logic seems to only support skipping forward once. It would be useful to preserve the original semantic.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org