[jira] [Created] (KAFKA-14407) Able to access offset index memory map even after memory map flag is turned off

["Jeff Kim (Jira)" <ji...@apache.org>]

Jeff Kim created KAFKA-14407:
--------------------------------

             Summary: Able to access offset index memory map even after memory map flag is turned off
                 Key: KAFKA-14407
                 URL: https://issues.apache.org/jira/browse/KAFKA-14407
             Project: Kafka
          Issue Type: Bug
            Reporter: Jeff Kim


Original investigation: [https://github.com/apache/kafka/pull/12783#issuecomment-1315804689]

 

we have mechanisms in place to check before accessing a memory map. an example:

  private def fetchHighWatermarkMetadata: LogOffsetMetadata = {
    localLog.checkIfMemoryMappedBufferClosed()
    ...
  }

 which makes it seem okay to read from the offset index. however, we should not allow any disk io if an offset index is unmapped. moreover, if a local log is closed, we should set `isMemoryMappedBufferClosed` to true in `LocalLog.close()`:

  private[log] def close(): Unit = {
    maybeHandleIOException(s"Error while renaming dir for $topicPartition in dir ${dir.getParent}") {
      checkIfMemoryMappedBufferClosed()
      segments.close()
      // isMemoryMappedBufferClosed = true?
    }
  }

note that besides altering replica log dir, UnifiedLog.close() is only called during shutdown.

i'm not sure if this is a bug but accessing the high watermark (by reading from the actual segment via offset index) will result in a NPE because isMemoryMappedBufferClosed is false. 

java.lang.NullPointerException
	at kafka.log.OffsetIndex.$anonfun$lookup$1(OffsetIndex.scala:90)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)