[jira] [Created] (KAFKA-14222) Exhausted BatchMemoryPool

["Jose Armando Garcia Sancio (Jira)" <ji...@apache.org>]

Jose Armando Garcia Sancio created KAFKA-14222:
--------------------------------------------------

             Summary: Exhausted BatchMemoryPool
                 Key: KAFKA-14222
                 URL: https://issues.apache.org/jira/browse/KAFKA-14222
             Project: Kafka
          Issue Type: Bug
          Components: kraft
            Reporter: Jose Armando Garcia Sancio
            Assignee: Jose Armando Garcia Sancio
             Fix For: 3.3.0


For a large number of topics and partition the broker can encounter this issue:
{code:java}
[2022-09-12 14:14:42,114] ERROR [BrokerMetadataSnapshotter id=4] Unexpected error handling CreateSnapshotEvent (kafka.server.metadata.BrokerMetadataSnapshotter)
org.apache.kafka.raft.errors.BufferAllocationException: Append failed because we failed to allocate memory to write the batch
        at org.apache.kafka.raft.internals.BatchAccumulator.append(BatchAccumulator.java:161)
        at org.apache.kafka.raft.internals.BatchAccumulator.append(BatchAccumulator.java:112)
        at org.apache.kafka.snapshot.RecordsSnapshotWriter.append(RecordsSnapshotWriter.java:167)
        at kafka.server.metadata.RecordListConsumer.accept(BrokerMetadataSnapshotter.scala:49)
        at kafka.server.metadata.RecordListConsumer.accept(BrokerMetadataSnapshotter.scala:42)
        at org.apache.kafka.image.TopicImage.write(TopicImage.java:78)
        at org.apache.kafka.image.TopicsImage.write(TopicsImage.java:79)
        at org.apache.kafka.image.MetadataImage.write(MetadataImage.java:129)
        at kafka.server.metadata.BrokerMetadataSnapshotter$CreateSnapshotEvent.run(BrokerMetadataSnapshotter.scala:116)
        at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:121)
        at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:200)
        at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:173)
        at java.base/java.lang.Thread.run(Thread.java:829) {code}
This can happen because the snapshot is larger than {{{}5 * 8MB{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)