[jira] [Created] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

["Jeff Chao (JIRA)" <ji...@apache.org>]

Jeff Chao created KAFKA-4725:
--------------------------------

             Summary: Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time
                 Key: KAFKA-4725
                 URL: https://issues.apache.org/jira/browse/KAFKA-4725
             Project: Kafka
          Issue Type: Bug
          Components: core, producer 
    Affects Versions: 0.10.1.1
         Environment: Ubuntu Trusty (14.04.5), Oracle JDK 8
            Reporter: Jeff Chao
         Attachments: oom-references.png

Steps to Reproduce:

1. Create a non-compacted topic with 1 partition
2. Set a produce quota of 512 KB/s
3. Send messages at 20 MB/s
4. Observe heap memory growth as time progresses

Investigation:

While running performance tests with a user configured with a produce quota, we found that the lead broker serving the requests would exhaust heap memory if the producer sustained a inbound request throughput greater than the produce quota. 

Upon further investigation, we took a heap dump from that broker process and discovered the ThrottledResponse object has a indirect reference to the byte[] holding the messages associated with the ProduceRequest. 

We're happy contributing a patch but in the meantime wanted to first raise the issue and get feedback from the community.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)