[jira] [Commented] (KAFKA-7422) Huge memory consumption (leak?) by Kafka Producer

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644383#comment-16644383 ] 

ASF GitHub Bot commented on KAFKA-7422:
---------------------------------------

huxihx opened a new pull request #5770: KAFKA-7422: forceUnmap mmap on linux when index resize
URL: https://github.com/apache/kafka/pull/5770
 
 
   https://issues.apache.org/jira/browse/KAFKA-7442
   
   In AbstractIndex#resize, `safeForceUnmap` should also be invoked for Linux.
   
   *More detailed description of your change,
   if necessary. The PR title and PR message become
   the squashed commit message, so use a separate
   comment to ping reviewers.*
   
   *Summary of testing strategy (including rationale)
   for the feature or bug fix. Unit and/or integration
   tests are expected for any behaviour change and
   system tests should be considered for larger changes.*
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Huge memory consumption (leak?) by Kafka Producer
> -------------------------------------------------
>
>                 Key: KAFKA-7422
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7422
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>    Affects Versions: 0.10.1.0
>         Environment: OS - AIX 7.1, Java - IBM Java 1.8
>            Reporter: Shantanu Deshmukh
>            Priority: Major
>         Attachments: heapdump.20180918.111423.4587876.0002.phd
>
>
> Hello,
>  
> We have a 3 broker Kafka 0.10.1.0 deployment in production. There are some applications which have Kafka Producers embedded in them which send application logs to a topic. This topic has 10 partitions with replication factor of 3.
>  
> We are observing that memory usage on some of these application servers keep shooting through the roof intermittently. After taking heapdump we found out that top suspects were:
> *---------------------*
> *org.apache.kafka.common.network.Selector -*
> occupies *352,519,104 (24.96%)* bytes. The memory is accumulated in one instance of *"byte[]"* loaded by *"<system class loader>"*.
> *org.apache.kafka.common.network.KafkaChannel -*
> occupies *352,527,424 (24.96%)* bytes. The memory is accumulated in one instance of *"byte[]"* loaded by *"<system class loader>"*
> *---------------------*
> Both of these were holding about 352MB of space. 3 such instances, so they were consuming about 1.2GB of memory.
> Now regarding usage of producers. Not a huge amount of logs are being sent to Kafka cluster. It is about 200 msgs/sec. Only one producer object is being used throughout application. Async send function is used.
> What could be the cause of such huge memory usage? Is this some sort of memory leak in this specific Kafka version?
> Here's producer config being used at the present.
> {{>>>>>>>>>>>>>>>>}}
> {{kafka.bootstrap.servers=x.x.x.x:9092,x.x.x.x:9092,x.x.x.x:9092}}
> {{ kafka.acks=0}}
> {{ kafka.key.serializer=org.apache.kafka.common.serialization.StringSerializer}}
> {{ kafka.value.serializer=org.apache.kafka.common.serialization.StringSerializer}}
> {{[kafka.max.block.ms|http://kafka.max.block.ms/]=1000}}
> {{[kafka.request.timeout.ms|http://kafka.request.timeout.ms/]=1000}}
> {{ kafka.max.in.flight.requests.per.connection=1}}
> {{ kafka.retries=0}}
> {{ kafka.compression.type=gzip}}
> {{ kafka.security.protocol=SSL}}
> {{ kafka.ssl.truststore.location=/data/kafka/kafka-server-truststore.jks}}
> {{ kafka.ssl.truststore.password=XXXXXX}}
> {{ logger.level=INFO }}{{<<<<<<<<<<<<<<}}
> Attaching heapdump for your perusal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)