[jira] [Commented] (KAFKA-5444) Producer.send() will hang 8+ hours

["Hongyuan Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054129#comment-16054129 ] 

Hongyuan Li commented on KAFKA-5444:
------------------------------------

I think this problem is associate with direct buffer allocation, use new producer api may ecountered similar problem which may not hang. I thought we can solve this in two steps:
1、move {{XX:+DisableExplicitGC}} from default {{kafka-run-class.sh}}
2、use {{sun.misc.cleaner}} to clean the directbuffer or use {{netty}} {{PlatformDependent}}#{{freeDirectBuffer}} to clean the direct buffer.

Is that right?

> Producer.send() will hang 8+ hours
> ----------------------------------
>
>                 Key: KAFKA-5444
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5444
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Hongyuan Li
>
> Frequent kafka old Producer open and close with cause the server hang with lots of error messages logged in the server.log .In my occasion,we may frequent open and close kafka producer,the procedure just like the code below:
> {code}
> Producer producer = ……
> producer.send(List<KeyedMessage> lists);
> producer.close();
> {code}
> the error is below:
> {code}
> 2017-06-13 00:00:00,084] ERROR Processor got uncaught exception. (kafka.network.Processor)
> java.lang.OutOfMemoryError: Direct buffer memory
> 	at java.nio.Bits.reserveMemory(Bits.java:658)
> 	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> 	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> 	at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174)
> 	at sun.nio.ch.IOUtil.read(IOUtil.java:195)
> 	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> 	at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110)
> 	at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97)
> 	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
> 	at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
> 	at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
> 	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:343)
> 	at org.apache.kafka.common.network.Selector.poll(Selector.java:291)
> 	at kafka.network.Processor.poll(SocketServer.scala:476)
> 	at kafka.network.Processor.run(SocketServer.scala:416)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> seen from all existing logs, all error is repeats of error above.
> Any good idea to solve this?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)