You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Vytenis Silgalis <vy...@riseinteractive.com.INVALID> on 2016/11/15 15:27:03 UTC
getting an error about: java.lang.OutOfMemoryError: Direct buffer memory
kafka version - 0.9.0.0
JVM flags:
-Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20
-XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC
-Djava.awt.headless=true
It's not easily reproducible but has caused some issues for us, any insight
would be appreciated.
Full stack:
[2016-11-15 07:27:47,850] ERROR [Replica Manager on Broker 2]: Error
processing append operation on partition [fancy_topic_name,1]
(kafka.server.ReplicaManager)
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:631)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174)
at sun.nio.ch.IOUtil.write(IOUtil.java:58)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
at
kafka.message.ByteBufferMessageSet.writeTo(ByteBufferMessageSet.scala:160)
at kafka.log.FileMessageSet.append(FileMessageSet.scala:229)
at kafka.log.LogSegment.append(LogSegment.scala:85)
at kafka.log.Log.append(Log.scala:360)
at kafka.cluster.Partition$$anonfun$9.apply(Partition.scala:442)
at kafka.cluster.Partition$$anonfun$9.apply(Partition.scala:428)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:268)
at
kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:428)
at
kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:401)
at
kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:386)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
at
scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:386)
at
kafka.server.ReplicaManager.appendMessages(ReplicaManager.scala:322)
at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:366)
at kafka.server.KafkaApis.handle(KafkaApis.scala:68)
at
kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
Thanks,
Vytenis
Re: getting an error about: java.lang.OutOfMemoryError: Direct buffer memory
Posted by Wei Li <we...@gmail.com>.
java.lang.OutOfMemoryError is not really necessary directly related with
memory usage. In your config, it requests only 1G. If your system is not
stressed, I would suggest you to check ulimit for kafka runtime user,
particularly check max number of open file descriptor and max number of
processes (nofile and nproc). If the problem is not easily reproduced,
this is likely to be the issue.
On Tue, Nov 15, 2016 at 10:27 AM, Vytenis Silgalis <
vytenis.silgalis@riseinteractive.com.invalid> wrote:
> kafka version - 0.9.0.0
> JVM flags:
> -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20
> -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC
> -Djava.awt.headless=true
>
> It's not easily reproducible but has caused some issues for us, any insight
> would be appreciated.
>
> Full stack:
> [2016-11-15 07:27:47,850] ERROR [Replica Manager on Broker 2]: Error
> processing append operation on partition [fancy_topic_name,1]
> (kafka.server.ReplicaManager)
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:631)
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174)
> at sun.nio.ch.IOUtil.write(IOUtil.java:58)
> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
> at
> kafka.message.ByteBufferMessageSet.writeTo(ByteBufferMessageSet.scala:160)
> at kafka.log.FileMessageSet.append(FileMessageSet.scala:229)
> at kafka.log.LogSegment.append(LogSegment.scala:85)
> at kafka.log.Log.append(Log.scala:360)
> at kafka.cluster.Partition$$anonfun$9.apply(Partition.scala:442)
> at kafka.cluster.Partition$$anonfun$9.apply(Partition.scala:428)
> at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
> at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:268)
> at
> kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:428)
> at
> kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.
> apply(ReplicaManager.scala:401)
> at
> kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.
> apply(ReplicaManager.scala:386)
> at
> scala.collection.TraversableLike$$anonfun$map$
> 1.apply(TraversableLike.scala:245)
> at
> scala.collection.TraversableLike$$anonfun$map$
> 1.apply(TraversableLike.scala:245)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.
> apply(HashMap.scala:99)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.
> apply(HashMap.scala:99)
> at
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
> at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> at
> kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:386)
> at
> kafka.server.ReplicaManager.appendMessages(ReplicaManager.scala:322)
> at kafka.server.KafkaApis.handleProducerRequest(
> KafkaApis.scala:366)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:68)
> at
> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks,
> Vytenis
>