You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Thuy Nguyen <th...@gmail.com> on 2014/01/28 07:09:10 UTC

nimbus+drpc outofmemory

Hi all,

Our Storm Nimbus server and DRPC server in version 0.9.0.1 crashed after
running fine for a few days with OutOfMemory error as below. Could you
please share insights into this error? The max heap space is 1024m for
nimbus, and 768m for drpc.

*nimbus.log:

2014-01-24 18:10:51 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
stormjar-5b4fee1f-8dd4-4669-bf5c-4b23def52e89.jar
2014-01-24 18:20:51 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
stormjar-561636bb-9b87-4c9f-b32f-a22c899c0aef.jar
2014-01-24 18:50:51 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
stormjar-a0d1c40c-c646-4e4b-84a2-fe402159674c.jar
2014-01-26 10:28:04 o.a.t.s.TNonblockingServer [ERROR] Read an invalid
frame size of -2138767101. Are you using TFramedTransport on the client
side?
2014-01-26 20:18:18 o.a.t.s.TNonblockingServer [ERROR] run() exiting due to
uncaught error
java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
~[na:1.6.0_26]
        at java.nio.ByteBuffer.allocate(ByteBuffer.java:312) ~[na:1.6.0_26]
        at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
2014-01-27 11:19:45 b.s.d.nimbus [INFO] Shutting down master
2014-01-27 11:19:45 o.a.z.ZooKeeper [INFO] Session: 0x430ce840d1585c closed
2014-01-27 11:19:45 o.a.z.ClientCnxn [INFO] EventThread shut down
2014-01-27 11:19:45 b.s.d.nimbus [INFO] Shut down master

*drpc.log:
2014-01-23 18:51:16 b.s.d.drpc [INFO] Starting Distributed RPC servers...
2014-01-26 10:29:34 o.a.t.s.TNonblockingServer [ERROR] Read an invalid
frame size of -2138767101. Are you using TFramedTransport on the client
side?
2014-01-26 10:53:31 o.a.t.s.TNonblockingServer [ERROR] Read an invalid
frame size of -2138767101. Are you using TFramedTransport on the client
side?
2014-01-26 10:53:32 o.a.t.s.TNonblockingServer [ERROR] run() exiting due to
uncaught error
java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:633) ~[na:1.6.0_26]
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:98)
~[na:1.6.0_26]
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
~[na:1.6.0_26]
        at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:155)
~[na:1.6.0_26]
        at sun.nio.ch.IOUtil.read(IOUtil.java:174) ~[na:1.6.0_26]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243)
~[na:1.6.0_26]
        at
org.apache.thrift7.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.internalRead(TNonblockingServer.java:669)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:507)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
2014-01-26 20:18:13 o.a.t.s.TNonblockingServer [ERROR] run() exiting due to
uncaught error
java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
~[na:1.6.0_26]
        at java.nio.ByteBuffer.allocate(ByteBuffer.java:312) ~[na:1.6.0_26]
        at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
~[libthrift7-0.7.0-2.jar:0.7.0-2]
        at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
~[libthrift7-0.7.0-2.jar:0.7.0-2]

Thanks,
Thuy