You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Rahul R <ra...@gmail.com> on 2016/08/11 00:00:54 UTC
[Storm v0.9.6] DRPC Server periodically crashes after sometime
I have a drpc server running in nimbus server. After every 'X' minutes, I
see the following in the
drpc log
===
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
~[storm-core-0.9.6.jar:0.9.6]
====
There is enough memory available in the system .
====
storm@alerts:~/apache-storm-0.9.6/logs$ free -m
total used free shared buffers cached
Mem: 7985 2217 5767 0 81 1203
-/+ buffers/cache: 932 7052
Swap: 0 0 0
====
My DRPC Spout throws the following error
=====
java.lang.RuntimeException:
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused at
backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at
backtype.storm.daemon.executor$fn*3371$fn*3386.invoke(executor.clj:529) at
backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at
clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift7.transport.TTransportException:
java.net.ConnectException:
Connection refused at
org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at
backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47)
at
backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38)
... 5 more Caused by: java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:589)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 8 more
=======
Restarting the DRPC Server solves the problem for the moment. Is there any
other better solution ?
Thanks,
./Rahul
Re: [Storm v0.9.6] DRPC Server periodically crashes after sometime
Posted by Rahul R <ra...@gmail.com>.
After increasing memory, Most of the drpc request fail. I get the
following error,
===
2016-08-12T23:50:12.461+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 10
start at 1471045208
2016-08-12T23:55:17.471+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 11
start at 1471045515
2016-08-12T23:55:32.473+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 12
start at 1471045530
2016-08-12T23:55:47.473+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 13
start at 1471045545
2016-08-12T23:56:27.475+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 14
start at 1471045586
2016-08-12T23:59:37.482+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 15
start at 1471045775
2016-08-13T00:01:12.486+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 16
start at 1471045867
2016-08-13T00:09:42.501+0000 b.s.d.drpc [WARN] Timeout DRPC request id: 17
start at 1471046381
===
Is there anyway to debug drpc based issues ? Sometime restarting drpc
server resolves the issue, sometimes, it doesnt.
Thanks,
./Rahul
On Wed, Aug 10, 2016 at 11:17 PM, Rahul R <ra...@gmail.com> wrote:
> Sure , thats one of the solutions. But I am not sure, whats causing DRPC
> to consume soo much of memory. Any suggestions of how to debug this ?
>
> Thanks,
> ./Rahul
>
> On Wed, Aug 10, 2016 at 6:24 PM, darion.yaphet <fl...@163.com> wrote:
>
>> Maybe you should check the JVM memory parameter -Xms and -Xmx to found
>> memory usage.
>>
>>
>>
>> At 2016-08-11 08:00:54, "Rahul R" <ra...@gmail.com> wrote:
>>
>> I have a drpc server running in nimbus server. After every 'X' minutes,
>> I see the following in the
>> drpc log
>>
>> ===
>> java.lang.OutOfMemoryError: Java heap space
>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
>> at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.
>> read(TNonblockingServer.java:491) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> handleRead(TNonblockingServer.java:359) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> select(TNonblockingServer.java:304) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> run(TNonblockingServer.java:243) ~[storm-core-0.9.6.jar:0.9.6]
>> ====
>>
>>
>> There is enough memory available in the system .
>>
>> ====
>> storm@alerts:~/apache-storm-0.9.6/logs$ free -m
>> total used free shared buffers cached
>> Mem: 7985 2217 5767 0 81 1203
>> -/+ buffers/cache: 932 7052
>> Swap: 0 0 0
>> ====
>>
>>
>>
>> My DRPC Spout throws the following error
>>
>> =====
>> java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException:
>> java.net.ConnectException: Connection refused at
>> backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
>> at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at
>> backtype.storm.daemon.executor$fn*3371$fn*3386.invoke(executor.clj:529)
>> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at
>> clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
>> Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
>> at backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47)
>> at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38)
>> ... 5 more Caused by: java.net.ConnectException: Connection refused at
>> java.net.PlainSocketImpl.socketConnect(Native Method) at java.net
>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net
>> .Socket.connect(Socket.java:589) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
>> ... 8 more
>> =======
>>
>>
>> Restarting the DRPC Server solves the problem for the moment. Is there
>> any other better solution ?
>>
>>
>>
>> Thanks,
>> ./Rahul
>>
>>
>>
>>
>>
>
>
Re: Re: [Storm v0.9.6] DRPC Server periodically crashes after sometime
Posted by Rahul R <ra...@gmail.com>.
I had cannot intentionally reproduce the problem. But it seems like,
doubling the memory given to drpc only stalls the OOM error for some more
time. Which is the following.
----
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
~[na:1.8.0_91]
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
~[storm-core-0.9.6.jar:0.9.6]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
~[storm-core-0.9.6.jar:0.9.6]
2016-08-16T06:34:43.528+0000 o.a.t.s.TNonblockingServer [ERROR] run()
exiting due to uncaught error
------
I havent tinkered much with jvm OOM errors. Any help/suggestions will be
much appreciated.
Thanks,
./Rahul
On Sat, Aug 13, 2016 at 6:13 AM, darion.yaphet <fl...@163.com> wrote:
> Could you reproduction this problem ?
>
> I guess you DRPC server message consume speed is slow than message product
> .
>
> jmap maybe could help you to pick out which object make OOM :)
>
> thanks
>
>
> 在 2016-08-11 14:17:46，"Rahul R" <ra...@gmail.com> 写道：
>
> Sure , thats one of the solutions. But I am not sure, whats causing DRPC
> to consume soo much of memory. Any suggestions of how to debug this ?
>
> Thanks,
> ./Rahul
>
> On Wed, Aug 10, 2016 at 6:24 PM, darion.yaphet <fl...@163.com> wrote:
>
>> Maybe you should check the JVM memory parameter -Xms and -Xmx to found
>> memory usage.
>>
>>
>>
>> At 2016-08-11 08:00:54, "Rahul R" <ra...@gmail.com> wrote:
>>
>> I have a drpc server running in nimbus server. After every 'X' minutes,
>> I see the following in the
>> drpc log
>>
>> ===
>> java.lang.OutOfMemoryError: Java heap space
>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
>> at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.
>> read(TNonblockingServer.java:491) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> handleRead(TNonblockingServer.java:359) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> select(TNonblockingServer.java:304) ~[storm-core-0.9.6.jar:0.9.6]
>> at org.apache.thrift7.server.TNonblockingServer$SelectThread.
>> run(TNonblockingServer.java:243) ~[storm-core-0.9.6.jar:0.9.6]
>> ====
>>
>>
>> There is enough memory available in the system .
>>
>> ====
>> storm@alerts:~/apache-storm-0.9.6/logs$ free -m
>> total used free shared buffers cached
>> Mem: 7985 2217 5767 0 81 1203
>> -/+ buffers/cache: 932 7052
>> Swap: 0 0 0
>> ====
>>
>>
>>
>> My DRPC Spout throws the following error
>>
>> =====
>> java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException:
>> java.net.ConnectException: Connection refused at
>> backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
>> at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at
>> backtype.storm.daemon.executor$fn*3371$fn*3386.invoke(executor.clj:529)
>> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at
>> clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
>> Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
>> at backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47)
>> at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38)
>> ... 5 more Caused by: java.net.ConnectException: Connection refused at
>> java.net.PlainSocketImpl.socketConnect(Native Method) at java.net
>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net
>> .Socket.connect(Socket.java:589) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
>> ... 8 more
>> =======
>>
>>
>> Restarting the DRPC Server solves the problem for the moment. Is there
>> any other better solution ?
>>
>>
>>
>> Thanks,
>> ./Rahul
>>
>>
>>
>>
>>
>
>
>
>
>
Re:Re: [Storm v0.9.6] DRPC Server periodically crashes after
sometime
Posted by "darion.yaphet" <fl...@163.com>.
Could you reproduction this problem ?
I guess you DRPC server message consume speed is slow than message product .
jmap maybe could help you to pick out which object make OOM :)
thanks
在 2016-08-11 14:17:46，"Rahul R" <ra...@gmail.com> 写道：
Sure , thats one of the solutions. But I am not sure, whats causing DRPC to consume soo much of memory. Any suggestions of how to debug this ?
Thanks,
./Rahul
On Wed, Aug 10, 2016 at 6:24 PM, darion.yaphet <fl...@163.com> wrote:
Maybe you should check the JVM memory parameter -Xms and -Xmx to found memory usage.
At 2016-08-11 08:00:54, "Rahul R" <ra...@gmail.com> wrote:
I have a drpc server running in nimbus server. After every 'X' minutes, I see the following in the
drpc log
===
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243) ~[storm-core-0.9.6.jar:0.9.6]
====
There is enough memory available in the system .
====
storm@alerts:~/apache-storm-0.9.6/logs$ free -m
total used free shared buffers cached
Mem: 7985 2217 5767 0 81 1203
-/+ buffers/cache: 932 7052
Swap: 0 0 0
====
My DRPC Spout throws the following error
=====
java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40) at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at backtype.storm.daemon.executor$fn3371$fn3386.invoke(executor.clj:529) at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) at backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47) at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38) ... 5 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 8 more
=======
Restarting the DRPC Server solves the problem for the moment. Is there any other better solution ?
Thanks,
./Rahul
Re: [Storm v0.9.6] DRPC Server periodically crashes after sometime
Posted by Rahul R <ra...@gmail.com>.
Sure , thats one of the solutions. But I am not sure, whats causing DRPC to
consume soo much of memory. Any suggestions of how to debug this ?
Thanks,
./Rahul
On Wed, Aug 10, 2016 at 6:24 PM, darion.yaphet <fl...@163.com> wrote:
> Maybe you should check the JVM memory parameter -Xms and -Xmx to found
> memory usage.
>
>
>
> At 2016-08-11 08:00:54, "Rahul R" <ra...@gmail.com> wrote:
>
> I have a drpc server running in nimbus server. After every 'X' minutes, I
> see the following in the
> drpc log
>
> ===
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
> at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491)
> ~[storm-core-0.9.6.jar:0.9.6]
> at org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
> ~[storm-core-0.9.6.jar:0.9.6]
> at org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
> ~[storm-core-0.9.6.jar:0.9.6]
> at org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
> ~[storm-core-0.9.6.jar:0.9.6]
> ====
>
>
> There is enough memory available in the system .
>
> ====
> storm@alerts:~/apache-storm-0.9.6/logs$ free -m
> total used free shared buffers cached
> Mem: 7985 2217 5767 0 81 1203
> -/+ buffers/cache: 932 7052
> Swap: 0 0 0
> ====
>
>
>
> My DRPC Spout throws the following error
>
> =====
> java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException:
> java.net.ConnectException: Connection refused at backtype.storm.drpc.
> DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40) at
> backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at
> backtype.storm.daemon.executor$fn*3371$fn*3386.invoke(executor.clj:529)
> at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at
> clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
> Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
> at backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47)
> at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38)
> ... 5 more Caused by: java.net.ConnectException: Connection refused at
> java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.
> AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at
> java.net.AbstractPlainSocketImpl.connectToAddress(
> AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.
> connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.
> connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589)
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 8 more
> =======
>
>
> Restarting the DRPC Server solves the problem for the moment. Is there any
> other better solution ?
>
>
>
> Thanks,
> ./Rahul
>
>
>
>
>
Re:[Storm v0.9.6] DRPC Server periodically crashes after sometime
Posted by "darion.yaphet" <fl...@163.com>.
Maybe you should check the JVM memory parameter -Xms and -Xmx to found memory usage.
At 2016-08-11 08:00:54, "Rahul R" <ra...@gmail.com> wrote:
I have a drpc server running in nimbus server. After every 'X' minutes, I see the following in the
drpc log
===
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.8.0_91]
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_91]
at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:491) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304) ~[storm-core-0.9.6.jar:0.9.6]
at org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243) ~[storm-core-0.9.6.jar:0.9.6]
====
There is enough memory available in the system .
====
storm@alerts:~/apache-storm-0.9.6/logs$ free -m
total used free shared buffers cached
Mem: 7985 2217 5767 0 81 1203
-/+ buffers/cache: 932 7052
Swap: 0 0 0
====
My DRPC Spout throws the following error
=====
java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40) at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:87) at backtype.storm.daemon.executor$fn3371$fn3386.invoke(executor.clj:529) at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) at backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:47) at backtype.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:38) ... 5 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 8 more
=======
Restarting the DRPC Server solves the problem for the moment. Is there any other better solution ?
Thanks,
./Rahul