You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Sam Goodwin <sa...@gmail.com> on 2014/06/24 21:01:43 UTC
Connection reset when submitting topology
I've been working on a Storm topology and recently it has become impossible
to submit my topology. It just times out. The only thing that has changed
since I was last able to successfully submit the topology is the size of
the jar/logic of the topology. It's worth mentioning that it has randomly
succeeded twice but fails 99% of the time. Log messages and details are
below. Any clue what's going on here?
Jar size: 28 MB
Storm version: 0.9.0_wip17_netty
Error message in nimbus.log
014-06-24 18:52:54 b.s.d.nimbus [INFO] Uploading file from client to
var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
2014-06-24 18:54:11 o.a.t.s.TNonblockingServer [WARN] Got an IOException in
internalRead!
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
~[na:1.7.0_55]
at
org.apache.thrift7.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
~[libthrift7-0.7.0.2.jar:0.7.0-2]
at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.internalRead(TNonblockingServer.java:669)
[libthrift7-0.7.0.2.jar:0.7.0-2]
at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:458)
[libthrift7-0.7.0.2.jar:0.7.0-2]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
[libthrift7-0.7.0.2.jar:0.7.0-2]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
[libthrift7-0.7.0.2.jar:0.7.0-2]
at
org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
[libthrift7-0.7.0.2.jar:0.7.0-2]
Error message on storm client:
670 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to
master yet. Submitting jar...
834 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar
build/lib/TestTopology-1.0.jar to assigned location:
var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
150532 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded
topology jar to assigned location:
var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
150532 [main] INFO backtype.storm.StormSubmitter - Submitting topology
TestTopology in distributed mode with conf {"topology.workers":12}
Exception in thread "main" java.lang.RuntimeException:
org.apache.thrift7.transport.TTransportException: java.net.SocketException:
Connection reset
at
backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:112)
at
backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58)
at com.TestTopology.main(TestTopology.java:83)
Caused by: org.apache.thrift7.transport.TTransportException:
java.net.SocketException: Connection reset
at
org.apache.thrift7.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
at
org.apache.thrift7.transport.TFramedTransport.flush(TFramedTransport.java:156)
at
org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:65)
at
backtype.storm.generated.Nimbus$Client.send_submitTopology(Nimbus.java:156)
at
backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:145)
at
backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:98)
... 2 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at
org.apache.thrift7.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
... 7 more
Re: Connection reset when submitting topology
Posted by Supun Kamburugamuva <su...@gmail.com>.
How long does it take to upload?
Thanks,
Supun..
On Tue, Jun 24, 2014 at 3:01 PM, Sam Goodwin <sa...@gmail.com>
wrote:
> I've been working on a Storm topology and recently it has become
> impossible to submit my topology. It just times out. The only thing that
> has changed since I was last able to successfully submit the topology is
> the size of the jar/logic of the topology. It's worth mentioning that it
> has randomly succeeded twice but fails 99% of the time. Log messages and
> details are below. Any clue what's going on here?
>
> Jar size: 28 MB
> Storm version: 0.9.0_wip17_netty
>
> Error message in nimbus.log
> 014-06-24 18:52:54 b.s.d.nimbus [INFO] Uploading file from client to
> var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
> 2014-06-24 18:54:11 o.a.t.s.TNonblockingServer [WARN] Got an IOException
> in internalRead!
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> ~[na:1.7.0_55]
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> ~[na:1.7.0_55]
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> ~[na:1.7.0_55]
> at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.7.0_55]
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> ~[na:1.7.0_55]
> at
> org.apache.thrift7.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
> ~[libthrift7-0.7.0.2.jar:0.7.0-2]
> at
> org.apache.thrift7.server.TNonblockingServer$FrameBuffer.internalRead(TNonblockingServer.java:669)
> [libthrift7-0.7.0.2.jar:0.7.0-2]
> at
> org.apache.thrift7.server.TNonblockingServer$FrameBuffer.read(TNonblockingServer.java:458)
> [libthrift7-0.7.0.2.jar:0.7.0-2]
> at
> org.apache.thrift7.server.TNonblockingServer$SelectThread.handleRead(TNonblockingServer.java:359)
> [libthrift7-0.7.0.2.jar:0.7.0-2]
> at
> org.apache.thrift7.server.TNonblockingServer$SelectThread.select(TNonblockingServer.java:304)
> [libthrift7-0.7.0.2.jar:0.7.0-2]
> at
> org.apache.thrift7.server.TNonblockingServer$SelectThread.run(TNonblockingServer.java:243)
> [libthrift7-0.7.0.2.jar:0.7.0-2]
>
> Error message on storm client:
> 670 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to
> master yet. Submitting jar...
> 834 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar
> build/lib/TestTopology-1.0.jar to assigned location:
> var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
> 150532 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded
> topology jar to assigned location:
> var/storm-local/nimbus/inbox/stormjar-b67d69bf-7ff1-4c99-954c-11924268d354.jar
> 150532 [main] INFO backtype.storm.StormSubmitter - Submitting topology
> TestTopology in distributed mode with conf {"topology.workers":12}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.thrift7.transport.TTransportException: java.net.SocketException:
> Connection reset
> at
> backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:112)
> at
> backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58)
> at com.TestTopology.main(TestTopology.java:83)
> Caused by: org.apache.thrift7.transport.TTransportException:
> java.net.SocketException: Connection reset
> at
> org.apache.thrift7.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
> at
> org.apache.thrift7.transport.TFramedTransport.flush(TFramedTransport.java:156)
> at
> org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:65)
> at
> backtype.storm.generated.Nimbus$Client.send_submitTopology(Nimbus.java:156)
> at
> backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:145)
> at
> backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:98)
> ... 2 more
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketOutputStream.socketWrite(Unknown Source)
> at java.net.SocketOutputStream.write(Unknown Source)
> at
> org.apache.thrift7.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
> ... 7 more
>
>
--
Supun Kamburugamuva
Member, Apache Software Foundation; http://www.apache.org
E-mail: supun06@gmail.com; Mobile: +1 812 369 6762
Blog: http://supunk.blogspot.com