You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by sofia tzima <tz...@gmail.com> on 2015/02/15 15:21:09 UTC
timeout while loading big file on bolt
Hello all!
I am having a trouble when trying to deploy a simple storm trident topology
to the cluster.
I use BaseFunction to implement a bolt to execute a resemblance calculator.
In order to achieve that, the bolt has to load some big graphs (from a
file) on prepare function.
Locally, the topology seems to run fine, even though it starts processing
tuples late due to the graph loading, but when i submit it to the cluster
it wont work the same way.
I took a look at the logs and found out that the topology loads the first
graph successfully, but then zookeeper experiences timeout (i guess) and
tries to reconnect to the client that loads that specific bolt, as appears
below. Then the bolt restarts.
I tried to increase zk timeout and restarted zk and the storm cluster, but
still get the same timeout (2000) on logs.
*zoo.cgf:# The number of milliseconds of each ticktickTime=10000*
Is there something i am missing here? Is there any way to achieve the bolt
not to timeout until the graphs are loaded?
Thank you in advance,
*Log of worker that loads the specific bolt:*
2015-02-15T16:00:09.519+0200 g.n.u.Classifier [INFO]
Loading graph #2...
2015-02-15T16:00:21.083+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 15489ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:00:21.193+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:00:22.698+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:00:22.700+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:00:22.708+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a00020,
negotiated timeout = 20000
2015-02-15T16:00:22.709+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:00:36.697+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 13473ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:00:36.811+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:00:39.024+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:00:39.027+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:00:39.032+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a00020,
negotiated timeout = 20000
2015-02-15T16:00:39.034+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:01:01.127+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 14385ms for sessionid 0x14b8d7c14a0001e,
closing socket connection and attempting reconnect
2015-02-15T16:01:01.127+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 14459ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:01:01.229+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:01:01.231+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:01:01.232+0200 b.s.cluster [WARN] Received event
:disconnected::none: with disconnected Zookeeper.
2015-02-15T16:01:02.417+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:01:02.419+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:01:02.427+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a00020,
negotiated timeout = 20000
2015-02-15T16:01:02.429+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:01:03.115+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:01:03.117+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:01:03.130+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a0001e,
negotiated timeout = 20000
2015-02-15T16:01:03.130+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:01:40.828+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 13722ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:01:40.940+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:01:42.287+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:01:42.290+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:01:42.302+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a00020,
negotiated timeout = 20000
2015-02-15T16:01:42.303+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:01:58.504+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 14435ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:01:58.504+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 14541ms for sessionid 0x14b8d7c14a0001e,
closing socket connection and attempting reconnect
2015-02-15T16:01:58.605+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:01:58.621+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:01:58.623+0200 b.s.cluster [WARN] Received event
:disconnected::none: with disconnected Zookeeper.
2015-02-15T16:02:00.001+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:02:00.003+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:02:00.011+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP:2181, sessionid = 0x14b8d7c14a00020,
negotiated timeout = 20000
2015-02-15T16:02:00.012+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:02:00.261+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:02:00.263+0200 o.a.s.z.ClientCnxn [INFO] Socket connection
established to IP/IP:2181, initiating session
2015-02-15T16:02:00.271+0200 o.a.s.z.ClientCnxn [INFO] Session
establishment complete on server IP/IP, sessionid = 0x14b8d7c14a0001e,
negotiated timeout = 20000
2015-02-15T16:02:00.272+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: RECONNECTED
2015-02-15T16:02:25.314+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 13822ms for sessionid 0x14b8d7c14a0001e,
closing socket connection and attempting reconnect
2015-02-15T16:02:25.315+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 13891ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:02:25.416+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:02:25.417+0200 b.s.cluster [WARN] Received event
:disconnected::none: with disconnected Zookeeper.
2015-02-15T16:02:25.419+0200 o.a.s.c.f.s.ConnectionStateManager [INFO]
State change: SUSPENDED
2015-02-15T16:02:45.826+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:02:45.827+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 20409ms for sessionid 0x14b8d7c14a00020,
closing socket connection and attempting reconnect
2015-02-15T16:02:45.827+0200 o.a.s.z.ClientCnxn [INFO] Opening socket
connection to server IP/IP:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-02-15T16:02:45.831+0200 o.a.s.z.ClientCnxn [INFO] Client session timed
out, have not heard from server in 20416ms for sessionid 0x14b8d7c14a0001e,
closing socket connection and attempting reconnect
2015-02-15T16:02:55.058+0200 o.a.s.c.ConnectionState [WARN] Connection
attempt unsuccessful after 29639 (greater than max timeout of 20000).
Resetting connection and trying again with a new connection.