You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Richards Peter <hb...@gmail.com> on 2015/05/13 13:23:26 UTC
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Hi,
I am using storm-0.8.2 in my system. My project, based on storm, opens
connections to database/cache-server from spout/bolts. I need some of the
connections to remain open till the topologies are active. However I would
like to release these open connections from spout/bolt code when I kill my
topology.
Could you please tell me how the resources(cache-server or database
connections) in bolts/spouts can be released using storm 0.9.3 and higher
versions?
I am planning to upgrade storm, used in my project, to storm-0.9.4 for
various reasons. If I can release resources (cache-server or database
connections) from spout and bolt, it would another good reason for storm
upgrade.
Thanks,
Richards Peter.
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by Javier Gonzalez <ja...@gmail.com>.
Isn't the cleanup method guaranteed to be called only while running as
local topology?
On May 13, 2015 9:20 AM, "Jeffery Maass" <ma...@gmail.com> wrote:
> Bolts which implement IBolt have a method called cleanup() which is called
> by the Storm framework.
>
> https://storm.apache.org/apidocs/backtype/storm/task/IBolt.html
>
> It is not guaranteed to be called, but then again, neither is any custom
> code you write to perform the same functionality.
>
>
> Thank you for your time!
>
> +++++++++++++++++++++
> Jeff Maass <ma...@gmail.com>
> linkedin.com/in/jeffmaass
> stackoverflow.com/users/373418/maassql
> +++++++++++++++++++++
>
>
> On Wed, May 13, 2015 at 6:23 AM, Richards Peter <hb...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am using storm-0.8.2 in my system. My project, based on storm, opens
>> connections to database/cache-server from spout/bolts. I need some of the
>> connections to remain open till the topologies are active. However I would
>> like to release these open connections from spout/bolt code when I kill my
>> topology.
>>
>> Could you please tell me how the resources(cache-server or database
>> connections) in bolts/spouts can be released using storm 0.9.3 and higher
>> versions?
>>
>> I am planning to upgrade storm, used in my project, to storm-0.9.4 for
>> various reasons. If I can release resources (cache-server or database
>> connections) from spout and bolt, it would another good reason for storm
>> upgrade.
>>
>> Thanks,
>> Richards Peter.
>>
>
>
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by Richards Peter <hb...@gmail.com>.
Thank you all for sharing your thoughts/comments.
Richards Peter.
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
Hi,
in my experience, the call to cleanup() is very unreliable... When
killing a topology, the worker process is killed via "kill -9" command
what is very harsh and cleanup() is not called most times (in my
experience).
If you want to make sure that DB connections are closed, I would follow
Nathan's suggestions to sent a special notification tuple through the
topology that is triggered by "deactivate".
-Matthias
On 05/13/2015 03:20 PM, Jeffery Maass wrote:
> Bolts which implement IBolt have a method called cleanup() which is
> called by the Storm framework.
>
> https://storm.apache.org/apidocs/backtype/storm/task/IBolt.html
>
> It is not guaranteed to be called, but then again, neither is any custom
> code you write to perform the same functionality.
>
>
> Thank you for your time!
>
> +++++++++++++++++++++
> Jeff Maass <ma...@gmail.com>
> linkedin.com/in/jeffmaass <http://linkedin.com/in/jeffmaass>
> stackoverflow.com/users/373418/maassql
> <http://stackoverflow.com/users/373418/maassql>
> +++++++++++++++++++++
>
>
> On Wed, May 13, 2015 at 6:23 AM, Richards Peter <hbkrichards@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am using storm-0.8.2 in my system. My project, based on storm,
> opens connections to database/cache-server from spout/bolts. I need
> some of the connections to remain open till the topologies are
> active. However I would like to release these open connections from
> spout/bolt code when I kill my topology.
>
> Could you please tell me how the resources(cache-server or database
> connections) in bolts/spouts can be released using storm 0.9.3 and
> higher versions?
>
> I am planning to upgrade storm, used in my project, to storm-0.9.4
> for various reasons. If I can release resources (cache-server or
> database connections) from spout and bolt, it would another good
> reason for storm upgrade.
>
> Thanks,
> Richards Peter.
>
>
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by Jeffery Maass <ma...@gmail.com>.
Bolts which implement IBolt have a method called cleanup() which is called
by the Storm framework.
https://storm.apache.org/apidocs/backtype/storm/task/IBolt.html
It is not guaranteed to be called, but then again, neither is any custom
code you write to perform the same functionality.
Thank you for your time!
+++++++++++++++++++++
Jeff Maass <ma...@gmail.com>
linkedin.com/in/jeffmaass
stackoverflow.com/users/373418/maassql
+++++++++++++++++++++
On Wed, May 13, 2015 at 6:23 AM, Richards Peter <hb...@gmail.com>
wrote:
> Hi,
>
> I am using storm-0.8.2 in my system. My project, based on storm, opens
> connections to database/cache-server from spout/bolts. I need some of the
> connections to remain open till the topologies are active. However I would
> like to release these open connections from spout/bolt code when I kill my
> topology.
>
> Could you please tell me how the resources(cache-server or database
> connections) in bolts/spouts can be released using storm 0.9.3 and higher
> versions?
>
> I am planning to upgrade storm, used in my project, to storm-0.9.4 for
> various reasons. If I can release resources (cache-server or database
> connections) from spout and bolt, it would another good reason for storm
> upgrade.
>
> Thanks,
> Richards Peter.
>
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by Richards Peter <hb...@gmail.com>.
Thanks Nathan. I will check this option.
I was also thinking whether we can have a notification mechanism for this
event in storm, the same way Hooks can notify spouts/bolts about
acknowledgement and failure of tuples. I do not know the feasibility of
implementing such a mechanism. I think it will be really useful to have
such a notification mechanism.
Richards Peter.
Re: Cleanup method not called for the BaseBasicBolt when the topology
is killed
Posted by Nathan Leung <nc...@gmail.com>.
The only way I know of is to deactivate the topology and have the spout
(our some other special spout) send a broadcast message on deactivation
that will trigger connection cleanup. If you ever want to reactivate you
will need to broadcast again when the spout is activated.
On May 13, 2015 7:23 AM, "Richards Peter" <hb...@gmail.com> wrote:
> Hi,
>
> I am using storm-0.8.2 in my system. My project, based on storm, opens
> connections to database/cache-server from spout/bolts. I need some of the
> connections to remain open till the topologies are active. However I would
> like to release these open connections from spout/bolt code when I kill my
> topology.
>
> Could you please tell me how the resources(cache-server or database
> connections) in bolts/spouts can be released using storm 0.9.3 and higher
> versions?
>
> I am planning to upgrade storm, used in my project, to storm-0.9.4 for
> various reasons. If I can release resources (cache-server or database
> connections) from spout and bolt, it would another good reason for storm
> upgrade.
>
> Thanks,
> Richards Peter.
>