You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by clay teahouse <cl...@gmail.com> on 2015/03/03 23:39:34 UTC
cleaning up storm generated files
Hi all,
How do I remove storm generated system files programmatically without
stepping over the files that I shouldn't be deleting? I need this to
cleanup the files left behind from the aborted topologies. I tried fuser
and lsof, but with no luck. For example, fuser shows stormconf.ser and
stormcode.ser not in use even in cases where these files are being used.
thanks
Clay
Re: cleaning up storm generated files
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
In local mode the cleanup method is supposed to be called on shutdown. The cleanup method is supposed to be called in regular mode on newer versions of storm, so long as it does not take too long to complete. The files that the supervisor downloads from nimbus stormconf.ser etc. how it gets them is different enough in local mode that I don't know for sure how/if they are cleaned up there.
- Bobby
On Friday, March 6, 2015 7:54 AM, clay teahouse <cl...@gmail.com> wrote:
Hi Bobby,Is this the case in local mode as well? If yes, it doesn't seem to be working. If no, how do I go about removing these files? I read a couple of places that the cleanup method is for that, but it doesn't seem that this method is called when a topology is aborted.thanksClay
On Wed, Mar 4, 2015 at 12:42 PM, Bobby Evans <ev...@yahoo-inc.com.invalid> wrote:
Even if the supervisor is killed, when it comes back up it should do the cleanup. The way it knows is that it will download from zookeeper the list of workers that it is supposed to be running, and then compare that with what it knows about. If a worker is detected that it does not know about it will kill the worker, and clean up the files. This happens when you see a log message like "Shutting down and clearing state for id..." in the supervisor. It is possible that we got an error and the local state of the supervisor got off.
The only way to know what is not needed is to look at zookeeper and do what the supervisor does to determine what should be running. - Bobby
On Wednesday, March 4, 2015 11:46 AM, clay teahouse <cl...@gmail.com> wrote:
Is this also the case if you kill a topology or the machine goes down? Or
do I need to write a jvm hook to handle this?
But in any case, is there a way to know which files are in use by storm (as
I said fuser and lsof are not of no help here).
thanks.
On Wed, Mar 4, 2015 at 11:34 AM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> Storm is supposed to be cleaning them up itself. The supervisor is
> supposed to know when a topology is no longer needed and remove them after
> a specific timeout. If this is not happening for you there may be a bug in
> the code. Could you file a JIRA and post the JIRA number here. If you
> could include the version number of storm and any logs you have from the
> supervisor that would be helpful. I cannot guarantee how quickly the
> community can get to this though.
> - Bobby
>
>
> On Tuesday, March 3, 2015 4:39 PM, clay teahouse <
> clayteahouse@gmail.com> wrote:
>
>
> Hi all,
> How do I remove storm generated system files programmatically without
> stepping over the files that I shouldn't be deleting? I need this to
> cleanup the files left behind from the aborted topologies. I tried fuser
> and lsof, but with no luck. For example, fuser shows stormconf.ser and
> stormcode.ser not in use even in cases where these files are being used.
> thanksClay
>
>
>
>
Re: cleaning up storm generated files
Posted by clay teahouse <cl...@gmail.com>.
Hi Bobby,
Is this the case in local mode as well? If yes, it doesn't seem to be
working. If no, how do I go about removing these files? I read a couple of
places that the cleanup method is for that, but it doesn't seem that this
method is called when a topology is aborted.
thanks
Clay
On Wed, Mar 4, 2015 at 12:42 PM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> Even if the supervisor is killed, when it comes back up it should do the
> cleanup. The way it knows is that it will download from zookeeper the list
> of workers that it is supposed to be running, and then compare that with
> what it knows about. If a worker is detected that it does not know about
> it will kill the worker, and clean up the files. This happens when you see
> a log message like "Shutting down and clearing state for id..." in the
> supervisor. It is possible that we got an error and the local state of the
> supervisor got off.
>
> The only way to know what is not needed is to look at zookeeper and do
> what the supervisor does to determine what should be running. - Bobby
>
>
> On Wednesday, March 4, 2015 11:46 AM, clay teahouse <
> clayteahouse@gmail.com> wrote:
>
>
> Is this also the case if you kill a topology or the machine goes down? Or
> do I need to write a jvm hook to handle this?
> But in any case, is there a way to know which files are in use by storm (as
> I said fuser and lsof are not of no help here).
>
> thanks.
>
> On Wed, Mar 4, 2015 at 11:34 AM, Bobby Evans <ev...@yahoo-inc.com.invalid>
> wrote:
>
> > Storm is supposed to be cleaning them up itself. The supervisor is
> > supposed to know when a topology is no longer needed and remove them
> after
> > a specific timeout. If this is not happening for you there may be a bug
> in
> > the code. Could you file a JIRA and post the JIRA number here. If you
> > could include the version number of storm and any logs you have from the
> > supervisor that would be helpful. I cannot guarantee how quickly the
> > community can get to this though.
> > - Bobby
> >
> >
> > On Tuesday, March 3, 2015 4:39 PM, clay teahouse <
> > clayteahouse@gmail.com> wrote:
> >
> >
> > Hi all,
> > How do I remove storm generated system files programmatically without
> > stepping over the files that I shouldn't be deleting? I need this to
> > cleanup the files left behind from the aborted topologies. I tried fuser
> > and lsof, but with no luck. For example, fuser shows stormconf.ser and
> > stormcode.ser not in use even in cases where these files are being used.
> > thanksClay
> >
> >
> >
> >
>
>
>
>
Re: cleaning up storm generated files
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
Even if the supervisor is killed, when it comes back up it should do the cleanup. The way it knows is that it will download from zookeeper the list of workers that it is supposed to be running, and then compare that with what it knows about. If a worker is detected that it does not know about it will kill the worker, and clean up the files. This happens when you see a log message like "Shutting down and clearing state for id..." in the supervisor. It is possible that we got an error and the local state of the supervisor got off.
The only way to know what is not needed is to look at zookeeper and do what the supervisor does to determine what should be running. - Bobby
On Wednesday, March 4, 2015 11:46 AM, clay teahouse <cl...@gmail.com> wrote:
Is this also the case if you kill a topology or the machine goes down? Or
do I need to write a jvm hook to handle this?
But in any case, is there a way to know which files are in use by storm (as
I said fuser and lsof are not of no help here).
thanks.
On Wed, Mar 4, 2015 at 11:34 AM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> Storm is supposed to be cleaning them up itself. The supervisor is
> supposed to know when a topology is no longer needed and remove them after
> a specific timeout. If this is not happening for you there may be a bug in
> the code. Could you file a JIRA and post the JIRA number here. If you
> could include the version number of storm and any logs you have from the
> supervisor that would be helpful. I cannot guarantee how quickly the
> community can get to this though.
> - Bobby
>
>
> On Tuesday, March 3, 2015 4:39 PM, clay teahouse <
> clayteahouse@gmail.com> wrote:
>
>
> Hi all,
> How do I remove storm generated system files programmatically without
> stepping over the files that I shouldn't be deleting? I need this to
> cleanup the files left behind from the aborted topologies. I tried fuser
> and lsof, but with no luck. For example, fuser shows stormconf.ser and
> stormcode.ser not in use even in cases where these files are being used.
> thanksClay
>
>
>
>
Re: cleaning up storm generated files
Posted by clay teahouse <cl...@gmail.com>.
Is this also the case if you kill a topology or the machine goes down? Or
do I need to write a jvm hook to handle this?
But in any case, is there a way to know which files are in use by storm (as
I said fuser and lsof are not of no help here).
thanks.
On Wed, Mar 4, 2015 at 11:34 AM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> Storm is supposed to be cleaning them up itself. The supervisor is
> supposed to know when a topology is no longer needed and remove them after
> a specific timeout. If this is not happening for you there may be a bug in
> the code. Could you file a JIRA and post the JIRA number here. If you
> could include the version number of storm and any logs you have from the
> supervisor that would be helpful. I cannot guarantee how quickly the
> community can get to this though.
> - Bobby
>
>
> On Tuesday, March 3, 2015 4:39 PM, clay teahouse <
> clayteahouse@gmail.com> wrote:
>
>
> Hi all,
> How do I remove storm generated system files programmatically without
> stepping over the files that I shouldn't be deleting? I need this to
> cleanup the files left behind from the aborted topologies. I tried fuser
> and lsof, but with no luck. For example, fuser shows stormconf.ser and
> stormcode.ser not in use even in cases where these files are being used.
> thanksClay
>
>
>
>
Re: cleaning up storm generated files
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
Storm is supposed to be cleaning them up itself. The supervisor is supposed to know when a topology is no longer needed and remove them after a specific timeout. If this is not happening for you there may be a bug in the code. Could you file a JIRA and post the JIRA number here. If you could include the version number of storm and any logs you have from the supervisor that would be helpful. I cannot guarantee how quickly the community can get to this though.
- Bobby
On Tuesday, March 3, 2015 4:39 PM, clay teahouse <cl...@gmail.com> wrote:
Hi all,
How do I remove storm generated system files programmatically without stepping over the files that I shouldn't be deleting? I need this to cleanup the files left behind from the aborted topologies. I tried fuser and lsof, but with no luck. For example, fuser shows stormconf.ser and stormcode.ser not in use even in cases where these files are being used.
thanksClay
Re: cleaning up storm generated files
Posted by Bobby Evans <ev...@yahoo-inc.com>.
Storm is supposed to be cleaning them up itself. The supervisor is supposed to know when a topology is no longer needed and remove them after a specific timeout. If this is not happening for you there may be a bug in the code. Could you file a JIRA and post the JIRA number here. If you could include the version number of storm and any logs you have from the supervisor that would be helpful. I cannot guarantee how quickly the community can get to this though.
- Bobby
On Tuesday, March 3, 2015 4:39 PM, clay teahouse <cl...@gmail.com> wrote:
Hi all,
How do I remove storm generated system files programmatically without stepping over the files that I shouldn't be deleting? I need this to cleanup the files left behind from the aborted topologies. I tried fuser and lsof, but with no luck. For example, fuser shows stormconf.ser and stormcode.ser not in use even in cases where these files are being used.
thanksClay