You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Deepak Gupta <de...@gmail.com> on 2017/09/06 11:19:03 UTC
Query regarding old storm deletion
Hi Team,
Please advise how can I enable old storm log deletion.
Regards
Deepak Gupta
Re: Query regarding old storm deletion
Posted by Stig Rohde Døssing <sr...@apache.org>.
I believe you only need to update storm.yaml on the worker machines, but
deploying the same storm.yaml to both the Nimbus host and the workers is
easier IMO. That way you don't have to worry about whether a given property
is being read by Nimbus, the supervisor, logviewer, UI etc.
As far as I know it should have no effect on your workers if you start
logviewer.
The bug I linked is a very rare occurrence if your Zookeeper connection is
stable. Storm will likely work fine most of the time, but the bug may cause
Storm to fail to restart crashed executor threads, which can cause your
topologies to stop processing.
2017-09-06 19:49 GMT+02:00 Deepak Gupta <de...@gmail.com>:
> Hi
>
> Thanks for providing valuable inputs. One more query: Do we need to
> override the required parameters only in yaml of workers node not in nimbus?
>
> As you pointed out serious bug in 0.10.0 version. Will it affect workers
> execution if we starting logviewer daemon? As of now, this version of
> storm is working fine. But still I will propose your suggest to my
> management.
>
> Thanks
> Deepak
>
>
> On 06-Sep-2017 8:00 PM, "Stig Rohde Døssing" <sr...@apache.org> wrote:
>
> This may be a dumb question, but are you running the logviewer daemon
> ("storm logviewer")?
>
> The default parameter configuration can be found at
> https://github.com/apache/storm/blob/v0.10.0/conf/defaults.yaml, this
> file is also present in your storm-core.jar. Unless you override them in
> storm.yaml, the settings in defaults.yaml are also applied to your cluster.
>
> Also you may want to upgrade to 0.10.2 (or later if you like), 0.10.0 has
> a fairly serious bug that can cause your workers to stop functioning
> https://issues.apache.org/jira/browse/STORM-1750
>
> 2017-09-06 13:24 GMT+02:00 Deepak Gupta <de...@gmail.com>:
>
>> Hi Team,
>>
>> We are using storm version 0.10.0 in both production and preprod
>> environments. Currently we are facing an issue related to deletion of old
>> storm logs. We have found system is not deleting the log older than 7 days.
>> I checked in storm.yaml and found that none of logviewer.cleanup.age.mins
>> and logviewer.cleanup.interval.secs have been setup but still I can see
>> logviewer.cleanup.age.mins = 10080 in storm UI.
>>
>>
>>
>> Could you please advise the solution so that we can implement the 7 days
>> log retention policy? Do we need to configure logviewer.cleanup.age.mins
>> and logviewer.cleanup.interval.secs in yaml explicitly or any other
>> steps to be considered to fulfill the objective.
>>
>>
>> Thanks
>>
>> Deepak
>>
>> Regards
>>
>> Deepak Gupta
>> Mobile: +91-8800628111 <+91%2088006%2028111>
>>
>> On Wed, Sep 6, 2017 at 4:49 PM, Deepak Gupta <
>> deepak.technoexperts@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>> Please advise how can I enable old storm log deletion.
>>>
>>> Regards
>>>
>>> Deepak Gupta
>>>
>>>
>>
>
>
>
Re: Query regarding old storm deletion
Posted by Deepak Gupta <de...@gmail.com>.
Hi
Thanks for providing valuable inputs. One more query: Do we need to
override the required parameters only in yaml of workers node not in nimbus?
As you pointed out serious bug in 0.10.0 version. Will it affect workers
execution if we starting logviewer daemon? As of now, this version of
storm is working fine. But still I will propose your suggest to my
management.
Thanks
Deepak
On 06-Sep-2017 8:00 PM, "Stig Rohde Døssing" <sr...@apache.org> wrote:
This may be a dumb question, but are you running the logviewer daemon
("storm logviewer")?
The default parameter configuration can be found at
https://github.com/apache/storm/blob/v0.10.0/conf/defaults.yaml, this file
is also present in your storm-core.jar. Unless you override them in
storm.yaml, the settings in defaults.yaml are also applied to your cluster.
Also you may want to upgrade to 0.10.2 (or later if you like), 0.10.0 has a
fairly serious bug that can cause your workers to stop functioning
https://issues.apache.org/jira/browse/STORM-1750
2017-09-06 13:24 GMT+02:00 Deepak Gupta <de...@gmail.com>:
> Hi Team,
>
> We are using storm version 0.10.0 in both production and preprod
> environments. Currently we are facing an issue related to deletion of old
> storm logs. We have found system is not deleting the log older than 7 days.
> I checked in storm.yaml and found that none of logviewer.cleanup.age.mins
> and logviewer.cleanup.interval.secs have been setup but still I can see
> logviewer.cleanup.age.mins = 10080 in storm UI.
>
>
>
> Could you please advise the solution so that we can implement the 7 days
> log retention policy? Do we need to configure logviewer.cleanup.age.mins
> and logviewer.cleanup.interval.secs in yaml explicitly or any other steps
> to be considered to fulfill the objective.
>
>
> Thanks
>
> Deepak
>
> Regards
>
> Deepak Gupta
> Mobile: +91-8800628111 <+91%2088006%2028111>
>
> On Wed, Sep 6, 2017 at 4:49 PM, Deepak Gupta <
> deepak.technoexperts@gmail.com> wrote:
>
>> Hi Team,
>>
>> Please advise how can I enable old storm log deletion.
>>
>> Regards
>>
>> Deepak Gupta
>>
>>
>
Re: Query regarding old storm deletion
Posted by Stig Rohde Døssing <sr...@apache.org>.
This may be a dumb question, but are you running the logviewer daemon
("storm logviewer")?
The default parameter configuration can be found at
https://github.com/apache/storm/blob/v0.10.0/conf/defaults.yaml, this file
is also present in your storm-core.jar. Unless you override them in
storm.yaml, the settings in defaults.yaml are also applied to your cluster.
Also you may want to upgrade to 0.10.2 (or later if you like), 0.10.0 has a
fairly serious bug that can cause your workers to stop functioning
https://issues.apache.org/jira/browse/STORM-1750
2017-09-06 13:24 GMT+02:00 Deepak Gupta <de...@gmail.com>:
> Hi Team,
>
> We are using storm version 0.10.0 in both production and preprod
> environments. Currently we are facing an issue related to deletion of old
> storm logs. We have found system is not deleting the log older than 7 days.
> I checked in storm.yaml and found that none of logviewer.cleanup.age.mins
> and logviewer.cleanup.interval.secs have been setup but still I can see
> logviewer.cleanup.age.mins = 10080 in storm UI.
>
>
>
> Could you please advise the solution so that we can implement the 7 days
> log retention policy? Do we need to configure logviewer.cleanup.age.mins
> and logviewer.cleanup.interval.secs in yaml explicitly or any other steps
> to be considered to fulfill the objective.
>
>
> Thanks
>
> Deepak
>
> Regards
>
> Deepak Gupta
> Mobile: +91-8800628111 <+91%2088006%2028111>
>
> On Wed, Sep 6, 2017 at 4:49 PM, Deepak Gupta <deepak.technoexperts@gmail.
> com> wrote:
>
>> Hi Team,
>>
>> Please advise how can I enable old storm log deletion.
>>
>> Regards
>>
>> Deepak Gupta
>>
>>
>
Re: Query regarding old storm deletion
Posted by Deepak Gupta <de...@gmail.com>.
Hi Team,
We are using storm version 0.10.0 in both production and preprod
environments. Currently we are facing an issue related to deletion of old
storm logs. We have found system is not deleting the log older than 7 days.
I checked in storm.yaml and found that none of logviewer.cleanup.age.mins
and logviewer.cleanup.interval.secs have been setup but still I can see
logviewer.cleanup.age.mins = 10080 in storm UI.
Could you please advise the solution so that we can implement the 7 days
log retention policy? Do we need to configure logviewer.cleanup.age.mins
and logviewer.cleanup.interval.secs in yaml explicitly or any other steps
to be considered to fulfill the objective.
Thanks
Deepak
Regards
Deepak Gupta
Mobile: +91-8800628111
On Wed, Sep 6, 2017 at 4:49 PM, Deepak Gupta <deepak.technoexperts@gmail.com
> wrote:
> Hi Team,
>
> Please advise how can I enable old storm log deletion.
>
> Regards
>
> Deepak Gupta
>
>