You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Chandra <ch...@rwth-aachen.de> on 2017/04/10 05:50:04 UTC
Max index limit in DefaultRolloverStrategy
Hi guys,
Something I noticed in my production, the maximum limit of index (i.e., the max #files to be rolled before an overwrite occurs) is 7 by default.
This is not mentioned anywhere in any of the documentation. I was only able to confirm this by actually looking into the source code [1]. Unfortunately, by the time we noticed there was huge data loss.
I wanted to confirm:
* Why is the default max index set to 7? I was naive to think that the default max index would be set to a large value (may be no max index at all, perhaps)
* Why wasn’t this mentioned anywhere in the documentation (if it is, can you point me to it. may be I wasn’t paying attention). If it happened to me, perhaps it might for others.
Best,
Chandra
[1] https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/DefaultRolloverStrategy.java#L126
Re: Max index limit in DefaultRolloverStrategy
Posted by Ralph Goers <ra...@dslextreme.com>.
FYI - the documentation for the web site is in the project at src/site/xdoc/manual. You can create a patch for that the same way you would patch any of the java source.
Ralph
> On Apr 18, 2017, at 10:20 AM, Chandra <ch...@rwth-aachen.de> wrote:
>
> Hi Gary,
>
> I’d love to, but this ticket is primarily to change the documentation and it’d have a minor amendments in the document on the website. I’m not exactly sure how to send a patch to this :-/
>
> 1)
> https://logging.apache.org/log4j/2.x/manual/appenders.html#DefaultRolloverStrategy
>
> Particularly, the section "DefaultRolloverStrategy Parameters” :
>
> From:
> max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers.
>
> To :
> max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers (The default value is 7).
>
> 2) Also, I’d also _recommend_ to update the Javadoc (this I can send a patch :) ) but, I think it’s not required.
>
> thanks,
> Chandra
>
> On 18 Apr 2017, 8:53 PM +0530, Gary Gregory <ga...@gmail.com>, wrote:
>> Patches welcome ;-)
>>
>> Gary
>>
>>
>> On Apr 18, 2017 1:18 AM, "Chandra" <chandra.tungathurthi@rwth-aachen.de
>> wrote:
>>
>> Hi guys, any update on this ticket?
>> https://issues.apache.org/jira/browse/LOG4J2-1877
>>
>>
>> thanks,
>> Chandra
>>
>> On 12 Apr 2017, 12:14 AM +0530, Chandra <ch...@rwth-aachen.de>,
>> wrote:
>>> Yes, that’d be great. 7 seems to too low for frequently filled logs.
>>> And also guys, please document it. I am raising a JIRA ticket in this
>> regards ( thanks Remko for the reminder).
>>> Also yes, I have set a default purge policy, this is 90 days in our use
>> case.
>>>
>>> Best,
>>> Chandra
>>>
>>> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <ra...@dslextreme.com>,
>> wrote:
>>>> I believe starting with 2.8.1 we no longer scan the directory so it is
>> much faster. We could probably raise the default limit.
>>>>
>>>> Ralph
>>>>
>>>>> On Apr 11, 2017, at 11:12 AM, Remko Popma <remko.popma@gmail.com
>> wrote:
>>>>>
>>>>> Sorry to hear that!
>>>>>
>>>>> You are right that the docs currently don't mention the default value
>> for
>>>>> the max attribute.
>>>>> We should fix the docs. Can I ask you to raise a Jira for this?
>>>>>
>>>>> As for why not a large default value, there is a performance impact:
>> all
>>>>> files within the range need to be renamed (bump version by one), and
>> the
>>>>> current logic scans the directory to look for files that match the
>> pattern.
>>>>>
>>>>> You may be interested in the custom Delete action which is more
>> efficient (
>>>>> https://logging.apache.org/log4j/2.x/manual/appenders.
>> html#CustomDeleteOnRollover
>>>>> ).
>>>>>
>>>>>
>>>>> On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
>>>>> chandra.tungathurthi@rwth-aachen.de> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> Something I noticed in my production, the maximum limit of index
>> (i.e.,
>>>>>> the max #files to be rolled before an overwrite occurs) is 7 by
>> default.
>>>>>> This is not mentioned anywhere in any of the documentation. I was
>> only
>>>>>> able to confirm this by actually looking into the source code [1].
>>>>>> Unfortunately, by the time we noticed there was huge data loss.
>>>>>>
>>>>>> I wanted to confirm:
>>>>>> * Why is the default max index set to 7? I was naive to think that
>> the
>>>>>> default max index would be set to a large value (may be no max
>> index at
>>>>>> all, perhaps)
>>>>>> * Why wasn’t this mentioned anywhere in the documentation (if it
>> is, can
>>>>>> you point me to it. may be I wasn’t paying attention). If it
>> happened to
>>>>>> me, perhaps it might for others.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Best,
>>>>>> Chandra
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/apache/logging-log4j2/blob/master/
>>>>>> log4j-core/src/main/java/org/apache/logging/log4j/core/
>> appender/rolling/
>>>>>> DefaultRolloverStrategy.java#L126
>>>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Max index limit in DefaultRolloverStrategy
Posted by Chandra <ch...@rwth-aachen.de>.
Hi Gary,
I’d love to, but this ticket is primarily to change the documentation and it’d have a minor amendments in the document on the website. I’m not exactly sure how to send a patch to this :-/
1)
https://logging.apache.org/log4j/2.x/manual/appenders.html#DefaultRolloverStrategy
Particularly, the section "DefaultRolloverStrategy Parameters” :
From:
max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers.
To :
max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers (The default value is 7).
2) Also, I’d also _recommend_ to update the Javadoc (this I can send a patch :) ) but, I think it’s not required.
thanks,
Chandra
On 18 Apr 2017, 8:53 PM +0530, Gary Gregory <ga...@gmail.com>, wrote:
> Patches welcome ;-)
>
> Gary
>
>
> On Apr 18, 2017 1:18 AM, "Chandra" <chandra.tungathurthi@rwth-aachen.de
> wrote:
>
> Hi guys, any update on this ticket?
> https://issues.apache.org/jira/browse/LOG4J2-1877
>
>
> thanks,
> Chandra
>
> On 12 Apr 2017, 12:14 AM +0530, Chandra <ch...@rwth-aachen.de>,
> wrote:
> > Yes, that’d be great. 7 seems to too low for frequently filled logs.
> > And also guys, please document it. I am raising a JIRA ticket in this
> regards ( thanks Remko for the reminder).
> > Also yes, I have set a default purge policy, this is 90 days in our use
> case.
> >
> > Best,
> > Chandra
> >
> > On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <ra...@dslextreme.com>,
> wrote:
> > > I believe starting with 2.8.1 we no longer scan the directory so it is
> much faster. We could probably raise the default limit.
> > >
> > > Ralph
> > >
> > > > On Apr 11, 2017, at 11:12 AM, Remko Popma <remko.popma@gmail.com
> wrote:
> > > >
> > > > Sorry to hear that!
> > > >
> > > > You are right that the docs currently don't mention the default value
> for
> > > > the max attribute.
> > > > We should fix the docs. Can I ask you to raise a Jira for this?
> > > >
> > > > As for why not a large default value, there is a performance impact:
> all
> > > > files within the range need to be renamed (bump version by one), and
> the
> > > > current logic scans the directory to look for files that match the
> pattern.
> > > >
> > > > You may be interested in the custom Delete action which is more
> efficient (
> > > > https://logging.apache.org/log4j/2.x/manual/appenders.
> html#CustomDeleteOnRollover
> > > > ).
> > > >
> > > >
> > > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > > chandra.tungathurthi@rwth-aachen.de> wrote:
> > > >
> > > > > Hi guys,
> > > > >
> > > > > Something I noticed in my production, the maximum limit of index
> (i.e.,
> > > > > the max #files to be rolled before an overwrite occurs) is 7 by
> default.
> > > > > This is not mentioned anywhere in any of the documentation. I was
> only
> > > > > able to confirm this by actually looking into the source code [1].
> > > > > Unfortunately, by the time we noticed there was huge data loss.
> > > > >
> > > > > I wanted to confirm:
> > > > > * Why is the default max index set to 7? I was naive to think that
> the
> > > > > default max index would be set to a large value (may be no max
> index at
> > > > > all, perhaps)
> > > > > * Why wasn’t this mentioned anywhere in the documentation (if it
> is, can
> > > > > you point me to it. may be I wasn’t paying attention). If it
> happened to
> > > > > me, perhaps it might for others.
> > > > >
> > > > >
> > > > >
> > > > > Best,
> > > > > Chandra
> > > > >
> > > > >
> > > > >
> > > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > > log4j-core/src/main/java/org/apache/logging/log4j/core/
> appender/rolling/
> > > > > DefaultRolloverStrategy.java#L126
> > > > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > > For additional commands, e-mail: log4j-user-help@logging.apache.org
> > >
Re: Max index limit in DefaultRolloverStrategy
Posted by Gary Gregory <ga...@gmail.com>.
Patches welcome ;-)
Gary
On Apr 18, 2017 1:18 AM, "Chandra" <ch...@rwth-aachen.de>
wrote:
Hi guys, any update on this ticket?
https://issues.apache.org/jira/browse/LOG4J2-1877
thanks,
Chandra
On 12 Apr 2017, 12:14 AM +0530, Chandra <ch...@rwth-aachen.de>,
wrote:
> Yes, that’d be great. 7 seems to too low for frequently filled logs.
> And also guys, please document it. I am raising a JIRA ticket in this
regards ( thanks Remko for the reminder).
> Also yes, I have set a default purge policy, this is 90 days in our use
case.
>
> Best,
> Chandra
>
> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <ra...@dslextreme.com>,
wrote:
> > I believe starting with 2.8.1 we no longer scan the directory so it is
much faster. We could probably raise the default limit.
> >
> > Ralph
> >
> > > On Apr 11, 2017, at 11:12 AM, Remko Popma <re...@gmail.com>
wrote:
> > >
> > > Sorry to hear that!
> > >
> > > You are right that the docs currently don't mention the default value
for
> > > the max attribute.
> > > We should fix the docs. Can I ask you to raise a Jira for this?
> > >
> > > As for why not a large default value, there is a performance impact:
all
> > > files within the range need to be renamed (bump version by one), and
the
> > > current logic scans the directory to look for files that match the
pattern.
> > >
> > > You may be interested in the custom Delete action which is more
efficient (
> > > https://logging.apache.org/log4j/2.x/manual/appenders.
html#CustomDeleteOnRollover
> > > ).
> > >
> > >
> > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > chandra.tungathurthi@rwth-aachen.de> wrote:
> > >
> > > > Hi guys,
> > > >
> > > > Something I noticed in my production, the maximum limit of index
(i.e.,
> > > > the max #files to be rolled before an overwrite occurs) is 7 by
default.
> > > > This is not mentioned anywhere in any of the documentation. I was
only
> > > > able to confirm this by actually looking into the source code [1].
> > > > Unfortunately, by the time we noticed there was huge data loss.
> > > >
> > > > I wanted to confirm:
> > > > * Why is the default max index set to 7? I was naive to think that
the
> > > > default max index would be set to a large value (may be no max
index at
> > > > all, perhaps)
> > > > * Why wasn’t this mentioned anywhere in the documentation (if it
is, can
> > > > you point me to it. may be I wasn’t paying attention). If it
happened to
> > > > me, perhaps it might for others.
> > > >
> > > >
> > > >
> > > > Best,
> > > > Chandra
> > > >
> > > >
> > > >
> > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > log4j-core/src/main/java/org/apache/logging/log4j/core/
appender/rolling/
> > > > DefaultRolloverStrategy.java#L126
> > > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
Re: Max index limit in DefaultRolloverStrategy
Posted by Chandra <ch...@rwth-aachen.de>.
Hi guys, any update on this ticket?
https://issues.apache.org/jira/browse/LOG4J2-1877
thanks,
Chandra
On 12 Apr 2017, 12:14 AM +0530, Chandra <ch...@rwth-aachen.de>, wrote:
> Yes, that’d be great. 7 seems to too low for frequently filled logs.
> And also guys, please document it. I am raising a JIRA ticket in this regards ( thanks Remko for the reminder).
> Also yes, I have set a default purge policy, this is 90 days in our use case.
>
> Best,
> Chandra
>
> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <ra...@dslextreme.com>, wrote:
> > I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.
> >
> > Ralph
> >
> > > On Apr 11, 2017, at 11:12 AM, Remko Popma <re...@gmail.com> wrote:
> > >
> > > Sorry to hear that!
> > >
> > > You are right that the docs currently don't mention the default value for
> > > the max attribute.
> > > We should fix the docs. Can I ask you to raise a Jira for this?
> > >
> > > As for why not a large default value, there is a performance impact: all
> > > files within the range need to be renamed (bump version by one), and the
> > > current logic scans the directory to look for files that match the pattern.
> > >
> > > You may be interested in the custom Delete action which is more efficient (
> > > https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> > > ).
> > >
> > >
> > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > chandra.tungathurthi@rwth-aachen.de> wrote:
> > >
> > > > Hi guys,
> > > >
> > > > Something I noticed in my production, the maximum limit of index (i.e.,
> > > > the max #files to be rolled before an overwrite occurs) is 7 by default.
> > > > This is not mentioned anywhere in any of the documentation. I was only
> > > > able to confirm this by actually looking into the source code [1].
> > > > Unfortunately, by the time we noticed there was huge data loss.
> > > >
> > > > I wanted to confirm:
> > > > * Why is the default max index set to 7? I was naive to think that the
> > > > default max index would be set to a large value (may be no max index at
> > > > all, perhaps)
> > > > * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> > > > you point me to it. may be I wasn’t paying attention). If it happened to
> > > > me, perhaps it might for others.
> > > >
> > > >
> > > >
> > > > Best,
> > > > Chandra
> > > >
> > > >
> > > >
> > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> > > > DefaultRolloverStrategy.java#L126
> > > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
Re: Max index limit in DefaultRolloverStrategy
Posted by Chandra <ch...@rwth-aachen.de>.
Yes, that’d be great. 7 seems to too low for frequently filled logs.
And also guys, please document it. I am raising a JIRA ticket in this regards ( thanks Remko for the reminder).
Also yes, I have set a default purge policy, this is 90 days in our use case.
Best,
Chandra
On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <ra...@dslextreme.com>, wrote:
> I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.
>
> Ralph
>
> > On Apr 11, 2017, at 11:12 AM, Remko Popma <re...@gmail.com> wrote:
> >
> > Sorry to hear that!
> >
> > You are right that the docs currently don't mention the default value for
> > the max attribute.
> > We should fix the docs. Can I ask you to raise a Jira for this?
> >
> > As for why not a large default value, there is a performance impact: all
> > files within the range need to be renamed (bump version by one), and the
> > current logic scans the directory to look for files that match the pattern.
> >
> > You may be interested in the custom Delete action which is more efficient (
> > https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> > ).
> >
> >
> > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > chandra.tungathurthi@rwth-aachen.de> wrote:
> >
> > > Hi guys,
> > >
> > > Something I noticed in my production, the maximum limit of index (i.e.,
> > > the max #files to be rolled before an overwrite occurs) is 7 by default.
> > > This is not mentioned anywhere in any of the documentation. I was only
> > > able to confirm this by actually looking into the source code [1].
> > > Unfortunately, by the time we noticed there was huge data loss.
> > >
> > > I wanted to confirm:
> > > * Why is the default max index set to 7? I was naive to think that the
> > > default max index would be set to a large value (may be no max index at
> > > all, perhaps)
> > > * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> > > you point me to it. may be I wasn’t paying attention). If it happened to
> > > me, perhaps it might for others.
> > >
> > >
> > >
> > > Best,
> > > Chandra
> > >
> > >
> > >
> > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> > > DefaultRolloverStrategy.java#L126
> > >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
Re: Max index limit in DefaultRolloverStrategy
Posted by Ralph Goers <ra...@dslextreme.com>.
I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.
Ralph
> On Apr 11, 2017, at 11:12 AM, Remko Popma <re...@gmail.com> wrote:
>
> Sorry to hear that!
>
> You are right that the docs currently don't mention the default value for
> the max attribute.
> We should fix the docs. Can I ask you to raise a Jira for this?
>
> As for why not a large default value, there is a performance impact: all
> files within the range need to be renamed (bump version by one), and the
> current logic scans the directory to look for files that match the pattern.
>
> You may be interested in the custom Delete action which is more efficient (
> https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> ).
>
>
> On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> chandra.tungathurthi@rwth-aachen.de> wrote:
>
>> Hi guys,
>>
>> Something I noticed in my production, the maximum limit of index (i.e.,
>> the max #files to be rolled before an overwrite occurs) is 7 by default.
>> This is not mentioned anywhere in any of the documentation. I was only
>> able to confirm this by actually looking into the source code [1].
>> Unfortunately, by the time we noticed there was huge data loss.
>>
>> I wanted to confirm:
>> * Why is the default max index set to 7? I was naive to think that the
>> default max index would be set to a large value (may be no max index at
>> all, perhaps)
>> * Why wasn’t this mentioned anywhere in the documentation (if it is, can
>> you point me to it. may be I wasn’t paying attention). If it happened to
>> me, perhaps it might for others.
>>
>>
>>
>> Best,
>> Chandra
>>
>>
>>
>> [1] https://github.com/apache/logging-log4j2/blob/master/
>> log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
>> DefaultRolloverStrategy.java#L126
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Max index limit in DefaultRolloverStrategy
Posted by Remko Popma <re...@gmail.com>.
Sorry to hear that!
You are right that the docs currently don't mention the default value for
the max attribute.
We should fix the docs. Can I ask you to raise a Jira for this?
As for why not a large default value, there is a performance impact: all
files within the range need to be renamed (bump version by one), and the
current logic scans the directory to look for files that match the pattern.
You may be interested in the custom Delete action which is more efficient (
https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
).
On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
chandra.tungathurthi@rwth-aachen.de> wrote:
> Hi guys,
>
> Something I noticed in my production, the maximum limit of index (i.e.,
> the max #files to be rolled before an overwrite occurs) is 7 by default.
> This is not mentioned anywhere in any of the documentation. I was only
> able to confirm this by actually looking into the source code [1].
> Unfortunately, by the time we noticed there was huge data loss.
>
> I wanted to confirm:
> * Why is the default max index set to 7? I was naive to think that the
> default max index would be set to a large value (may be no max index at
> all, perhaps)
> * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> you point me to it. may be I wasn’t paying attention). If it happened to
> me, perhaps it might for others.
>
>
>
> Best,
> Chandra
>
>
>
> [1] https://github.com/apache/logging-log4j2/blob/master/
> log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> DefaultRolloverStrategy.java#L126
>