You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Eremikhin Alexey <a....@corp.badoo.com> on 2014/04/17 12:28:35 UTC
log4j.appender.DRFA.MaxBackupIndex is not it nonsense!?
Hi everyone!
I've started looking about logs retention on Hadoop and noticed
interesting option in default Hadoop log4j.properties configuration
# 30-day backup
# log4j.appender.DRFA.MaxBackupIndex=30
I've enabled it and it produced no effect on existing files even after
rotation happened. That caused me start reading source code of log4j.
There is a base class FileAppender which has 2 children
RollingFileAppender(RFA) and DailyRollingFileAppender(DRFA).
RFA enumerates files ascending and on each rotation increases log file
id from 1 up to MaxBackupIndex and removes last file.
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html
DRFA uses date to distinct logs files but has no retention techniques
inside.
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
That means MaxBackupIndex can't have any effect on DRFA at all. And that
means property name log4j.appender.DRFA.MaxBackupIndex has no sense and
no effect.
You can easily google this parameter and find plenty complaints it does
not work.
Yes there are some projects about adding retention to DRFA but these
classes have different names.
http://wiki.apache.org/logging-log4j/DailyRollingFileAppender -
DailyMaxRollingFileAppender
You can find this parameter even in last stable distribution.
http://www.eu.apache.org/dist/hadoop/common/stable/hadoop-2.2.0.tar.gz
file
/hadoop-2.2.0/etc/hadoop/log4.properties
What I'm doing wrong? 8-\
Re: log4j.appender.DRFA.MaxBackupIndex is not it nonsense!?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Hi Eremikhin,
Thank you for the detailed information!
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
Since the comment is confusing to the user,
I think it should be removed.
I've filed a jira to track this issue.
https://issues.apache.org/jira/browse/HADOOP-10525
Thanks,
Akira
(2014/04/17 19:28), Eremikhin Alexey wrote:
> Hi everyone!
> I've started looking about logs retention on Hadoop and noticed
> interesting option in default Hadoop log4j.properties configuration
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
>
> I've enabled it and it produced no effect on existing files even after
> rotation happened. That caused me start reading source code of log4j.
> There is a base class FileAppender which has 2 children
> RollingFileAppender(RFA) and DailyRollingFileAppender(DRFA).
>
> RFA enumerates files ascending and on each rotation increases log file
> id from 1 up to MaxBackupIndex and removes last file.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html
>
> DRFA uses date to distinct logs files but has no retention techniques
> inside.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
>
> That means MaxBackupIndex can't have any effect on DRFA at all. And that
> means property name log4j.appender.DRFA.MaxBackupIndex has no sense and
> no effect.
> You can easily google this parameter and find plenty complaints it does
> not work.
>
> Yes there are some projects about adding retention to DRFA but these
> classes have different names.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender -
> DailyMaxRollingFileAppender
>
>
> You can find this parameter even in last stable distribution.
> http://www.eu.apache.org/dist/hadoop/common/stable/hadoop-2.2.0.tar.gz
> file
> /hadoop-2.2.0/etc/hadoop/log4.properties
>
>
> What I'm doing wrong? 8-\
Re: log4j.appender.DRFA.MaxBackupIndex is not it nonsense!?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Hi Eremikhin,
Thank you for the detailed information!
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
Since the comment is confusing to the user,
I think it should be removed.
I've filed a jira to track this issue.
https://issues.apache.org/jira/browse/HADOOP-10525
Thanks,
Akira
(2014/04/17 19:28), Eremikhin Alexey wrote:
> Hi everyone!
> I've started looking about logs retention on Hadoop and noticed
> interesting option in default Hadoop log4j.properties configuration
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
>
> I've enabled it and it produced no effect on existing files even after
> rotation happened. That caused me start reading source code of log4j.
> There is a base class FileAppender which has 2 children
> RollingFileAppender(RFA) and DailyRollingFileAppender(DRFA).
>
> RFA enumerates files ascending and on each rotation increases log file
> id from 1 up to MaxBackupIndex and removes last file.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html
>
> DRFA uses date to distinct logs files but has no retention techniques
> inside.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
>
> That means MaxBackupIndex can't have any effect on DRFA at all. And that
> means property name log4j.appender.DRFA.MaxBackupIndex has no sense and
> no effect.
> You can easily google this parameter and find plenty complaints it does
> not work.
>
> Yes there are some projects about adding retention to DRFA but these
> classes have different names.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender -
> DailyMaxRollingFileAppender
>
>
> You can find this parameter even in last stable distribution.
> http://www.eu.apache.org/dist/hadoop/common/stable/hadoop-2.2.0.tar.gz
> file
> /hadoop-2.2.0/etc/hadoop/log4.properties
>
>
> What I'm doing wrong? 8-\
Re: log4j.appender.DRFA.MaxBackupIndex is not it nonsense!?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Hi Eremikhin,
Thank you for the detailed information!
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
Since the comment is confusing to the user,
I think it should be removed.
I've filed a jira to track this issue.
https://issues.apache.org/jira/browse/HADOOP-10525
Thanks,
Akira
(2014/04/17 19:28), Eremikhin Alexey wrote:
> Hi everyone!
> I've started looking about logs retention on Hadoop and noticed
> interesting option in default Hadoop log4j.properties configuration
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
>
> I've enabled it and it produced no effect on existing files even after
> rotation happened. That caused me start reading source code of log4j.
> There is a base class FileAppender which has 2 children
> RollingFileAppender(RFA) and DailyRollingFileAppender(DRFA).
>
> RFA enumerates files ascending and on each rotation increases log file
> id from 1 up to MaxBackupIndex and removes last file.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html
>
> DRFA uses date to distinct logs files but has no retention techniques
> inside.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
>
> That means MaxBackupIndex can't have any effect on DRFA at all. And that
> means property name log4j.appender.DRFA.MaxBackupIndex has no sense and
> no effect.
> You can easily google this parameter and find plenty complaints it does
> not work.
>
> Yes there are some projects about adding retention to DRFA but these
> classes have different names.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender -
> DailyMaxRollingFileAppender
>
>
> You can find this parameter even in last stable distribution.
> http://www.eu.apache.org/dist/hadoop/common/stable/hadoop-2.2.0.tar.gz
> file
> /hadoop-2.2.0/etc/hadoop/log4.properties
>
>
> What I'm doing wrong? 8-\
Re: log4j.appender.DRFA.MaxBackupIndex is not it nonsense!?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Hi Eremikhin,
Thank you for the detailed information!
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
Since the comment is confusing to the user,
I think it should be removed.
I've filed a jira to track this issue.
https://issues.apache.org/jira/browse/HADOOP-10525
Thanks,
Akira
(2014/04/17 19:28), Eremikhin Alexey wrote:
> Hi everyone!
> I've started looking about logs retention on Hadoop and noticed
> interesting option in default Hadoop log4j.properties configuration
> # 30-day backup
> # log4j.appender.DRFA.MaxBackupIndex=30
>
> I've enabled it and it produced no effect on existing files even after
> rotation happened. That caused me start reading source code of log4j.
> There is a base class FileAppender which has 2 children
> RollingFileAppender(RFA) and DailyRollingFileAppender(DRFA).
>
> RFA enumerates files ascending and on each rotation increases log file
> id from 1 up to MaxBackupIndex and removes last file.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html
>
> DRFA uses date to distinct logs files but has no retention techniques
> inside.
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
>
> That means MaxBackupIndex can't have any effect on DRFA at all. And that
> means property name log4j.appender.DRFA.MaxBackupIndex has no sense and
> no effect.
> You can easily google this parameter and find plenty complaints it does
> not work.
>
> Yes there are some projects about adding retention to DRFA but these
> classes have different names.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender -
> DailyMaxRollingFileAppender
>
>
> You can find this parameter even in last stable distribution.
> http://www.eu.apache.org/dist/hadoop/common/stable/hadoop-2.2.0.tar.gz
> file
> /hadoop-2.2.0/etc/hadoop/log4.properties
>
>
> What I'm doing wrong? 8-\