You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Jonathan Aquilina <ja...@eagleeyet.net> on 2015/02/24 10:06:53 UTC
changing log verbosity
How does one go about changing the log verbosity in hadoop? What
configuration file should I be looking at?
--
Regards,
Jonathan Aquilina
Founder Eagle Eye T
Re: changing log verbosity
Posted by Vikas Parashar <pa...@gmail.com>.
Please look into log4j.properties
Rg:-
Vicky
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Ram Kumar <ra...@gmail.com>.
Hi Jonathan,
For Audit Log you can look log4.properties file. By default, the
log4j.properties file has the log threshold set to WARN. By setting this
level to INFO, audit logging can be turned on. The following snippet shows
the log4j.properties configuration when HDFS and MapReduce audit logs
are turned on.
#
# hdfs audit logging
#
hdfs.audit.logger=INFO,NullAppender
hdfs.audit.log.maxfilesize=256MB
hdfs.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger}
log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false
log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=${hdfs.audit.log.maxfilesize}
log4j.appender.RFAAUDIT.MaxBackupIndex=${hdfs.audit.log.maxbackupindex}
#
# mapred audit logging
#
mapred.audit.logger=INFO,NullAppender
mapred.audit.log.maxfilesize=256MB
mapred.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.mapred.AuditLogger=${mapred.audit.logger}
log4j.additivity.org.apache.hadoop.mapred.AuditLogger=false
log4j.appender.MRAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.MRAUDIT.File=${hadoop.log.dir}/mapred-audit.log
log4j.appender.MRAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.MRAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.MRAUDIT.MaxFileSize=${mapred.audit.log.maxfilesize}
log4j.appender.MRAUDIT.MaxBackupIndex=${mapred.audit.log.maxbackupindex}
Regards,
Ramkumar Bashyam
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Ram Kumar <ra...@gmail.com>.
Hi Jonathan,
For Audit Log you can look log4.properties file. By default, the
log4j.properties file has the log threshold set to WARN. By setting this
level to INFO, audit logging can be turned on. The following snippet shows
the log4j.properties configuration when HDFS and MapReduce audit logs
are turned on.
#
# hdfs audit logging
#
hdfs.audit.logger=INFO,NullAppender
hdfs.audit.log.maxfilesize=256MB
hdfs.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger}
log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false
log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=${hdfs.audit.log.maxfilesize}
log4j.appender.RFAAUDIT.MaxBackupIndex=${hdfs.audit.log.maxbackupindex}
#
# mapred audit logging
#
mapred.audit.logger=INFO,NullAppender
mapred.audit.log.maxfilesize=256MB
mapred.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.mapred.AuditLogger=${mapred.audit.logger}
log4j.additivity.org.apache.hadoop.mapred.AuditLogger=false
log4j.appender.MRAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.MRAUDIT.File=${hadoop.log.dir}/mapred-audit.log
log4j.appender.MRAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.MRAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.MRAUDIT.MaxFileSize=${mapred.audit.log.maxfilesize}
log4j.appender.MRAUDIT.MaxBackupIndex=${mapred.audit.log.maxbackupindex}
Regards,
Ramkumar Bashyam
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Vikas Parashar <pa...@gmail.com>.
Please look into log4j.properties
Rg:-
Vicky
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Vikas Parashar <pa...@gmail.com>.
Please look into log4j.properties
Rg:-
Vicky
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Ram Kumar <ra...@gmail.com>.
Hi Jonathan,
For Audit Log you can look log4.properties file. By default, the
log4j.properties file has the log threshold set to WARN. By setting this
level to INFO, audit logging can be turned on. The following snippet shows
the log4j.properties configuration when HDFS and MapReduce audit logs
are turned on.
#
# hdfs audit logging
#
hdfs.audit.logger=INFO,NullAppender
hdfs.audit.log.maxfilesize=256MB
hdfs.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger}
log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false
log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=${hdfs.audit.log.maxfilesize}
log4j.appender.RFAAUDIT.MaxBackupIndex=${hdfs.audit.log.maxbackupindex}
#
# mapred audit logging
#
mapred.audit.logger=INFO,NullAppender
mapred.audit.log.maxfilesize=256MB
mapred.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.mapred.AuditLogger=${mapred.audit.logger}
log4j.additivity.org.apache.hadoop.mapred.AuditLogger=false
log4j.appender.MRAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.MRAUDIT.File=${hadoop.log.dir}/mapred-audit.log
log4j.appender.MRAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.MRAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.MRAUDIT.MaxFileSize=${mapred.audit.log.maxfilesize}
log4j.appender.MRAUDIT.MaxBackupIndex=${mapred.audit.log.maxbackupindex}
Regards,
Ramkumar Bashyam
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Vikas Parashar <pa...@gmail.com>.
Please look into log4j.properties
Rg:-
Vicky
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>
Re: changing log verbosity
Posted by Ram Kumar <ra...@gmail.com>.
Hi Jonathan,
For Audit Log you can look log4.properties file. By default, the
log4j.properties file has the log threshold set to WARN. By setting this
level to INFO, audit logging can be turned on. The following snippet shows
the log4j.properties configuration when HDFS and MapReduce audit logs
are turned on.
#
# hdfs audit logging
#
hdfs.audit.logger=INFO,NullAppender
hdfs.audit.log.maxfilesize=256MB
hdfs.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger}
log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false
log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=${hdfs.audit.log.maxfilesize}
log4j.appender.RFAAUDIT.MaxBackupIndex=${hdfs.audit.log.maxbackupindex}
#
# mapred audit logging
#
mapred.audit.logger=INFO,NullAppender
mapred.audit.log.maxfilesize=256MB
mapred.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.mapred.AuditLogger=${mapred.audit.logger}
log4j.additivity.org.apache.hadoop.mapred.AuditLogger=false
log4j.appender.MRAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.MRAUDIT.File=${hadoop.log.dir}/mapred-audit.log
log4j.appender.MRAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.MRAUDIT.layout.ConversionPattern=%d{ISO8601} %p%c{2}: %m%n
log4j.appender.MRAUDIT.MaxFileSize=${mapred.audit.log.maxfilesize}
log4j.appender.MRAUDIT.MaxBackupIndex=${mapred.audit.log.maxbackupindex}
Regards,
Ramkumar Bashyam
On Tue, Feb 24, 2015 at 2:36 PM, Jonathan Aquilina <ja...@eagleeyet.net>
wrote:
> How does one go about changing the log verbosity in hadoop? What
> configuration file should I be looking at?
>
>
>
> --
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
>
>