Hadoop yarn tasklevel logging with log4j

[Sherif Akoush <sh...@gmail.com>]

Hi,

I have been trying to setup up Hadoop logging at the task level but
with no success so far. I have modified log4j.properties and set many
parameters to DEBUG level
(log4j.logger.org.apache.hadoop.mapred.Task=DEBUG
log4j.logger.org.apache.hadoop.mapred.MapTask=DEBUG
log4j.logger.org.apache.hadoop.mapred.ReduceTask=DEBUG

and log4j.logger.org.apache.hadoop = DEBUG )

but I cant see any LOG.info or LOG.debug messages from the tasks being
recorded. This is with yarn being enabled.

I am not sure what I have missed in the configurations to make it work
properly. I think I should eventually get these messages in the
nodemanager.out log file?

Thanks in advance,

Sherif

Re: Hadoop yarn tasklevel logging with log4j

[Harsh J <ha...@cloudera.com>]

Sherif,

For 2.x, use via your Job's configuration, the properties
"mapreduce.map.log.level" and "mapreduce.reduce.log.level" (valid
values are TRACE/DEBUG/INFO/WARN/ERROR/FATAL) to switch the Child
JVM's task logging levels.

On Wed, Jun 27, 2012 at 5:53 PM, Sherif Akoush <sh...@gmail.com> wrote:
> Hi,
>
> I have been trying to setup up Hadoop logging at the task level but
> with no success so far. I have modified log4j.properties and set many
> parameters to DEBUG level
> (log4j.logger.org.apache.hadoop.mapred.Task=DEBUG
> log4j.logger.org.apache.hadoop.mapred.MapTask=DEBUG
> log4j.logger.org.apache.hadoop.mapred.ReduceTask=DEBUG
>
> and log4j.logger.org.apache.hadoop = DEBUG )
>
> but I cant see any LOG.info or LOG.debug messages from the tasks being
> recorded. This is with yarn being enabled.
>
> I am not sure what I have missed in the configurations to make it work
> properly. I think I should eventually get these messages in the
> nodemanager.out log file?
>
> Thanks in advance,
>
> Sherif



-- 
Harsh J