Log4j changes not working inside static mapper and reducer classes

[Himanish Kushary <hi...@gmail.com>]

Hi,

I have enabled debug for my Map-Reduce package inside the log4j.properties
under the $HADOOP_HOME/conf directory (using CDH3).

log4j.logger.com.himanish.analytics.mapreduce=DEBUG

The logging messages are getting logged for the main enclosing Map-Reduce
job class but not for the static Mapper-Reducer classes inside it. I am
using the same logger instance from the enclosing class inside the static
classes.I tried creating seperate logger for the static mapper-reducer
classes to no effect.

Could anybody please provide any suggestions to resolve this. Also is
modifying the log4j.properties in the conf directory a good approach to
enable logging or is there some other better way.

-- 
Thanks & Regards
Himanish

Re: Log4j changes not working inside static mapper and reducer classes

[Himanish Kushary <hi...@gmail.com>]

Thanks Ted and Dave for the suggestion.I am able to see the logging now in
the proper logfiles by setting log4j.logger.{*my-project-package-name*}=DEBUG
in the $HADOOP_HOME/conf/log4j.properties on all the task tracker machines.

On Wed, May 25, 2011 at 7:10 PM, Ted Yu <yu...@gmail.com> wrote:

> >> Also is modifying the log4j.properties in the conf directory a good
> approach to ...
> I assume you have distributed the modified log4j.properties onto the task
> tracker machines.
>
> On Wed, May 25, 2011 at 3:52 PM, Himanish Kushary <himanish@gmail.com
> >wrote:
>
> > The log4j logging statements work when I run the Map-Reduce job from
> > eclipse
> > using the LocalTaskTracker. But the logging is not working when I ran the
> > Map-Reduce through hadoop jar command on the cluster. Strangely only the
> > logging statements in the main enclosing class(the job class with main
> > method) are working (so seems to be picking up the log4j settings)
> whereas
> > the logging statements inside the static Mapper and Reducer classes are
> not
> > getting printed. This is a HBase Map-Reduce job.
> >
> > Could it be some Hadoop/HBase specific log4j settings which could help to
> > get the logging messages inside the static Mapper-Reducer classes to get
> > printed.
> >
> > On Wed, May 25, 2011 at 4:07 PM, Jean-Daniel Cryans <jdcryans@apache.org
> > >wrote:
> >
> > > I'm not sure why you are asking this question on the hbase user
> > > mailing list, it seems like you have a log4j issue.
> > >
> > > J-D
> > >
> > > On Wed, May 25, 2011 at 1:03 PM, Himanish Kushary <hi...@gmail.com>
> > > wrote:
> > > > Could anybody please help me with this.
> > > >
> > > > On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <
> himanish@gmail.com
> > > >wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I have enabled debug for my Map-Reduce package inside the
> > > log4j.properties
> > > >> under the $HADOOP_HOME/conf directory (using CDH3).
> > > >>
> > > >> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
> > > >>
> > > >> The logging messages are getting logged for the main enclosing
> > > Map-Reduce
> > > >> job class but not for the static Mapper-Reducer classes inside it. I
> > am
> > > >> using the same logger instance from the enclosing class inside the
> > > static
> > > >> classes.I tried creating seperate logger for the static
> mapper-reducer
> > > >> classes to no effect.
> > > >>
> > > >> Could anybody please provide any suggestions to resolve this. Also
> is
> > > >> modifying the log4j.properties in the conf directory a good approach
> > to
> > > >> enable logging or is there some other better way.
> > > >>
> > > >> --
> > > >> Thanks & Regards
> > > >> Himanish
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks & Regards
> > > > Himanish
> > > >
> > >
> >
> >
> >
> > --
> > Thanks & Regards
> > Himanish
> >
>



-- 
Thanks & Regards
Himanish

Re: Log4j changes not working inside static mapper and reducer classes

[Ted Yu <yu...@gmail.com>]

>> Also is modifying the log4j.properties in the conf directory a good
approach to ...
I assume you have distributed the modified log4j.properties onto the task
tracker machines.

On Wed, May 25, 2011 at 3:52 PM, Himanish Kushary <hi...@gmail.com>wrote:

> The log4j logging statements work when I run the Map-Reduce job from
> eclipse
> using the LocalTaskTracker. But the logging is not working when I ran the
> Map-Reduce through hadoop jar command on the cluster. Strangely only the
> logging statements in the main enclosing class(the job class with main
> method) are working (so seems to be picking up the log4j settings) whereas
> the logging statements inside the static Mapper and Reducer classes are not
> getting printed. This is a HBase Map-Reduce job.
>
> Could it be some Hadoop/HBase specific log4j settings which could help to
> get the logging messages inside the static Mapper-Reducer classes to get
> printed.
>
> On Wed, May 25, 2011 at 4:07 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
>
> > I'm not sure why you are asking this question on the hbase user
> > mailing list, it seems like you have a log4j issue.
> >
> > J-D
> >
> > On Wed, May 25, 2011 at 1:03 PM, Himanish Kushary <hi...@gmail.com>
> > wrote:
> > > Could anybody please help me with this.
> > >
> > > On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <himanish@gmail.com
> > >wrote:
> > >
> > >> Hi,
> > >>
> > >> I have enabled debug for my Map-Reduce package inside the
> > log4j.properties
> > >> under the $HADOOP_HOME/conf directory (using CDH3).
> > >>
> > >> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
> > >>
> > >> The logging messages are getting logged for the main enclosing
> > Map-Reduce
> > >> job class but not for the static Mapper-Reducer classes inside it. I
> am
> > >> using the same logger instance from the enclosing class inside the
> > static
> > >> classes.I tried creating seperate logger for the static mapper-reducer
> > >> classes to no effect.
> > >>
> > >> Could anybody please provide any suggestions to resolve this. Also is
> > >> modifying the log4j.properties in the conf directory a good approach
> to
> > >> enable logging or is there some other better way.
> > >>
> > >> --
> > >> Thanks & Regards
> > >> Himanish
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks & Regards
> > > Himanish
> > >
> >
>
>
>
> --
> Thanks & Regards
> Himanish
>

Re: Log4j changes not working inside static mapper and reducer classes

[Dave Latham <la...@davelink.net>]

I'd recommend adding -Dlog4j.debug to the JVM args for any JVM that's not
giving you what you expect.  In this case, if it's the map/reduce tasks, add
it to mapred.child.java.opts in mapred-site.xml.  It should show you what
configuration log4j is actually picking up.

Dave

On Wed, May 25, 2011 at 3:52 PM, Himanish Kushary <hi...@gmail.com>wrote:

> The log4j logging statements work when I run the Map-Reduce job from
> eclipse
> using the LocalTaskTracker. But the logging is not working when I ran the
> Map-Reduce through hadoop jar command on the cluster. Strangely only the
> logging statements in the main enclosing class(the job class with main
> method) are working (so seems to be picking up the log4j settings) whereas
> the logging statements inside the static Mapper and Reducer classes are not
> getting printed. This is a HBase Map-Reduce job.
>
> Could it be some Hadoop/HBase specific log4j settings which could help to
> get the logging messages inside the static Mapper-Reducer classes to get
> printed.
>
> On Wed, May 25, 2011 at 4:07 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
>
> > I'm not sure why you are asking this question on the hbase user
> > mailing list, it seems like you have a log4j issue.
> >
> > J-D
> >
> > On Wed, May 25, 2011 at 1:03 PM, Himanish Kushary <hi...@gmail.com>
> > wrote:
> > > Could anybody please help me with this.
> > >
> > > On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <himanish@gmail.com
> > >wrote:
> > >
> > >> Hi,
> > >>
> > >> I have enabled debug for my Map-Reduce package inside the
> > log4j.properties
> > >> under the $HADOOP_HOME/conf directory (using CDH3).
> > >>
> > >> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
> > >>
> > >> The logging messages are getting logged for the main enclosing
> > Map-Reduce
> > >> job class but not for the static Mapper-Reducer classes inside it. I
> am
> > >> using the same logger instance from the enclosing class inside the
> > static
> > >> classes.I tried creating seperate logger for the static mapper-reducer
> > >> classes to no effect.
> > >>
> > >> Could anybody please provide any suggestions to resolve this. Also is
> > >> modifying the log4j.properties in the conf directory a good approach
> to
> > >> enable logging or is there some other better way.
> > >>
> > >> --
> > >> Thanks & Regards
> > >> Himanish
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks & Regards
> > > Himanish
> > >
> >
>
>
>
> --
> Thanks & Regards
> Himanish
>

Re: Log4j changes not working inside static mapper and reducer classes

[Himanish Kushary <hi...@gmail.com>]

The log4j logging statements work when I run the Map-Reduce job from eclipse
using the LocalTaskTracker. But the logging is not working when I ran the
Map-Reduce through hadoop jar command on the cluster. Strangely only the
logging statements in the main enclosing class(the job class with main
method) are working (so seems to be picking up the log4j settings) whereas
the logging statements inside the static Mapper and Reducer classes are not
getting printed. This is a HBase Map-Reduce job.

Could it be some Hadoop/HBase specific log4j settings which could help to
get the logging messages inside the static Mapper-Reducer classes to get
printed.

On Wed, May 25, 2011 at 4:07 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> I'm not sure why you are asking this question on the hbase user
> mailing list, it seems like you have a log4j issue.
>
> J-D
>
> On Wed, May 25, 2011 at 1:03 PM, Himanish Kushary <hi...@gmail.com>
> wrote:
> > Could anybody please help me with this.
> >
> > On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <himanish@gmail.com
> >wrote:
> >
> >> Hi,
> >>
> >> I have enabled debug for my Map-Reduce package inside the
> log4j.properties
> >> under the $HADOOP_HOME/conf directory (using CDH3).
> >>
> >> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
> >>
> >> The logging messages are getting logged for the main enclosing
> Map-Reduce
> >> job class but not for the static Mapper-Reducer classes inside it. I am
> >> using the same logger instance from the enclosing class inside the
> static
> >> classes.I tried creating seperate logger for the static mapper-reducer
> >> classes to no effect.
> >>
> >> Could anybody please provide any suggestions to resolve this. Also is
> >> modifying the log4j.properties in the conf directory a good approach to
> >> enable logging or is there some other better way.
> >>
> >> --
> >> Thanks & Regards
> >> Himanish
> >>
> >
> >
> >
> > --
> > Thanks & Regards
> > Himanish
> >
>



-- 
Thanks & Regards
Himanish

Re: Log4j changes not working inside static mapper and reducer classes

[Jean-Daniel Cryans <jd...@apache.org>]

I'm not sure why you are asking this question on the hbase user
mailing list, it seems like you have a log4j issue.

J-D

On Wed, May 25, 2011 at 1:03 PM, Himanish Kushary <hi...@gmail.com> wrote:
> Could anybody please help me with this.
>
> On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <hi...@gmail.com>wrote:
>
>> Hi,
>>
>> I have enabled debug for my Map-Reduce package inside the log4j.properties
>> under the $HADOOP_HOME/conf directory (using CDH3).
>>
>> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
>>
>> The logging messages are getting logged for the main enclosing Map-Reduce
>> job class but not for the static Mapper-Reducer classes inside it. I am
>> using the same logger instance from the enclosing class inside the static
>> classes.I tried creating seperate logger for the static mapper-reducer
>> classes to no effect.
>>
>> Could anybody please provide any suggestions to resolve this. Also is
>> modifying the log4j.properties in the conf directory a good approach to
>> enable logging or is there some other better way.
>>
>> --
>> Thanks & Regards
>> Himanish
>>
>
>
>
> --
> Thanks & Regards
> Himanish
>

Re: Log4j changes not working inside static mapper and reducer classes

[Himanish Kushary <hi...@gmail.com>]

Could anybody please help me with this.

On Tue, May 24, 2011 at 10:17 AM, Himanish Kushary <hi...@gmail.com>wrote:

> Hi,
>
> I have enabled debug for my Map-Reduce package inside the log4j.properties
> under the $HADOOP_HOME/conf directory (using CDH3).
>
> log4j.logger.com.himanish.analytics.mapreduce=DEBUG
>
> The logging messages are getting logged for the main enclosing Map-Reduce
> job class but not for the static Mapper-Reducer classes inside it. I am
> using the same logger instance from the enclosing class inside the static
> classes.I tried creating seperate logger for the static mapper-reducer
> classes to no effect.
>
> Could anybody please provide any suggestions to resolve this. Also is
> modifying the log4j.properties in the conf directory a good approach to
> enable logging or is there some other better way.
>
> --
> Thanks & Regards
> Himanish
>



-- 
Thanks & Regards
Himanish