You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by xiaohe lan <zo...@gmail.com> on 2015/03/24 15:25:57 UTC
Fwd: How to enable debug log in HDFS
---------- Forwarded message ----------
From: xiaohe lan <zo...@gmail.com>
Date: Tue, Mar 24, 2015 at 9:04 PM
Subject: How to enable debug log in HDFS
To: user@hadoop.apache.org
Hi expert,
I am using Hadoop 2.4, I need to see the debug log of HDFS when performing
some operations. Where can I set it. I did a search in log4j.properties but
I don't what can I do with it.
Thanks,
Xiaohe
Re: How to enable debug log in HDFS
Posted by Charles Lamb <cl...@cloudera.com>.
On 3/25/2015 11:13 AM, xiaohe lan wrote:
> Hi Charles,
>
> In my hadoop-env.sh, I set
>
> export HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console -Xmx512m
> $HADOOP_CLIENT_OPTS"
>
> However, I could not see the debug log in NN's log file
> hadoop-xilan-namenode-xxxx.out. By setting export
> HADOOP_ROOT_LOGGER="DEBUG,console" I could see lots of debug log.
>
> One more question is what will be written to the .log file and what
> will be written to the .out file ?
Logger output goes to the .log file. Very little goes to the .out file IIRC.
Charles
Re: How to enable debug log in HDFS
Posted by Allen Wittenauer <aw...@altiscale.com>.
This obviously doesn't help you with 2.4, but in trunk/3.x this is significantly easier to do:
hdfs --loglevel DEBUG namenode (interactive/non-daemon mode, output to console, usually)
hdfs --loglevel DEBUG --daemon start namenode (daemon mode, output to log file, usually)
On Mar 25, 2015, at 8:13 AM, xiaohe lan <zo...@gmail.com> wrote:
> Hi Charles,
>
> In my hadoop-env.sh, I set
>
> export HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console -Xmx512m
> $HADOOP_CLIENT_OPTS"
>
> However, I could not see the debug log in NN's log file
> hadoop-xilan-namenode-xxxx.out. By setting export
> HADOOP_ROOT_LOGGER="DEBUG,console" I could see lots of debug log.
>
> One more question is what will be written to the .log file and what will be
> written to the .out file ?
>
> @Rich,
>
> I also tried your method but I could not see the debug log as well. I must
> be missing something.
>
> Thanks,
> Xiaohe
>
> On Tue, Mar 24, 2015 at 10:55 PM, Rich Haase <rh...@pandora.com> wrote:
>
>> If you want to enable debug logging of all hadoop components you can add
>> this line to your log4j.properties.
>>
>> log4j.logger.org.apache.hadoop=DEBUG
>>
>> However, in a 3 node cluster that is doing next to nothing this produces a
>> lot of log output, so I find it preferable to specify an one daemon type at
>> a time to enable debug logging on, e.g.
>>
>> log4j.logger.org.apache.hadoop.NameNode=DEBUG
>>
>> It’s also worth while to review the hadoop daemonlog command, which can be
>> used to change the log level of running daemons.
>>
>> Cheers,
>>
>> Rich
>>
>> On Mar 24, 2015, at 10:38 AM, Charles Lamb <clamb@cloudera.com<mailto:
>> clamb@cloudera.com>> wrote:
>>
>> One way is to do:
>>
>> HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console"
>>
>> IHTH.
>>
>> Charles
>>
>> On 3/24/2015 10:25 AM, xiaohe lan wrote:
>> ---------- Forwarded message ----------
>> From: xiaohe lan <zo...@gmail.com>>
>> Date: Tue, Mar 24, 2015 at 9:04 PM
>> Subject: How to enable debug log in HDFS
>> To: user@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>> Hi expert,
>>
>> I am using Hadoop 2.4, I need to see the debug log of HDFS when performing
>> some operations. Where can I set it. I did a search in log4j.properties but
>> I don't what can I do with it.
>>
>> Thanks,
>> Xiaohe
>>
>>
>>
>>
Re: How to enable debug log in HDFS
Posted by xiaohe lan <zo...@gmail.com>.
Hi Charles,
In my hadoop-env.sh, I set
export HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console -Xmx512m
$HADOOP_CLIENT_OPTS"
However, I could not see the debug log in NN's log file
hadoop-xilan-namenode-xxxx.out. By setting export
HADOOP_ROOT_LOGGER="DEBUG,console" I could see lots of debug log.
One more question is what will be written to the .log file and what will be
written to the .out file ?
@Rich,
I also tried your method but I could not see the debug log as well. I must
be missing something.
Thanks,
Xiaohe
On Tue, Mar 24, 2015 at 10:55 PM, Rich Haase <rh...@pandora.com> wrote:
> If you want to enable debug logging of all hadoop components you can add
> this line to your log4j.properties.
>
> log4j.logger.org.apache.hadoop=DEBUG
>
> However, in a 3 node cluster that is doing next to nothing this produces a
> lot of log output, so I find it preferable to specify an one daemon type at
> a time to enable debug logging on, e.g.
>
> log4j.logger.org.apache.hadoop.NameNode=DEBUG
>
> It’s also worth while to review the hadoop daemonlog command, which can be
> used to change the log level of running daemons.
>
> Cheers,
>
> Rich
>
> On Mar 24, 2015, at 10:38 AM, Charles Lamb <clamb@cloudera.com<mailto:
> clamb@cloudera.com>> wrote:
>
> One way is to do:
>
> HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console"
>
> IHTH.
>
> Charles
>
> On 3/24/2015 10:25 AM, xiaohe lan wrote:
> ---------- Forwarded message ----------
> From: xiaohe lan <zo...@gmail.com>>
> Date: Tue, Mar 24, 2015 at 9:04 PM
> Subject: How to enable debug log in HDFS
> To: user@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> Hi expert,
>
> I am using Hadoop 2.4, I need to see the debug log of HDFS when performing
> some operations. Where can I set it. I did a search in log4j.properties but
> I don't what can I do with it.
>
> Thanks,
> Xiaohe
>
>
>
>
Re: How to enable debug log in HDFS
Posted by Rich Haase <rh...@pandora.com>.
If you want to enable debug logging of all hadoop components you can add this line to your log4j.properties.
log4j.logger.org.apache.hadoop=DEBUG
However, in a 3 node cluster that is doing next to nothing this produces a lot of log output, so I find it preferable to specify an one daemon type at a time to enable debug logging on, e.g.
log4j.logger.org.apache.hadoop.NameNode=DEBUG
It’s also worth while to review the hadoop daemonlog command, which can be used to change the log level of running daemons.
Cheers,
Rich
On Mar 24, 2015, at 10:38 AM, Charles Lamb <cl...@cloudera.com>> wrote:
One way is to do:
HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console"
IHTH.
Charles
On 3/24/2015 10:25 AM, xiaohe lan wrote:
---------- Forwarded message ----------
From: xiaohe lan <zo...@gmail.com>>
Date: Tue, Mar 24, 2015 at 9:04 PM
Subject: How to enable debug log in HDFS
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Hi expert,
I am using Hadoop 2.4, I need to see the debug log of HDFS when performing
some operations. Where can I set it. I did a search in log4j.properties but
I don't what can I do with it.
Thanks,
Xiaohe
Re: Fwd: How to enable debug log in HDFS
Posted by Charles Lamb <cl...@cloudera.com>.
One way is to do:
HADOOP_CLIENT_OPTS="-Dhadoop.root.logger=DEBUG,console"
IHTH.
Charles
On 3/24/2015 10:25 AM, xiaohe lan wrote:
> ---------- Forwarded message ----------
> From: xiaohe lan <zo...@gmail.com>
> Date: Tue, Mar 24, 2015 at 9:04 PM
> Subject: How to enable debug log in HDFS
> To: user@hadoop.apache.org
>
>
> Hi expert,
>
> I am using Hadoop 2.4, I need to see the debug log of HDFS when performing
> some operations. Where can I set it. I did a search in log4j.properties but
> I don't what can I do with it.
>
> Thanks,
> Xiaohe
>