You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Li Shengmei <li...@ict.ac.cn> on 2012/08/24 05:44:23 UTC

Does anyone do related work on the log of hadoop?

Hi, all

         I want to do some study on the log of hadoop. I intend to konw how
hadoop record the IO in the logs. 

Does anyone do related work and give some suggestions? 

Thanks,

May

Re: 答复: Does anyone do related work on the log of hadoop?

Posted by Steve Loughran <st...@hortonworks.com>.
On 29 August 2012 10:36, Li Shengmei <li...@ict.ac.cn> wrote:

> Hi, Steven,
>         Thank you very much. I just start to study the log analysis of
> HADOOP. I will review your work carefully.
> Btw, which part of hadoop source codes should be read carefully if I want
> to get know the log record?
>


There's lots of log.info() statements throughout the code -explicit logging
and not any log-injection stuff. If you know what you want to log,


   1. Look at the existing log output at info, and from the name of the log
   that appears in the text (org.apache.hadoop.something.somethingelse)
   2. rummage through the source code in that class to find the bit of
   interest.

The normal log4j settings for Hadoop strip off the full log heirarchy; if
you set the log to a better conversion pattern like
log4j.appender.STDOUT.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t]
%p %c %x: %m%n

then you get more

答复: Does anyone do related work on the log of hadoop?

Posted by Li Shengmei <li...@ict.ac.cn>.
Hi, Steven, 
	Thank you very much. I just start to study the log analysis of HADOOP. I will review your work carefully. 
Btw, which part of hadoop source codes should be read carefully if I want to get know the log record?

Thanks
May
---------------------------------------------

What are you trying to log/analyse?

I self-assigned to me "do better logging for machine analysis", a long time ago, but never sat down to do any of it -yet-

https://issues.apache.org/jira/browse/HADOOP-7466


On 23 August 2012 20:44, Li Shengmei <li...@ict.ac.cn> wrote:

> Hi, all
>
>          I want to do some study on the log of hadoop. I intend to 
> konw how hadoop record the IO in the logs.
>
> Does anyone do related work and give some suggestions?
>
> Thanks,
>
> May
>
>

Re: Does anyone do related work on the log of hadoop?

Posted by Steve Loughran <st...@hortonworks.com>.
What are you trying to log/analyse?

I self-assigned to me "do better logging for machine analysis", a long time
ago, but never sat down to do any of it -yet-

https://issues.apache.org/jira/browse/HADOOP-7466


On 23 August 2012 20:44, Li Shengmei <li...@ict.ac.cn> wrote:

> Hi, all
>
>          I want to do some study on the log of hadoop. I intend to konw how
> hadoop record the IO in the logs.
>
> Does anyone do related work and give some suggestions?
>
> Thanks,
>
> May
>
>