[jira] Updated: (HADOOP-53) MapReduce log files should be storable in dfs.

["Enis Soztutar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Enis Soztutar updated HADOOP-53:
--------------------------------

    Attachment: mapredDFSLog_v1.patch

Attaching a patch to store the tasks' logs to the FileSystem used. This is useful for example to store the logs permanently or to access the logs in a centralized way. 

The patch adds a new log4j Appender, called FsLogAppender, which appends logs to the files in the FileSystem. The appender is called FSLA. The old appender(TLA) still continues. The user can select which appender to use, and which log level to use via JobConf(). ex : job.setTaskLogRootLogger("INFO,TLA,FSLA"); 
The user can also specify the location to save the logs : job.setTaskLogDir(Path);

Now, at DEBUG level the logs from org.apache.hadoop will continue to polute the logs of the user's program, but we will defer this to a separate issue. 

The appender can also be used to store the logs of the framework(for debugging etc,), but again it is a seperate issue. 

> MapReduce log files should be storable in dfs.
> ----------------------------------------------
>
>                 Key: HADOOP-53
>                 URL: https://issues.apache.org/jira/browse/HADOOP-53
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>    Affects Versions: 0.2.0
>            Reporter: Doug Cutting
>            Assignee: Enis Soztutar
>         Attachments: mapredDFSLog_v1.patch
>
>
> It should be possible to cause a job's log output to be stored in dfs.  The jobtracker's log output and (optionally) all tasktracker log output related to a job should be storable in a job-specified dfs directory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.