[jira] [Work started] (MAPREDUCE-6283) MRHistoryServer log files management optimization

["Zhang Wei (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on MAPREDUCE-6283 started by Zhang Wei.
--------------------------------------------
> MRHistoryServer log files management optimization
> -------------------------------------------------
>
>                 Key: MAPREDUCE-6283
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6283
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver
>            Reporter: Zhang Wei
>            Assignee: Zhang Wei
>            Priority: Minor
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> In some heavy computation clusters, there might be a potential hdfs small files problem. The continually submitted MR jobs will create millions of log files (include the application master logs and application logs which are aggregated to hdfs by nodemanager). This optimization design helps to reduce the numbers of log files by merging them into bigger ones.
> Get the details from the design doc which I will upload later.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)