[jira] [Commented] (YARN-950) Ability to limit or avoid aggregating logs beyond a certain size

["Eric Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219874#comment-17219874 ] 

Eric Payne commented on YARN-950:
---------------------------------

{quote}1. Prevent logs for any container from becoming larger than a configurable size.
{quote}
I opened YARN-10471 to cover item 1.

 
{quote}2. Truncate everything from the middle of large log files, leaving a configurable amount of the head and the tail.
{quote}
Let's use this ticket (YARN-950) to cover item 2:

> Ability to limit or avoid aggregating logs beyond a certain size
> ----------------------------------------------------------------
>
>                 Key: YARN-950
>                 URL: https://issues.apache.org/jira/browse/YARN-950
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation, nodemanager
>    Affects Versions: 0.23.9, 2.6.0
>            Reporter: Jason Darrell Lowe
>            Priority: Major
>
> It would be nice if ops could configure a cluster such that any container log beyond a configured size would either only have a portion of the log aggregated or not aggregated at all.  This would help speed up the recovery path for cases where a container creates an enormous log and fills a disk, as currently it tries to aggregate the entire, enormous log rather than only aggregating a small portion or simply deleting it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org