You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Eric Payne (Jira)" <ji...@apache.org> on 2020/10/23 19:39:00 UTC
[jira] [Commented] (YARN-10471) Prevent logs for any container from
becoming larger than a configurable size.
[ https://issues.apache.org/jira/browse/YARN-10471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219901#comment-17219901 ]
Eric Payne commented on YARN-10471:
-----------------------------------
I propose a solution that includes the option to limit the disk space for a single container log directory or for all of a container's logs. We have been running with this solution internally for the past 3 years with positive results.
> Prevent logs for any container from becoming larger than a configurable size.
> -----------------------------------------------------------------------------
>
> Key: YARN-10471
> URL: https://issues.apache.org/jira/browse/YARN-10471
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 3.2.1, 3.1.4
> Reporter: Eric Payne
> Assignee: Eric Payne
> Priority: Minor
>
> Configure a cluster such that a task attempt will be killed if any container log exceeds a configured size. This would help prevent logs from filling disks and also prevent the need to aggregate enormous logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org