[jira] [Commented] (YARN-6509) Add a size threshold beyond which yarn logs will require a force option

["Junping Du (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991305#comment-15991305 ] 

Junping Du commented on YARN-6509:
----------------------------------

bq. Minor point of correction, 2^63 -1 is something like 8.3 million TiB.
My bad. You are right, it is ~8 EB which should be large enough.

bq. I'd like it if we could find some operational users to weigh in on wether 10GiB is enough for a default. could we ask on the yarn user list?
I doubt we can have a uniformed value for different users that operated on different scale of cluster. My personal feeling is over 10GB log file is hard to analysis but it happen often for large job on some large clusters. Or we can keep above size as default which almost means no limit by default and let each user/admin set this value case by case if they want to?​

> Add a size threshold beyond which yarn logs will require a force option
> -----------------------------------------------------------------------
>
>                 Key: YARN-6509
>                 URL: https://issues.apache.org/jira/browse/YARN-6509
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Xuan Gong
>             Fix For: 2.9.0
>
>         Attachments: YARN-6509.1.patch
>
>
> An accidental fetch for a long running application can lead to scenario which the large size of log can fill up a disk.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org