You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Rohith Sharma K S (JIRA)" <ji...@apache.org> on 2016/06/02 07:32:59 UTC
[jira] [Commented] (YARN-5193) For long running services, aggregate
logs when a container completes instead of when the app completes
[ https://issues.apache.org/jira/browse/YARN-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311880#comment-15311880 ]
Rohith Sharma K S commented on YARN-5193:
-----------------------------------------
Looking into this JIRA, currently log rolling can be enabled. Doesn't it help?
bq. There's some added benefits like cleaning out local disk space early, instead of waiting till the app completes.
+1 for this. Recently in our production cluster container started failing because of sub folder creation under application folder. One of the improvement which I raised for this is YARN-4953.
> For long running services, aggregate logs when a container completes instead of when the app completes
> ------------------------------------------------------------------------------------------------------
>
> Key: YARN-5193
> URL: https://issues.apache.org/jira/browse/YARN-5193
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Siddharth Seth
>
> For a long running service, containers will typically not complete very often. However, when a container completes - it would be useful to aggregate the logs right then, instead of waiting for the app to complete.
> This will allow the command line log tool to lookup containers for an app from the log file index itself, instead of having to go and talk to YARN. Talking to YARN really only works if ATS is enabled, and YARN is configured to publish container information to ATS (That may not always be the case - since this can overload ATS quite fast).
> There's some added benefits like cleaning out local disk space early, instead of waiting till the app completes. (There's probably a separate jira somewhere about cleanup of container for long running services anyway)
> cc [~vinodkv], [~xgong]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org