You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org> on 2012/10/19 02:00:03 UTC
[jira] [Created] (YARN-171) NodeManager should serve logs directly
if log-aggregation is not enabled
Vinod Kumar Vavilapalli created YARN-171:
--------------------------------------------
Summary: NodeManager should serve logs directly if log-aggregation is not enabled
Key: YARN-171
URL: https://issues.apache.org/jira/browse/YARN-171
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 0.23.3
Reporter: Vinod Kumar Vavilapalli
NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-171) NodeManager should serve logs
directly if log-aggregation is not enabled
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479880#comment-13479880 ]
Siddharth Seth commented on YARN-171:
-------------------------------------
Ignore the last comment. At least the bit about the NM not serving logs after App completion.
When aggregation is disabled, the NM is keeping track of applications till the logs are active on the NM - which sets it up nicely for an OOM on busy clusters, depending on the log-retention time.
Accessing the logs requires some application info - ACLs, owner. Maintaining this in memory for the log-retention time could be problematic. One option could be to create an index similar to the history server on local FS. Would prefer not creating a single file per app though. Any suggestions on a lightweight key value store with a compatible license ?
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Siddharth Seth
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-171) NodeManager should serve logs directly
if log-aggregation is not enabled
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated YARN-171:
--------------------------------
Assignee: Siddharth Seth
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Siddharth Seth
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-171) NodeManager should serve logs
directly if log-aggregation is not enabled
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479544#comment-13479544 ]
Siddharth Seth commented on YARN-171:
-------------------------------------
I believe redirect to HS is enabled for all logs. The HS does not know how to contact the NM for the logs with aggregation enabled, and the NM in any case does not serve out logs once an Application is complete.
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Siddharth Seth
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-171) NodeManager should serve logs
directly if log-aggregation is not enabled
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479508#comment-13479508 ]
Siddharth Seth commented on YARN-171:
-------------------------------------
I don't think this needs to be dependent on LOG_AGGREGATION_ENABLED. The NMs can serve logs by default; if log aggregation is enabled, they route over to the aggregated logs once aggregation completes, otherwise they serve logs till they're available.
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-171) NodeManager should serve logs directly
if log-aggregation is not enabled
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated YARN-171:
--------------------------------
Attachment: YARN171_WIP.txt
Partial patch. Will get back to this after several days if noone has taken it over by then.
The current patch is persisting per app ACLs via the LogHandler, and into NM private directories.
The acl files could go along with logs - but with the current localization model, that's a bigger change - which could be done at a later point.
Also, the log handler is probably not the best place for ACL verification. The log handler writing ACLs as well as verifying ACLs does have an advantage in terms of being able to deal with logs after an NM restart.
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Siddharth Seth
> Attachments: YARN171_WIP.txt
>
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-171) NodeManager should serve logs
directly if log-aggregation is not enabled
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479535#comment-13479535 ]
Jason Lowe commented on YARN-171:
---------------------------------
It's already doing this for AM logs today off of the RM app page, correct? I remember seeing redirect-to-HS support in the NM webapp pages.
> NodeManager should serve logs directly if log-aggregation is not enabled
> ------------------------------------------------------------------------
>
> Key: YARN-171
> URL: https://issues.apache.org/jira/browse/YARN-171
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Siddharth Seth
>
> NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.
> We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira