You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/05/27 19:02:00 UTC

[jira] [Updated] (YARN-11698) Finished containers shouldn't be stored indefinitely in the NM state store

     [ https://issues.apache.org/jira/browse/YARN-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated YARN-11698:
----------------------------------
    Labels: pull-request-available  (was: )

> Finished containers shouldn't be stored indefinitely in the NM state store
> --------------------------------------------------------------------------
>
>                 Key: YARN-11698
>                 URL: https://issues.apache.org/jira/browse/YARN-11698
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 3.4.0
>            Reporter: Adam Binford
>            Priority: Major
>              Labels: pull-request-available
>
> https://issues.apache.org/jira/browse/YARN-4771 updated the container tracking in the state store to only remove containers when their application ends, in order to make sure all containers logs get aggregated even during NM restarts. This can lead to a significant number of containers building up in the state store and a lot of things to recover. Since this was purely for making sure logs get aggregated, it could be done smarter that takes into account both rolling log aggregation or not having log aggregation enabled at all.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org