You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Morty Zhong (JIRA)" <ji...@apache.org> on 2019/03/12 04:11:00 UTC
[jira] [Updated] (YARN-9380) FederationInterceptor get Containers
from RM may return not all the containers when RM/NM restart
[ https://issues.apache.org/jira/browse/YARN-9380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Morty Zhong updated YARN-9380:
------------------------------
Description:
FederationInterceptor will recover the map of containerId to subClusterId(field named
containerIdToSubClusterIdMap) by getting containers from RMs(home and secondary RM) when recover is enabled.However, this may fail in follow condition(RM NM both restart):
# RM is restart(recover is enabled),recover tokens, apps, but no containers(waiting NM reporting containers when rsync)
# RM waiting NM rsync, but before NM rsync, NM is restart.
# before NM rsync to RM, NM recover itself, and FederationInterceptor pull containers from RM(RM has no containers in this moment) and will return containers without the containers from NM that hasn`t rsync with RM
maybe the containerId to subClusterId map store in NMStateStore can solve this?
was:
FederationInterceptor will recover the map of containerId to subClusterId(field named
containerIdToSubClusterIdMap) by getting containers from RMs(home and secondary RM) when recover is enabled.However, this may fail in follow condition(RM NM both restart):
# RM is restart(recover is enabled),recover tokens, apps, but no containers(waiting NM reporting containers when rsync)
# RM waiting NM rsync, but before NM rsync, NM is restart.
# before NM rsync to RM, NM recover itself, and FederationInterceptor pull containers from RM(RM has no containers in this moment) and will return empty list
maybe the containerId to subClusterId map store in NMStateStore can solve this?
Summary: FederationInterceptor get Containers from RM may return not all the containers when RM/NM restart (was: FederationInterceptor get Containers from RM may return empty list when RM/NM restart)
> FederationInterceptor get Containers from RM may return not all the containers when RM/NM restart
> -------------------------------------------------------------------------------------------------
>
> Key: YARN-9380
> URL: https://issues.apache.org/jira/browse/YARN-9380
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
> Reporter: Morty Zhong
> Priority: Major
>
> FederationInterceptor will recover the map of containerId to subClusterId(field named
> containerIdToSubClusterIdMap) by getting containers from RMs(home and secondary RM) when recover is enabled.However, this may fail in follow condition(RM NM both restart):
> # RM is restart(recover is enabled),recover tokens, apps, but no containers(waiting NM reporting containers when rsync)
> # RM waiting NM rsync, but before NM rsync, NM is restart.
> # before NM rsync to RM, NM recover itself, and FederationInterceptor pull containers from RM(RM has no containers in this moment) and will return containers without the containers from NM that hasn`t rsync with RM
> maybe the containerId to subClusterId map store in NMStateStore can solve this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org