You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Bibin A Chundatt (JIRA)" <ji...@apache.org> on 2019/07/10 16:58:00 UTC
[jira] [Updated] (YARN-9645) Fix Invalid event
FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart
[ https://issues.apache.org/jira/browse/YARN-9645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bibin A Chundatt updated YARN-9645:
-----------------------------------
Summary: Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart (was: Restaring NM's throwing Invalid event: FINISHED_CONTAINERS_PULLED_BY_AM at NEW)
> Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart
> -----------------------------------------------------------------------
>
> Key: YARN-9645
> URL: https://issues.apache.org/jira/browse/YARN-9645
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: krishna reddy
> Assignee: Bilwa S T
> Priority: Major
> Attachments: YARN-9645-001.patch, YARN-9645-002.patch
>
>
> *Description: *While Restarting NM throughing org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event: FINISHED_CONTAINERS_PULLED_BY_AM at NEW"
> *Environment: *
> Server OS :- UBUNTU
> No. of Cluster Node:- 2 RM / 4850 NMs
> total 240 machines, in each machine 21 docker containers (1 DN & 20 NM's)
> *Steps:*
> 1. Total number of containers running state : ~53000
> 2. Restart the NM's and check in the log
> {noformat}
> 019-06-24 09:37:35,345 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application with id 32744 submitted by user root
> 2019-06-24 09:37:35,346 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=root IP=255.255.19.245 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1561358926330_32744 QUEUENAME=default
> 2019-06-24 09:37:35,345 ERROR org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event: FINISHED_CONTAINERS_PULLED_BY_AM at NEW
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)
> at org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl.handle(RMNodeImpl.java:669)
> at org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl.handle(RMNodeImpl.java:99)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher.handle(ResourceManager.java:1107)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher.handle(ResourceManager.java:1091)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:221)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:143)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org