You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Weiwei Yang (JIRA)" <ji...@apache.org> on 2019/01/03 08:54:00 UTC
[jira] [Comment Edited] (YARN-9164) Shutdown NM may cause NPE when
opportunistic container scheduling is enabled
[ https://issues.apache.org/jira/browse/YARN-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732769#comment-16732769 ]
Weiwei Yang edited comment on YARN-9164 at 1/3/19 8:53 AM:
-----------------------------------------------------------
Thanks [~xiaoheipangzi], latest patch looks good, also thanks for adding the unit test. +1. I will commit this today. BTW, I have modified the title a bit more informative.
was (Author: cheersyang):
Thanks [~xiaoheipangzi], latest patch looks good, also thanks for adding the unit test. +1. I will commit this today.
> Shutdown NM may cause NPE when opportunistic container scheduling is enabled
> ----------------------------------------------------------------------------
>
> Key: YARN-9164
> URL: https://issues.apache.org/jira/browse/YARN-9164
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: lujie
> Assignee: lujie
> Priority: Critical
> Attachments: YARN-9164-0.patch, YARN-9164-1.patch, YARN-9164-2.patch, hadoop-hires-resourcemanager-hadoop11.log
>
>
> We have meeted an NPE which can crash the whole cluster
> {code:java}
> 2018-12-31 22:18:11,924 FATAL org.apache.hadoop.yarn.event.EventDispatcher: Error in handling event type APP_ATTEMPT_REMOVED to the Event Dispatcher
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.completedContainer(AbstractYarnScheduler.java:696)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.doneApplicationAttempt(CapacityScheduler.java:1123)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1827)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:171)
> at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> this bug also happens in the latest trunk!
>
> workload is
> {code:java}
> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-$VERSION.jar pi -Dmapreduce.job.num-opportunistic-maps-percent="100" 50 100
> {code}
> while job is running , shutdown one NM
> also need inject sleep before AbstractYarnScheduler.getNode()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org