You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Robert Kanter (JIRA)" <ji...@apache.org> on 2018/10/08 22:08:00 UTC
[jira] [Comment Edited] (YARN-8807) FairScheduler crashes RM with
oversubscription turned on if an application is killed.
[ https://issues.apache.org/jira/browse/YARN-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642549#comment-16642549 ]
Robert Kanter edited comment on YARN-8807 at 10/8/18 10:07 PM:
---------------------------------------------------------------
Looks good to me, though something must have changed since the patch was created because it doesn't compile anymore:
{noformat}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hadoop-yarn-server-resourcemanager: Compilation failure: Compilation failure:
[ERROR] /Users/rkanter/dev/hadoop-git/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java:[4012,11] cannot find symbol
[ERROR] symbol: method updateContainersAndNodeUtilization(org.apache.hadoop.yarn.server.resourcemanager.rmnode.UpdatedContainerInfo,org.apache.hadoop.yarn.api.records.ResourceUtilization)
[ERROR] location: variable node of type org.apache.hadoop.yarn.server.resourcemanager.MockNodes.MockRMNodeImpl
[ERROR] /Users/rkanter/dev/hadoop-git/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java:[4041,11] cannot find symbol
[ERROR] symbol: method updateContainersAndNodeUtilization(org.apache.hadoop.yarn.server.resourcemanager.rmnode.UpdatedContainerInfo,org.apache.hadoop.yarn.api.records.ResourceUtilization)
[ERROR] location: variable node of type org.apache.hadoop.yarn.server.resourcemanager.MockNodes.MockRMNodeImpl
{noformat}
was (Author: rkanter):
Looks good to me, though something must have changed since the patch was created because it doesn't compile anymore.
> FairScheduler crashes RM with oversubscription turned on if an application is killed.
> -------------------------------------------------------------------------------------
>
> Key: YARN-8807
> URL: https://issues.apache.org/jira/browse/YARN-8807
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler, resourcemanager
> Affects Versions: YARN-1011
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8807-YARN-1011.00.patch
>
>
> When an application, that has got opportunistic containers allocated, is killed, its containers are not released immediately.
> Fair scheduler would therefore continue to try to promote such orphaned containers, which results in NPE.
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptToAssignReservedResourcesOrPromoteOpportunisticContainers(FairScheduler.java:1158)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1129)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:1001)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1275)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testKillingApplicationWithOpportunisticContainersAssigned(TestFairScheduler.java:4019){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org