You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2016/11/11 15:40:59 UTC
[jira] [Commented] (YARN-5545) Fix issues related to Max App in
capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657360#comment-15657360 ]
Hudson commented on YARN-5545:
------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10820 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10820/])
YARN-5545. Fix issues related to Max App in capacity scheduler. (naganarasimha_gr: rev 503e73e849cbdd1194cc0d16b4969c60929aca11)
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
> Fix issues related to Max App in capacity scheduler
> ---------------------------------------------------
>
> Key: YARN-5545
> URL: https://issues.apache.org/jira/browse/YARN-5545
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Labels: oct16-medium
> Attachments: YARN-5545.0001.patch, YARN-5545.0002.patch, YARN-5545.0003.patch, YARN-5545.0005.patch, YARN-5545.0006.patch, YARN-5545.0007.patch, YARN-5545.0008.patch, YARN-5545.004.patch, capacity-scheduler.xml
>
>
> Issues as part of Max apps in Capacity scheduler:
> 1. Cap total applications across the queue hierarchy based on existing max app calculation
> 2. Introduce a new configuration to take default max apps per queue irrespective of the queue capacity configuration
> 3. When the capacity configuration of the default partition is ZERO but queue has capacity for other partition then app is not getting submitted, though app is submitted in other partition
> Steps to reproduce Issue 3 :
> Configure capacity scheduler
> yarn.scheduler.capacity.root.default.capacity=0
> yarn.scheduler.capacity.root.queue1.accessible-node-labels.labelx.capacity=50
> yarn.scheduler.capacity.root.default.accessible-node-labels.labelx.capacity=50
> Submit application as below
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-alpha2-SNAPSHOT-tests.jar sleep -Dmapreduce.job.node-label-expression=labelx -Dmapreduce.job.queuename=default -m 1 -r 1 -mt 10000000 -rt 1
> {noformat}
> 2016-08-21 18:21:31,375 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/root/.staging/job_1471670113386_0001
> java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1471670113386_0001 to YARN : org.apache.hadoop.security.AccessControlException: Queue root.default already has 0 applications, cannot accept submission of application: application_1471670113386_0001
> at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:316)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:255)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1344)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341)
> ...
> Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1471670113386_0001 to YARN : org.apache.hadoop.security.AccessControlException: Queue root.default already has 0 applications, cannot accept submission of application: application_1471670113386_0001
> at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:286)
> at org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:296)
> at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org