You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Panagiotis Garefalakis (JIRA)" <ji...@apache.org> on 2018/02/02 15:14:00 UTC
[jira] [Commented] (YARN-7839) Check node capacity before placing
in the Algorithm
[ https://issues.apache.org/jira/browse/YARN-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350496#comment-16350496 ]
Panagiotis Garefalakis commented on YARN-7839:
----------------------------------------------
Submitting a simple patch tracking available cluster resources in the DefaultPlacement algorithm - to support capacity check before placement.
The actual check is part of the attemptPlacementOnNode method which could be configured with the **ignoreResourceCheck** flag.
In the current patch the check is enabled on placement step and disabled on the validation step.
A wrapper class SchedulingRequestWithPlacementAttempt was also introduced to keep track of the failed attempts on the rejected SchedulingRequests.
Thoughts? [~asuresh] [~kkaranasos] [~cheersyang]
> Check node capacity before placing in the Algorithm
> ---------------------------------------------------
>
> Key: YARN-7839
> URL: https://issues.apache.org/jira/browse/YARN-7839
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Panagiotis Garefalakis
> Priority: Major
> Attachments: YARN-7839-YARN-6592.001.patch
>
>
> Currently, the Algorithm assigns a node to a request purely based on if the constraints are met. It is later in the scheduling phase that the Queue capacity and Node capacity are checked. If the request cannot be placed because of unavailable Queue/Node capacity, the request is retried by the Algorithm.
> For clusters that are running at high utilization, we can reduce the retries if we perform the Node capacity check in the Algorithm as well. The Queue capacity check and the other user limit checks can still be handled by the scheduler (since queues and other limits are tied to the scheduler, and not scheduler agnostic)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org