[jira] [Updated] (YARN-11798) Precheck request separately to avoid redundant node checks and optimize performance for global scheduler.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated YARN-11798:
----------------------------------
    Labels: pull-request-available  (was: )

> Precheck request separately to avoid redundant node checks and optimize performance for global scheduler.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-11798
>                 URL: https://issues.apache.org/jira/browse/YARN-11798
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Tao Yang
>            Assignee: Tao Yang
>            Priority: Major
>              Labels: pull-request-available
>
> The current implementation of the RegularContainerAllocator#preCheckCandidateNodes performs redundant check-request logic for each node, leading to significant latency during global scheduling with a large number of candidate nodes. 
> By splitting this method into two separate methods: preCheckRequest and preCheckNode, we can ensure that the check-request logic is executed only once per request before checking all nodes, thereby optimizing scheduling performance and reducing latency.
>  
> In a scheduling scenario with 2000 candidate nodes and an unsatisfied request that reaches the headroom, the global scheduler currently checks the request for each node repeatedly, causing a scheduling cycle to exceed 200ms. With this improvement, the global scheduler checks the request only once before evaluating all nodes, reducing the scheduling cycle to less than 2ms.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org