[jira] [Commented] (YARN-11781) Implement Dynamic Requests Handling in CapacityScheduler

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17931730#comment-17931730 ] 

ASF GitHub Bot commented on YARN-11781:
---------------------------------------

TaoYang526 opened a new pull request, #7448:
URL: https://github.com/apache/hadoop/pull/7448

   <!--
     Thanks for sending a pull request!
       1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
       2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
   -->
   
   ### Description of PR
   Details:
   * Dynamic Request Handling: The RequestsHandler class has been added to handle and modify resource and scheduling requests at the beginning of the CapacityScheduler#allocate method.
   * Configuration-Driven Updates: The behavior of the request handler can be configured dynamically via capacity-scheduler.xml configuration, allowing for updates to be applied without restarting the ResourceManager.
   * Scriptable Matching: Requests can be matched and modified based on JavaScript expressions, providing flexibility in defining which requests should be updated and how.
   * Support for ResourceRequest to SchedulingRequest Conversion: The handler supports converting ResourceRequest objects to SchedulingRequest objects, enabling more advanced scheduling capabilities.
   
   Usage: To enable and configure the request handler, administrators can set the following properties in capacity-scheduler.xml:
   yarn.scheduler.capacity.request-handler.enabled: Enables or disables the request handler.
   yarn.scheduler.capacity.request-handler.updates: Specifies the JSON configuration for the request updates.
   
   Configuration example:
   ```xml
   <property>
     <name>yarn.scheduler.capacity.request-handler.enabled</name>
     <value>true</value>
   </property>
   <property>
     <name>yarn.scheduler.capacity.request-handler.updates</name>
     <value>
       {
         "items": [
           {
             "appMatchExpr": "queue == 'default'",
             "requestMatchExpr": "priority > 10",
             "isRRToSR": true,
             "partition": "partitionA",
             "executionType": "GUARANTEED",
             "allocationTags": ["tag1", "tag2"],
             "placementConstraint": "and(in,rack,hbase:notin,node,zk)"
           }
         ]
       }
     </value>
   </property>
   ```
   
   ### How was this patch tested?
   
   
   ### For code changes:
   
   - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




> Implement Dynamic Requests Handling in CapacityScheduler
> --------------------------------------------------------
>
>                 Key: YARN-11781
>                 URL: https://issues.apache.org/jira/browse/YARN-11781
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacityscheduler
>            Reporter: Tao Yang
>            Assignee: Tao Yang
>            Priority: Major
>
> This feature introduces a dynamic request handling mechanism in the CapacityScheduler, allowing for the modification of resource and scheduling requests based on configurable rules. This enables more flexible and adaptive resource management without requiring a restart of the ResourceManager.
> Benefits:
>  * Increased Flexibility: Administrators can define custom rules for modifying requests, allowing for more tailored resource allocation strategies.
>  * Enhanced Scheduling: The ability to convert ResourceRequest to SchedulingRequest opens up new possibilities for advanced scheduling and placement constraints.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org