[jira] [Commented] (YUNIKORN-790) Implement MaxApplications enforcement

["Wilfred Spiegelenburg (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YUNIKORN-790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503990#comment-17503990 ] 

Wilfred Spiegelenburg commented on YUNIKORN-790:
------------------------------------------------

To clarify the feature:

MaxApplications is for _running_ applications.
 * allowed: more than the MaxApplications in a queue waiting. (no impact on AddApplication)
 * NOT allowed: more than MaxApplications applications in the running state at the same time.

We set the value to enforce in the queue based on the configuration. Each queue will have a _maxRunningApps_ as an uint64
Tracking the number of running applications in the queue. Each queue will have a _runningApps_ as an uint64.
A _maxRunningApps_ set to 0 means that we do not enforce or check the number of running applications.

Enforcement during scheduling: {{runningApps < maxRunningApps}} the application can start and progress from _Accepted_ into the _Starting_ state. In detail:
The queue should provide a method that allows this check: {{canRun() boolean}}. It returns true if the application can change to become running, false if it can not progress. The application should check if it can run during scheduling based on the state the application is in. This {{canRun}} check should thus only be triggered when the application is in the _Accepted_ state, before moving into the _Starting_ state.
The queue should provide methods to increment and decrement the number of running applications.

The running count changes based on the _applicationState_ changes:
 * application leaves _Running_ state, _runningApps_ counter for the queue decrements (-1)
 * application enters _Starting_ state, _runningApps_ counter for the queue increments (+1)

> Implement MaxApplications enforcement
> -------------------------------------
>
>                 Key: YUNIKORN-790
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-790
>             Project: Apache YuniKorn
>          Issue Type: New Feature
>          Components: core - scheduler
>            Reporter: Wilfred Spiegelenburg
>            Assignee: Chung En Lee
>            Priority: Major
>
> Queues have an option to set the MaxApplications that can run in a queue. There is currently no code in the scheduler that checks this setting.
> As a new feature we should add the enforcement for this setting:
>  * enforce the setting on a leaf queue
>  * enforce the setting on a parent, the apps running in a parent queue is defined as the sum of all the apps running in all leaf queues of the parent.
> As a side note from a config check: we need to make sure that the parent setting cannot be lower than any of the child queues it has. We _must not_ enforce that the parent setting must be larger than sum of all leafs.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org