You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by cc...@apache.org on 2024/02/26 19:28:03 UTC

(yunikorn-site) branch master updated: [YUNIKORN-2439] Deprecate state-aware sorting policy (#403)

This is an automated email from the ASF dual-hosted git repository.

ccondit pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yunikorn-site.git


The following commit(s) were added to refs/heads/master by this push:
     new df5ddac3f4 [YUNIKORN-2439] Deprecate state-aware sorting policy (#403)
df5ddac3f4 is described below

commit df5ddac3f4bd543241554d33d14eee5d9206d871
Author: Craig Condit <cc...@apache.org>
AuthorDate: Mon Feb 26 13:27:50 2024 -0600

    [YUNIKORN-2439] Deprecate state-aware sorting policy (#403)
    
    Closes: #403
---
 docs/user_guide/gang_scheduling.md                    |  3 ++-
 docs/user_guide/labels_and_annotations_in_yunikorn.md | 14 +++++++-------
 docs/user_guide/queue_config.md                       |  6 +++++-
 docs/user_guide/resource_quota_mgmt.md                |  5 ++---
 docs/user_guide/sorting_policies.md                   |  6 ++++++
 5 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/docs/user_guide/gang_scheduling.md b/docs/user_guide/gang_scheduling.md
index 21739bce2d..1b8dd24bf2 100644
--- a/docs/user_guide/gang_scheduling.md
+++ b/docs/user_guide/gang_scheduling.md
@@ -52,7 +52,7 @@ To configure queue sorting policy, please refer to doc: [app sorting policies](s
 #### Why the `FIFO` sorting policy
 
 When Gang Scheduling is enabled, the scheduler proactively reserves resources
-for each application. If the queue sorting policy is not FIFO based (StateAware is FIFO based sorting policy),
+for each application. If the queue sorting policy is not FIFO-based (StateAware is FIFO based sorting policy),
 the scheduler might reserve partial resources for each app and causing resource segmentation issues.
 
 #### Side effects of `StateAware` sorting policy
@@ -61,6 +61,7 @@ We do not recommend using `StateAware`, even-though it is a FIFO based policy. A
 This is specifically an issue with Spark jobs when the driver performs a lot of pre-processing before requesting the executors.
 The `StateAware` timeout in those cases would slow down processing to just one application per timeout.
 This in effect will overrule the gang reservation and cause slowdowns and excessive resource usage.
+`StateAware` sorting is **deprecated** in YuniKorn 1.5.0 and will be **removed** from YuniKorn 1.6.0.
 
 ### App Configuration
 
diff --git a/docs/user_guide/labels_and_annotations_in_yunikorn.md b/docs/user_guide/labels_and_annotations_in_yunikorn.md
index efd9ee8df6..550feaef03 100644
--- a/docs/user_guide/labels_and_annotations_in_yunikorn.md
+++ b/docs/user_guide/labels_and_annotations_in_yunikorn.md
@@ -25,13 +25,13 @@ under the License.
 YuniKorn utilizes several Kubernetes labels and annotations to support various features:
 
 ### Labels in YuniKorn
-| Name                 | Description                                                                                                                                             |
-|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `applicationId`      | Associates this pod with an application.                                                                                                                |
-| `queue`              | Selects the YuniKorn queue this application should be scheduled in. This may be ignored if a placement policy is in effect.                             |
-| `spark-app-selector` | Alternative method of specifying `applicationId` used by Spark Operator if the label `applicationId` and annotation `yunikorn.apache.org/app-id` unset. |
-| `disableStateAware`  | If present, disables the YuniKorn state-aware scheduling policy for this pod. Set internally by the YuniKorn admission controller.                      |
-| `placeholder`        | Set if this pod represents a placeholder for gang scheduling. Set internally by YuniKorn.                                                               |
+| Name                              | Description                                                                                                                                             |
+|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `applicationId`                   | Associates this pod with an application.                                                                                                                |
+| `queue`                           | Selects the YuniKorn queue this application should be scheduled in. This may be ignored if a placement policy is in effect.                             |
+| `spark-app-selector`              | Alternative method of specifying `applicationId` used by Spark Operator if the label `applicationId` and annotation `yunikorn.apache.org/app-id` unset. |
+| [DEPRECATED] `disableStateAware`  | If present, disables the YuniKorn state-aware scheduling policy for this pod. Set internally by the YuniKorn admission controller.                      |
+| `placeholder`                     | Set if this pod represents a placeholder for gang scheduling. Set internally by YuniKorn.                                                               |
 
 ### Annotations in YuniKorn
 All annotations are under the namespace `yunikorn.apache.org`. For example `yunikorn.apache.org/app-id`.
diff --git a/docs/user_guide/queue_config.md b/docs/user_guide/queue_config.md
index 5362785729..75bd24133a 100644
--- a/docs/user_guide/queue_config.md
+++ b/docs/user_guide/queue_config.md
@@ -299,6 +299,10 @@ Default value: `fifo`
 Sets the policy to be used when sorting applications within a queue. This
 setting has no effect on a _parent_ queue.
 
+**NOTE:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and will
+be **removed** from YuniKorn 1.6.0, where it will be treated as an alias for
+`fifo`.
+
 See the documentation on [application sorting](sorting_policies.md#application-sorting)
 for more information.
 
@@ -436,7 +440,7 @@ As an example:
          childtemplate:
            maxapplications: 10
            properties:
-             application.sort.policy: stateaware
+             application.sort.policy: fifo
            resources:
              guaranteed:
                vcore: 1
diff --git a/docs/user_guide/resource_quota_mgmt.md b/docs/user_guide/resource_quota_mgmt.md
index 1c8bf11983..a29bbbd47a 100644
--- a/docs/user_guide/resource_quota_mgmt.md
+++ b/docs/user_guide/resource_quota_mgmt.md
@@ -207,7 +207,7 @@ partitions:
       - name: root
         submitacl: '*'
         properties:
-          application.sort.policy: stateaware
+          application.sort.policy: fifo
 ```
 
 This configuration places an application based on the `tag` rule.
@@ -215,8 +215,7 @@ The tag selected is the `namespace` tag which is automatically added by the k8s
 The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist. 
 
 Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
-In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
-This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#stateawarepolicy).
+In this case the property `application.sort.policy` is in this configuration set to `fifo`.
 
 You can change the configuration using the helm charts during the installation by overwriting the configuration in the
 [helm chart template](https://github.com/apache/yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
diff --git a/docs/user_guide/sorting_policies.md b/docs/user_guide/sorting_policies.md
index 982877a138..f5de40af04 100644
--- a/docs/user_guide/sorting_policies.md
+++ b/docs/user_guide/sorting_policies.md
@@ -91,6 +91,12 @@ Short description: limit of one (1) application in Starting or Accepted state
 
 Config value: `stateaware`
 
+**DEPRECATED:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and
+will be **removed** in YuniKorn 1.6.0. To preserve backwards compatibility,
+`stateaware` will become an alias for `fifo` in YuniKorn 1.6.0 and later.
+Users are encouraged to migrate to `fifo` and utilize either gang scheduling or
+`maxapplications` to limit concurrency instead.
+
 This sorting policy requires an understanding of the application states.
 Applications states are described in the [application states](developer_guide/scheduler_object_states.md#application-state) documentation.
 


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org