You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by GitBox <gi...@apache.org> on 2020/10/06 02:08:23 UTC
[GitHub] [incubator-yunikorn-site] sunilgovind commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
sunilgovind commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499965808
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
Review comment:
This is a real difference for YuniKorn vs Namespace. But many cannot understand this from a one liner. So could we expand this a bit more.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
Review comment:
Is this more like Parent Queue mapping with namespace?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
### Goal
-Though the tag placement rule using the `namespace` tag is capable of putting applications based on the name of the namespace, sometimes more dynamic placement is required.
+Though the tag placement rule using the `namespace` tag is capable of placing an application in a queue this might not be enough in all setups.
+In some cases, multi tenancy for example, namespaces need to be grouped together.
+Administrators could annotate namespaces which allows dynamic placement of applications based on multiple annotations if placement rules were setup.
+YuniKorn cannot and does not just add all annotations from a namespace to an application.
-Users can annotate namespaces which allows dynamic placement of applications based on the annotation value if proper placement rules are present.
+To help support this grouping case a parent queue can be tagged on a namespace.
### Configuration
Review comment:
Below section looks very complex to understand.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota would thus not have any effect and is handled as a configuration error.
-### Configuration
+In the hierarchy there are some further rules that need to be considered.
+If a parent queue has multiple children the sum of the **usage** of all children combined can never exceed the quota **configured** on the parent.
+However, from a configuration perspective this does not mean that the sum of the **configured** quotas for all children must be smaller than the parent quota.
+
+As an example the `root.parent` queue has a quota of 900.
Review comment:
Is there a possibility to insert some diagram or use some arrows to show this queue?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
Review comment:
*its place -> it's placed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org