You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by GitBox <gi...@apache.org> on 2020/10/05 08:47:53 UTC
[GitHub] [incubator-yunikorn-site] wilfred-s opened a new pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
wilfred-s opened a new pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27
Documentation on how quotas in a queue hierarchy interact and work.
Additional updates to the examples to explain the same points in each
example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r519677485
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
Review comment:
updated, extended the text to explain what we do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499960025
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -82,36 +106,33 @@ partitions:
vcore: 10000
```
-in this example, we are going to setup 3 queues under root, and each of them has
-a specific min/max capacity set up.
-
-### Run workloads
-
-In order to run jobs in specific queues, you will need to set the following label in all pods' spec:
+### Run a workload
+In order to run applications in specific queues, you will need to set the following labels in all pod specs.
+All pods with the same `applicationID` label are considered ti be one application.
+In the below example the application `my-test-app` will run in the queue `root.sandbox`:
```yaml
labels:
app: my-test-app
- applicationId: " my-test-app-01"
+ applicationId: "my-test-app-01"
queue: root.sandbox
```
-## Option 2) 1:1 mapping from namespaces to queues
+## Namespace to queue mapping
### Goal
-
-User just needs to setup namespaces, YuniKorn automatically maps each namespace to an internal resource queue (AKA dynamical queue).
-There is no additional steps to create YuniKorn queues, all queues will be created dynamically,
-resource allocation and quotas will be managed by YuniKorn instead of the namespace resource quota.
+Automatically map a Kubernetes `namespace` to a queue in YuniKorn.
+The user creates the required namespaces in Kubernetes.
+The YuniKorn k8-shim and core scheduler automatically pass the required information and map the namespace to a queue, creating the queue if it does not exist.
Review comment:
fixed all occurrences
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s merged pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s merged pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] sunilgovind commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
sunilgovind commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499965808
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
Review comment:
This is a real difference for YuniKorn vs Namespace. But many cannot understand this from a one liner. So could we expand this a bit more.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
Review comment:
Is this more like Parent Queue mapping with namespace?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
### Goal
-Though the tag placement rule using the `namespace` tag is capable of putting applications based on the name of the namespace, sometimes more dynamic placement is required.
+Though the tag placement rule using the `namespace` tag is capable of placing an application in a queue this might not be enough in all setups.
+In some cases, multi tenancy for example, namespaces need to be grouped together.
+Administrators could annotate namespaces which allows dynamic placement of applications based on multiple annotations if placement rules were setup.
+YuniKorn cannot and does not just add all annotations from a namespace to an application.
-Users can annotate namespaces which allows dynamic placement of applications based on the annotation value if proper placement rules are present.
+To help support this grouping case a parent queue can be tagged on a namespace.
### Configuration
Review comment:
Below section looks very complex to understand.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota would thus not have any effect and is handled as a configuration error.
-### Configuration
+In the hierarchy there are some further rules that need to be considered.
+If a parent queue has multiple children the sum of the **usage** of all children combined can never exceed the quota **configured** on the parent.
+However, from a configuration perspective this does not mean that the sum of the **configured** quotas for all children must be smaller than the parent quota.
+
+As an example the `root.parent` queue has a quota of 900.
Review comment:
Is there a possibility to insert some diagram or use some arrows to show this queue?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
Review comment:
*its place -> it's placed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r521855106
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
### Goal
-Though the tag placement rule using the `namespace` tag is capable of putting applications based on the name of the namespace, sometimes more dynamic placement is required.
+Though the tag placement rule using the `namespace` tag is capable of placing an application in a queue this might not be enough in all setups.
+In some cases, multi tenancy for example, namespaces need to be grouped together.
+Administrators could annotate namespaces which allows dynamic placement of applications based on multiple annotations if placement rules were setup.
+YuniKorn cannot and does not just add all annotations from a namespace to an application.
-Users can annotate namespaces which allows dynamic placement of applications based on the annotation value if proper placement rules are present.
+To help support this grouping case a parent queue can be tagged on a namespace.
### Configuration
Review comment:
Streamlined and moved things around. The diagram should also help.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] yangwwei commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
yangwwei commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499875783
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -82,36 +106,33 @@ partitions:
vcore: 10000
```
-in this example, we are going to setup 3 queues under root, and each of them has
-a specific min/max capacity set up.
-
-### Run workloads
-
-In order to run jobs in specific queues, you will need to set the following label in all pods' spec:
+### Run a workload
+In order to run applications in specific queues, you will need to set the following labels in all pod specs.
+All pods with the same `applicationID` label are considered ti be one application.
+In the below example the application `my-test-app` will run in the queue `root.sandbox`:
```yaml
labels:
app: my-test-app
- applicationId: " my-test-app-01"
+ applicationId: "my-test-app-01"
queue: root.sandbox
```
-## Option 2) 1:1 mapping from namespaces to queues
+## Namespace to queue mapping
### Goal
-
-User just needs to setup namespaces, YuniKorn automatically maps each namespace to an internal resource queue (AKA dynamical queue).
-There is no additional steps to create YuniKorn queues, all queues will be created dynamically,
-resource allocation and quotas will be managed by YuniKorn instead of the namespace resource quota.
+Automatically map a Kubernetes `namespace` to a queue in YuniKorn.
+The user creates the required namespaces in Kubernetes.
+The YuniKorn k8-shim and core scheduler automatically pass the required information and map the namespace to a queue, creating the queue if it does not exist.
Review comment:
`k8-shim` -> `k8shim` or `k8s shim`
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota will not have any effect.
Review comment:
> Setting a quota on a child queue larger than its parent queue's quota will not have any effect
What does this mean?
I think it is not allowed to set the child quota larger than its parent quota
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r519680850
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs, you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation by overwriting the configuration in the
+[helm chart template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set on the namespace.
+To allow specifying a quota on the namespace the following annotations should be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64 CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods in
-the corresponding queue. For example, if a job is submitted to namespace `development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the application to the queue.
+
+For example, if an application is submitted to namespace `development`, then the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
Review comment:
yes correct, changed title slightly adding a diagram as an example too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] yangwwei commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
yangwwei commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r500003382
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota will not have any effect.
Review comment:
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r519677640
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
Review comment:
it's correct, `its place` is the place of the queue in the hierarchy (possessive form)
`it's placed` is the short for of `it is placed` which does not make sense.
https://www.grammarly.com/blog/its-vs-its/
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota would thus not have any effect and is handled as a configuration error.
-### Configuration
+In the hierarchy there are some further rules that need to be considered.
+If a parent queue has multiple children the sum of the **usage** of all children combined can never exceed the quota **configured** on the parent.
+However, from a configuration perspective this does not mean that the sum of the **configured** quotas for all children must be smaller than the parent quota.
+
+As an example the `root.parent` queue has a quota of 900.
Review comment:
yep added that
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-site] wilfred-s commented on a change in pull request #27: [YUNIKORN-308] Document queue quota in hierarchy
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on a change in pull request #27:
URL: https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499964822
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout the hierarchy.
+This means that a child queue can never use more resources than the **configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota will not have any effect.
Review comment:
We only check the direct parent at the moment. I changed the sentence slightly to explain that it would not have an effect and it would be treated as a configuration failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org