You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2022/07/20 20:28:59 UTC
[GitHub] [openwhisk] bdoyle0182 opened a new pull request, #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
bdoyle0182 opened a new pull request, #5288:
URL: https://github.com/apache/openwhisk/pull/5288
## Description
POEM for allowing users to configure fairness within their namespace across their actions. This allows users to control how much an action can autoscale within the confines of the system admin controlled namespace concurrency limit.
## Related issue and scope
- [ ] I opened an issue to propose and discuss this change (#????)
## My changes affect the following components
- [ ] API
- [ ] Controller
- [ ] Message Bus (e.g., Kafka)
- [ ] Loadbalancer
- [ ] Scheduler
- [ ] Invoker
- [ ] Intrinsic actions (e.g., sequences, conductors)
- [ ] Data stores (e.g., CouchDB)
- [ ] Tests
- [ ] Deployment
- [ ] CLI
- [ ] General tooling
- [X] Documentation
## Types of changes
- [ ] Bug fix (generally a non-breaking change which closes an issue).
- [X] Enhancement or new feature (adds new functionality).
- [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
## Checklist:
- [X] I signed an [Apache CLA](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md).
- [X] I reviewed the [style guides](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md#coding-standards) and followed the recommendations (Travis CI will check :).
- [ ] I added tests to cover my changes.
- [X] My changes require further changes to the documentation.
- [X] I updated the documentation where necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927833001
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
Yes every action except A, B, and C. A, B, and C can only have up to the max of whatever their configured limit is regardless of whether their is still additional capacity in the namespace. If action D came in with no limit configured for itself, its action limit is just inferred by the scheduler to be the namespace limit so it can have up to the full 20.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 merged pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 merged PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927271434
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
Review Comment:
```suggestion
if there is capacity for the action to scale out more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
```
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
+Since the scheduler just decides what the limit is to use to determine capacity based on the min of the namespace and action limit, it will therefore just use
+the namespace limit as the capacity limit. Therefore, there is no action required or side effects or coordination required from the system admin wanting to lower the namespace limit.
+However, if the user wants to redeploy the same function with the same limit that is now over the namespace limit; the api will now reject the action upload until the action limit
+is lowered below the new namespace limit.
+- A user may want to update their action to go back to just relying on the namespace limit. Since updates to action documents copy over limits in the update even if not
Review Comment:
Would this be included in [the PR](https://github.com/apache/openwhisk/pull/5287/files) too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927296786
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
Yes it's the user's responsibility. The idea is that users can configure high priority functions to have a high limit such that it can still potentially get more than another low priority function and that lower priority action may not get up to its lower limit. As an example of a namespace with a limit of 20 and 3 actions A, B, and C:
- A configures limit of 5
- B configures limit of 5
- C configures limit of 15
C gets a burst of traffic that uses all 15 of its limit, there is now only 5 remaining for both A and B. That's fine as a user might want this level of scaling at C at the expense of the max A and B can now scale to being 2 or 3 each instead of 5.
Or the user can configure their limits perfectly up to their namespace limit to guarantee fairness such that A gets 5, B gets 5, and C gets 10. I think this level of flexibility is good.
I didn't want to refernce aws lambda explicitly, but in comparison to their concept of reserved concurrency for individual functions I think this provides more flexibility to the user. Reserved concurrency on lambda takes away from the total pool when configuring on a function so if I configure 5 to an action and the account limit is 20, there's now 15 capacity for other functions at all times. Well what if that function is not doing something most of the time? You've now taken away capacity permanently from your pool for a function that barely runs in exchange for a guarantee it will always be able to scale up to 5. In this proposal, you still have the ability to give yourself that guarantee if you provision evenly across your namespace as the user; but also have the flexibility to be smarter with what your traffic patterns look like in giving yourself additional overprovisioned / high priority capacity across your functions (which is sort of what serverless is all about.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927296786
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
Yes it's the user's responsibility. The idea is that users can configure high priority functions to have a high limit such that it can still potentially get more than another low priority function and that lower priority action may not get up to its lower limit. As an example of a namespace with a limit of 20 and 3 actions A, B, and C:
- A configures limit of 5
- B configures limit of 5
- C configures limit of 15
C gets a burst of traffic that uses all 15 of its limit, there is now only 5 remaining for both A and B. That's fine as a user might want this level of scaling at C at the expense of the max A and B can now scale to being 2 or 3 each instead of 5.
Or the user can configure their limits perfectly up to their namespace limit to guarantee fairness such that A gets 5, B gets 5, and C gets 10. I think this level of flexibility to the user is a good thing.
I didn't want to refernce aws lambda explicitly, but in comparison to their concept of reserved concurrency for individual functions I think this provides more flexibility to the user. Reserved concurrency on lambda takes away from the total pool when configuring on a function so if I configure 5 to an action and the account limit is 20, there's now 15 capacity for other functions at all times. Well what if that function is not doing something most of the time? You've now taken away capacity permanently from your pool for a function that barely runs in exchange for a guarantee it will always be able to scale up to 5. In this proposal, you still have the ability to give yourself that guarantee if you provision evenly across your namespace as the user; but also have the flexibility to be smarter with what your traffic patterns look like in giving yourself additional overprovisioned / high priority capacity across your functions (which is sort of what serverless is all about.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927833001
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
Yes every action except A, B, and C. A, B, and C can have the max of whatever their configured limit is regardless of whether their is additional capacity in the namespace. If action D came in with no limit configured for itself, its action limit is just inferred by the scheduler to be the namespace limit so it can have up to the full 20.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927432628
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
ok, so this is mainly to limit an action that may consume huge resources.
What is the relationship between actions with/without `maxContainerConcurrency` when they are invoked concurrently?
Even if action A, B, and C are configured like the above limits, any actions can still be invoked as long as there is enough capacity in the namespace, is that correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927833001
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
Yes every action except A, B, and C. A, B, and C can have up to the max of whatever their configured limit is regardless of whether their is still additional capacity in the namespace. If action D came in with no limit configured for itself, its action limit is just inferred by the scheduler to be the namespace limit so it can have up to the full 20.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-commenter commented on pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#issuecomment-1190752834
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5288?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#5288](https://codecov.io/gh/apache/openwhisk/pull/5288?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4644eb1) into [master](https://codecov.io/gh/apache/openwhisk/commit/4016aa803ee84b849db8a907cbd17a41cc374e7a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4016aa8) will **decrease** coverage by `75.38%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #5288 +/- ##
==========================================
- Coverage 79.96% 4.57% -75.39%
==========================================
Files 238 238
Lines 14122 14161 +39
Branches 599 603 +4
==========================================
- Hits 11292 648 -10644
- Misses 2830 13513 +10683
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5288?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../main/scala/org/apache/openwhisk/core/WarmUp.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2FybVVwLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/common/Prometheus.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Qcm9tZXRoZXVzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ala/org/apache/openwhisk/common/ConfigMXBean.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNWEJlYW4uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/core/entity/WhiskAuth.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L1doaXNrQXV0aC5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../org/apache/openwhisk/core/entity/EntityPath.scala](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0VudGl0eVBhdGguc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [201 more](https://codecov.io/gh/apache/openwhisk/pull/5288/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927289884
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
+Since the scheduler just decides what the limit is to use to determine capacity based on the min of the namespace and action limit, it will therefore just use
+the namespace limit as the capacity limit. Therefore, there is no action required or side effects or coordination required from the system admin wanting to lower the namespace limit.
+However, if the user wants to redeploy the same function with the same limit that is now over the namespace limit; the api will now reject the action upload until the action limit
+is lowered below the new namespace limit.
+- A user may want to update their action to go back to just relying on the namespace limit. Since updates to action documents copy over limits in the update even if not
Review Comment:
Yes it will, just haven't gotten a chance to add it yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927287868
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
BTW, if the sum of the `maxContainerConcurrency` of all actions exceeds the namespace limit, how does it guarantee fairness?
Do we leave it to the users?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] bdoyle0182 commented on a diff in pull request #5288: [POEM] Allow Users To Configure Max Action Container Concurrency Under Their Namespace Limit
Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on code in PR #5288:
URL: https://github.com/apache/openwhisk/pull/5288#discussion_r927835320
##########
proposals/POEM-4-action-concurrency-limit-within-namespace.md:
##########
@@ -0,0 +1,82 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Title
+User Defined Action Level Concurrency Limits Within Confines of Global Namespace Limit
+
+## Status
+* Current state: In-progress
+* Author(s): @bdoyle0182 (Github ID)
+
+## Summary and Motivation
+
+Currently, openwhisk has a single concurrency limit for managing auto scaling within a namespace. This limit for each namespace is managed
+rightly by system administrators to maintain a good balance between the namespaces of the system and the total system's resources.
+
+However, this does not allow for the user to control how their applications scale within the namespace that they are operating. There is no
+fairness across functions within a namespace. The semantics of a namespace can vary heavily depending on how openwhisk is being used. A namespace
+could represent an organization for public cloud, a group within an organization, an application of functions, a logical grouping of applications
+(for example putting all of your interactions with slack in one namespace).
+
+The problem is that a single function can runaway and end up using all of the namespace's resources. It shouldn't be on the system administrators
+to provide this fairness as it's dependent on the application and what the user wants. They may want the existing behavior to allow any action
+to scale up to the total namespace's resources, they may want to restrict one less prioritized function scale up to a smaller threshold so it can't eat
+the entire namespace's resources but still allow other high priority functions access to the entire namespace's resources, or they may want to provide
+limits to all of their actions that add up to their namespace limit which will guarantee each action in their namespace can have up to their defined
+action concurrency limits similar to other FaaS providers concept of reserved concurrency for actions.
+
+With the major revision to how Openwhisk processes activations with the new scheduler, such a feature becomes extremely easy to implement by just adding
+a single new limit that users can configure on their action document.
+
+## Proposed changes: Architecture Diagram (optional), and Design
+
+Add a optional `maxContainerConcurrency` limit field to action documents in the limits section. This limit will be used in the scheduler when deciding
+if there is capacity for the action to scale up more containers. Previously, the scheduler was completely naive of functions across a namespace when provisioning
+more containers, but if this limit is defined the scheduler will only allow to provision containers up to the defined action limit (which must be less than or equal to the namespace limit).
+
+### Implementation details
+
+A working PR of this POEM is already done in which implementation details can be reviewed but I will describe implementation considerations here. Once the POEM is approved,
+I will add any feedback from the POEM, tests, and documentation.
+
+- The scheduler decision maker uses the min of action container concurrency limit and the namespace concurrency limit. If the action limit is less than the namespace
+limit, it will check both that the action hasn't used up all its capacity and that the namespace still has capacity if the action does still have capacity.
+- The new limit `maxContainerConcurrency` on the action document is an optional field. If the field does not exist, the action limit used by the system is
+the namespace limit making this an optional feature.
+- The one thing not yet included in the implementation param is a parameter on the create action api which will allow the user to delete the limit field so that
+the action will rely on the namespace limit again.
+- When creating an action, the api will validate that your action container concurrency limit is less than or equal to the namespace concurrency limit. If it is greater,
+the upload will fail with a BadRequest and error message that the limit must be less than the namespace limit with the namespace limit value included in the message.
+- If the system admin lowers a namespace's concurrency limit below an amount that an existing action document has already configured, it will not break the action.
Review Comment:
And thus if no limits are configured for any action in a namespace, the behavior remains identical to the existing behavior of openwhisk; any action can have up to the full namespace limit making this feature completely opt-in / non-breaking to the existing paradigm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org