You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "fmiguelez (via GitHub)" <gi...@apache.org> on 2023/11/27 13:40:33 UTC
[I] [Bug] Function worker properties `functionInstanceMinResources` and `functionInstanceMaxResources` not honored [pulsar]
fmiguelez opened a new issue, #21626:
URL: https://github.com/apache/pulsar/issues/21626
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Pulsar 2.8.4
### Minimal reproduce step
Set following broker environment variables:
```
PF_functionInstanceMaxResources_cpu: "16"
PF_functionInstanceMaxResources_disk: "1073741824"
PF_functionInstanceMaxResources_ram: "17179869184"
PF_functionInstanceMinResources_cpu: "0.256"
PF_functionInstanceMinResources_disk: "1073741824"
PF_functionInstanceMinResources_ram: "536870912"
```
Instantiate a sink without establishing any "resources" in SinkConfig DTO in a K8s environment.
### What did you expect to see?
A pod that executes the sink (pf-xxx) with following resource parameters set for container:
```
Limits:
cpu: 16
memory: 17179869184
Requests:
cpu: 0.256
memory: 1073741824
```
### What did you see instead?
A pod with following parameters set:
```
Limits:
cpu: 1
memory: 1073741824
Requests:
cpu: 1
memory: 1073741824
```
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Function worker properties `functionInstanceMinResources` and `functionInstanceMaxResources` not honored [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #21626:
URL: https://github.com/apache/pulsar/issues/21626#issuecomment-6381706707
Thanks for the report, @fmiguelez. `functionInstanceMinResources` and `functionInstanceMaxResources` are bounds that the Kubernetes runtime checks a function's requested resources against (`validateMinResourcesRequired` / `validateMaxResourcesRequired` in [KubernetesRuntimeFactory](https://github.com/apache/pulsar/blob/master/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeFactory.java#L442-L492)). A request outside the range is rejected. They are not used as the pod's requests and limits. A sink submitted without `resources` gets the defaults (1 CPU, 1 GiB RAM, see [Resources](https://github.com/apache/pulsar/blob/master/pulsar-client-admin-api/src/main/java/org/apache/pulsar/common/functions/Resources.java#L37-L45)), which fall inside your range, so the pod you saw is the expected result. To get different values, set `resources` in the sink config. To make requests lower than limits, set `cpuOverCommitRatio` / `memoryOverCom
mitRatio` under `kubernetesContainerFactory` in `functions_worker.yml`. I'm closing this as working as designed. Please reopen if a currently supported release (4.0.x, 4.2.x or 5.0.x) behaves differently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Function worker properties `functionInstanceMinResources` and `functionInstanceMaxResources` not honored [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #21626: [Bug] Function worker properties `functionInstanceMinResources` and `functionInstanceMaxResources` not honored
URL: https://github.com/apache/pulsar/issues/21626
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org