You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/11/05 13:46:23 UTC
[GitHub] [pulsar] jdbeck opened a new pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
jdbeck opened a new pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452
Fixes #8012
### Motivation
Currently (as of 2.6.x), the Pulsar KubernetesRuntime class hardcodes the jobName (the name assigned to the StatefulSet used to create the function pods) to the format "pf-[tenant]-[namespace]-[function][-optional 8 char hash]." While the intent of this name format was no doubt both to provide a human readable name for the k8s objects and ensure uniqueness within k8s, we've found it -- when combined with the 55 character size restriction imposed by KubernetesRuntime -- to be unnecessarily limiting. In our environment, we ensure that Pulsar functions under a particular Pulsar tenant deploy into a kubernetes namespace dedicated to that tenant; hence, for us the [tenant] portion of the function name is redundant. Further, the "pf-" prefix is unnecessary, as we're able to distinguish the function pods from other pods based on the function name alone. These issues may seem minor, but they consume precious characters against the 55 character max!
### Modifications
Updated Kubernetes runtime and factory classes under pulsar/pulsar-functions to add "jobName" to customRuntimeOptions just like jobNamespace for setting the k8s namespace.
### Verifying this change
- [X ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- Updated/extended src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java to contain news tests for overridden job Name (jobName).
- Updated other tests to account for new parameters for checks but does not change any functionality in the tests.
### Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
- The rest endpoints: (yes)
It does not change the endpoint for creating functions but it does change the available string (json) that can be submitted for 'customRuntimeOptions: string'. Like 'jobNamespace: <name>' you may add 'jobName: <name>' to the json string for changing the k8s pod name for the function.
Example:
curl -v --form "functionConfig={\"tenant\": \"public\", \"namespace\": \"default\", \"name\": \"exclamation\", \"className\": \"org.apache.pulsar.functions.api.examples.ExclamationFunction\", \"inputs\": [\"persistent://public/default/in-content\"], \"output\": \"persistent://public/default/out-content\", \"customRuntimeOptions\": \"{\\\"jobNamespace\\\": \\\"pulsar\\\", **\\\"jobName\\\": \\\"test123\\\**"}\", \"jar\": \"api-examples.jar\"};type=application/json" --form data=@/pulsar/examples/api-examples.jar http://pulsar-broker.pulsar.svc.cluster.local:8080/admin/v3/functions/public/default/exclamation
- The admin cli options: (yes)
Same as above. Adds an available option for creating functions 'customRuntimeOptions'.
Example:
./bin/pulsar-admin functions create --custom-runtime-options '{"jobNamespace": "pulsar", **"jobName": "test123"**}' --auto-ack false --classname org.apache.pulsar.functions.api.examples.ExclamationFunction -i persistent://public/default/in-content -o persistent://public/default/out-content --jar /pulsar/examples/api-examples.jar --name exclamation
- Anything that affects deployment: (no)
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (not documented)
Probably should be added to functions-runtime.md for 2.7.0 ... hopefully (like 'jobNamespace' under 'customRuntimeOptions')
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452#issuecomment-955984331
Confirmed with @freeznet, we have documented here:
- https://pulsar.apache.org/docs/en/next/functions-runtime/#customize-kubernetes-runtime
- https://pulsar.apache.org/docs/en/next/admin-api-overview/#how-to-define-pulsar-resource-names-when-running-pulsar-in-kubernetes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jdbeck commented on pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
Posted by GitBox <gi...@apache.org>.
jdbeck commented on pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452#issuecomment-722510257
/pulsarbot run-failure-check
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jdbeck removed a comment on pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
Posted by GitBox <gi...@apache.org>.
jdbeck removed a comment on pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452#issuecomment-722510257
/pulsarbot run-failure-check
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jdbeck commented on pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
Posted by GitBox <gi...@apache.org>.
jdbeck commented on pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452#issuecomment-722513090
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #8452: [Issue 8012][pulsar_functions] For the Kubernetes function worker runtime, allow customization of the pulsar function "jobName"
Posted by GitBox <gi...@apache.org>.
codelipenghui merged pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org