[GitHub] [pulsar] jdbeck opened a new issue #8507: Overridden 'jobName' in Kubernetes runtime with custom runtime options can have collisions when same 'jobName' is used

[GitBox <gi...@apache.org>]

jdbeck opened a new issue #8507:
URL: https://github.com/apache/pulsar/issues/8507


   **Describe the bug**
   As part of the closed issue: https://github.com/apache/pulsar/issues/8012 a customRuntimeOption of "jobName" was added in order to allow an override of the pod name for the function in the Kubernetes runtime.  While it covers most issues one was discovered where a jobName could be re-used for a different function and it would cause collisions with the pod and potentially statefulset.  In order to alleviate this make sure that a short hash is added to the pod name to avoid collisions like is done when job names are converted when illegal characters are added.
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. Create a function with overridden 'jobName'
   
    ./bin/pulsar-admin functions create --custom-runtime-options '{"jobName": "custom-name"}' --auto-ack false --classname org.apache.pulsar.functions.api.examples.ExclamationFunction -i persistent://public/default/in-content -o persistent://public/default/out-content --jar /pulsar/examples/api-examples.jar --name exclamation
   
   2. Create a second function with a different name but with same 'jobName' custom runtime options
   
    ./bin/pulsar-admin functions create --custom-runtime-options '{"jobNamespace": "custom-name"}' --auto-ack false --classname org.apache.pulsar.functions.api.examples.ExclamationFunction -i persistent://public/default/in-content -o persistent://public/default/out-content --jar /pulsar/examples/api-examples.jar --name second-exclamation
   
   3. k get pods
   
   A second pod will not be there as the second call above collided with the pod of the first.  Normally, pulsar would prevent this if a similar name was used for the function.  We just need to make sure there are no collisions when the jobName is overridden with custom runtime options.
   
   **Expected behavior**
   
   In the case 'jobName' is overridden we need to make sure there are no collisions.  Use short hash based on tenant-namespace-function_name which is unique just as is done in the case when 'jobName' is NOT overridden.  
   
   **Additional context**
   
   Update KubernetesRuntime.java -> function createJobName() to handle this case correctly and add unit tests.
   
   Original code:
   // toValidPodName may cause naming collisions, add a short hash here to avoid it
   final String jobName = String.format("%s-%s-%s", tenant, namespace, functionName);
   final String shortHash = DigestUtils.sha1Hex(jobName).toLowerCase().substring(0, 8);
   
   In case where its overridden 'jobName' should be more like this:
   final String hashName = String.format("%s-%s-%s-%s", jobName, tenant, namespace, functionName);
   final String shortHash = DigestUtils.sha1Hex(hashName).toLowerCase().substring(0, 8);
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] sijie closed issue #8507: Overridden 'jobName' in Kubernetes runtime with custom runtime options can have collisions when same 'jobName' is used

[GitBox <gi...@apache.org>]

sijie closed issue #8507:
URL: https://github.com/apache/pulsar/issues/8507


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org