You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/01/25 20:23:55 UTC

[GitHub] [pulsar] klwilson227 opened a new issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files. 

klwilson227 opened a new issue #9315:
URL: https://github.com/apache/pulsar/issues/9315


   **Describe the bug**
   When configuring a pod to run as a different users, we found that the download files for the kubernetes runtime are placed in pulsarRoot vs. in the configured workerConfig:downloadDirectory as we had expected. For kubernetes to be able to write the files as other users, these directories should be mounted as volumes, however, since the volume cannot be specified for pulsarRoot, you cannot move where the download files are located, as suggested by the worker configuration. 
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. Use kubernetes runtime, create a new functions worker for a java based worker, and inspect the functions pod command line to identify the location of the Jar files
   
   /pulsar/bin/pulsar-admin --admin-url https://itomdipulsar-proxy:8443 functions
         download --tenant public --namespace default --name provider --destination-file
         **/pulsar**/minio-source-0.0.1-SNAPSHOT.nar && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID}
         && exec java -cp /pulsar/instances/java-instance.jar:/pulsar/instances/deps/*
         -Dpulsar.functions.extra.dependencies.dir=/pulsar/instances/deps -Dpulsar.functions.instance.classpath=/pulsar/conf:::/pulsar/lib/*:
         -Dlog4j.configurationFile=kubernetes_instance_log4j2.xml -Dpulsar.function.log.dir=logs/functions/public/default/provider
         -Dpulsar.function.log.file=provider-$SHARD_ID -Xmx1073741824 org.apache.pulsar.functions.instance.JavaInstanceMain
         --jar **/pulsar/**minio-source-0.0.1-SNAPSHOT.nar --instance_id $SHARD_ID --function_id
   
   
   **Expected behavior**
   Expected to see the configured download directory prefixing the nar file and specified as the location of the sources. 
   
   **Screenshots**
   If applicable, add screenshots to help explain your problem.
   
   **Desktop (please complete the following information):**
    - OS: [e.g. iOS]
   
   **Additional context**
   We tweaked the KubernetesRuntime code to set the:
   this.originalCodeFileName = pulsarRootDir + "/" + originalCodeFileName;
   to
   this.originalCodeFileName = narExtractionDirectory + "/" + originalCodeFileName;
   ...
   Although not correct, this allowed us to configure the resulting directory, and work around the problem, so that as a custom user ID we have access to write the jar file into the mounted /tmp location of the container image. Where narExtractionDirectory is set to "/tmp" by default. 
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] klwilson227 commented on issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.

Posted by GitBox <gi...@apache.org>.
klwilson227 commented on issue #9315:
URL: https://github.com/apache/pulsar/issues/9315#issuecomment-770266202


   Thank you


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] freeznet commented on issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.

Posted by GitBox <gi...@apache.org>.
freeznet commented on issue #9315:
URL: https://github.com/apache/pulsar/issues/9315#issuecomment-770230212


   I have initialized a pr to fix this issue. Please take a look when the pr is ready to review, thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] freeznet commented on issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.

Posted by GitBox <gi...@apache.org>.
freeznet commented on issue #9315:
URL: https://github.com/apache/pulsar/issues/9315#issuecomment-770587244


   @klwilson227 the pr is ready to review, PTAL, thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] sijie closed issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.

Posted by GitBox <gi...@apache.org>.
sijie closed issue #9315:
URL: https://github.com/apache/pulsar/issues/9315


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] freeznet edited a comment on issue #9315: pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.

Posted by GitBox <gi...@apache.org>.
freeznet edited a comment on issue #9315:
URL: https://github.com/apache/pulsar/issues/9315#issuecomment-770230212


   Hi @klwilson227, it appears that k8s runtime does not use `downloadDirectory` as expected. I have initialized a pr to fix this issue. Please take a look when the pr is ready to review, thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org