[GitHub] [arrow-datafusion] andygrove commented on pull request #2131: [Ballista] Add ballista plugin manager and UDF plugin

[GitBox <gi...@apache.org>]

andygrove commented on pull request #2131:
URL: https://github.com/apache/arrow-datafusion/pull/2131#issuecomment-1086424955


   > Currently I believe the "plugin_dir is a local dir, I think it is better to support distributed file systems(HDFS/Object store) so that both the Executors and Scheduler can load the plugin files from a single place.
   
   Alternatively, users could package up dependencies in a Docker container and deploy that way. This could be more efficient in the case where multiple executors are running on the same node since the image will be downloaded once and cached. It also provides better version control - all executors will be guaranteed to be running the same code (assume a specific version of the image is deployed).
   
   I would be interested to hear more about the use case of loading dependencies from object store though. What would be the motivation of this approach? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org