[GitHub] [spark] xkrogen commented on pull request #40147: [SPARK-42543][CONNECT] Specify protocol for UDF artifact transfer in JVM/Scala client

["xkrogen (via GitHub)" <gi...@apache.org>]

xkrogen commented on PR #40147:
URL: https://github.com/apache/spark/pull/40147#issuecomment-1444053186

   > This code could be executed within the same engine, however we can also use separate processes or VMs to execute this code.
   
   Good point; nothing about the protocol itself precludes us from using separate UDF processes in the future.
   
   > Finally it is early days for connect. We currently want to make it easy to folks to try and to move to connect ...
   
   Generally agreed, but APIs and contracts can be challenging to change once adopted, so it's important to get it right on the first try, at least to the extent that it doesn't force us into bad decisions or tech debt later. That's why I'm interested in discussing a longer-term roadmap about this kind of support.
   
   But I think you have convinced me that this protocol is generally useful!
   
   cc @mridulm in case you're interested in taking a look


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org