[GitHub] [spark] hvanhovell commented on a diff in pull request #40147: [SPARK-42543][CONNECT] Specify protocol for UDF artifact transfer in JVM/Scala client

["hvanhovell (via GitHub)" <gi...@apache.org>]

hvanhovell commented on code in PR #40147:
URL: https://github.com/apache/spark/pull/40147#discussion_r1116395939


##########
connector/connect/common/src/main/protobuf/spark/connect/base.proto:
##########
@@ -183,6 +183,60 @@ message ExecutePlanResponse {
   }
 }
 
+// Request to transfer client-local artifacts.
+message AddArtifactsRequest {
+
+  // Definition of an Artifact.
+  message Artifact {
+    // The name of the artifact is expected in the form of a "Relative Path" that is made up of a
+    // sequence of directories and the final file element.
+    // Examples of "Relative Path"s: "jars/test.jar", "classes/xyz.class", "abc.xyz", "a/b/X.jar".
+    // The server is expected to maintain the hierarchy of files as defined by their name. (i.e
+    // The relative path of the file on the server's filesystem will be the same as the name of
+    // the provided artifact)
+    string name = 1;
+    // Raw data.
+    bytes data = 2;
+  }

Review Comment:
   We can add it. Ivy coordinates become a bit of a hassle when someone is using a private repository. Quite a few of our customers provides jars as is, they don't want to or can't manage repositories for various reasons. Furthermore we also need to deal with REPL generated code, and there are a bunch of other use cases that still need this kind of synchronization.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org