Re: [PR] Support uploading extra resources in REST batch PySpark jobs [kyuubi]

["turboFei (via GitHub)" <gi...@apache.org>]

turboFei commented on code in PR #6335:
URL: https://github.com/apache/kyuubi/pull/6335#discussion_r1697846336


##########
kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/BatchRestApi.java:
##########
@@ -53,6 +54,19 @@ public Batch createBatch(BatchRequest request, File resourceFile) {
     return this.getClient().post(API_BASE_PATH, multiPartMap, Batch.class, client.getAuthHeader());
   }
 
+  public Batch createBatch(BatchRequest request, File resourceFile, List<String> extraResources) {
+    setClientVersion(request);
+    Map<String, MultiPart> multiPartMap = new HashMap<>();
+    multiPartMap.put("batchRequest", new MultiPart(MultiPart.MultiPartType.JSON, request));
+    multiPartMap.put("resourceFile", new MultiPart(MultiPart.MultiPartType.FILE, resourceFile));
+    for (String path : extraResources) {
+      File file = new File(path);
+      String fileName = file.getName();
+      multiPartMap.put(fileName, new MultiPart(MultiPart.MultiPartType.FILE, file));

Review Comment:
   how do you define the mapping between config and uploaded files?
   
   For example: `spark.submit.pyFiles` -> extraResource?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org