[GitHub] [spark] antonipp commented on a diff in pull request #38376: [SPARK-40817] [Kubernetes] Do not discard remote user-specified files when launching Spark jobs on Kubernetes

[GitBox <gi...@apache.org>]

antonipp commented on code in PR #38376:
URL: https://github.com/apache/spark/pull/38376#discussion_r1026638180


##########
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala:
##########
@@ -1609,6 +1609,16 @@ class TestFileSystem extends org.apache.hadoop.fs.LocalFileSystem {
   }
 
   override def open(path: Path): FSDataInputStream = super.open(local(path))
+
+  // No-op methods
+
+  override def copyFromLocalFile(

Review Comment:
   Well, it took me a a bit of time to get to it but I finally have a working local setup and I wrote an integration test for this PR. I added it in https://github.com/apache/spark/pull/38376/commits/cbda5403b00f4e66ad4651ceda7786de1f5e1e1a
   
   It worked for me locally with:
   ```bash
   mvn integration-test -am -pl :spark-kubernetes-integration-tests_2.12 \
                           -Pkubernetes -Pkubernetes-integration-tests -Phadoop-3 \
                           -Dspark.kubernetes.test.sparkTgz=/path/to/the/tgz/built/from/this/branch \
                           -Dspark.kubernetes.test.deployMode=minikube \
                           -Dspark.kubernetes.test.imageRepo=docker.io/kubespark \
                           -Dspark.kubernetes.test.namespace=spark \
                           -Dspark.kubernetes.test.serviceAccountName=spark
   ```
   
   @dongjoon-hyun @holdenk, would appreciate your reviews as well 🙏 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org