[GitHub] [spark] srowen commented on a change in pull request #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

[GitBox <gi...@apache.org>]

srowen commented on a change in pull request #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system
URL: https://github.com/apache/spark/pull/25910#discussion_r334512476
 
 

 ##########
 File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
 ##########
 @@ -216,20 +218,27 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
       val uri = new URI(primaryResource)
       val uriScheme = uri.getScheme()
 
-      uriScheme match {
-        case "file" =>
-          try {
+      try {
+        uriScheme match {
+          case "file" =>
             Utils.tryWithResource(new JarFile(uri.getPath)) { jar =>
               // Note that this might still return null if no main-class is set; we catch that later
               mainClass = jar.getManifest.getMainAttributes.getValue("Main-Class")
             }
-          } catch {
-            case _: Exception =>
-              error(s"Cannot load main class from JAR $primaryResource")
-          }
-        case _ =>
+          case _ =>
+            val sparkConf = new SparkConf()
+            sparkConf.setAll(sparkProperties)
+            val fs = FileSystem.get(uri, SparkHadoopUtil.newConfiguration(sparkConf))
+
+            Utils.tryWithResource(new JarInputStream(fs.open(new Path(primaryResource)))) { jar =>
 
 Review comment:
   @igozali have you had a chance to take another look at this?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org