You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Shawn Lavelle (Jira)" <ji...@apache.org> on 2022/06/03 15:19:00 UTC

[jira] [Commented] (SPARK-21012) Support glob path for resources adding to Spark

    [ https://issues.apache.org/jira/browse/SPARK-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17546011#comment-17546011 ] 

Shawn Lavelle commented on SPARK-21012:
---------------------------------------

[https://stackoverflow.com/questions/50220378/using-wildcards-in-spark-jars-when-submitting-apache-spark-jobs]

I played with this for a bit today as I was also hoping to use globs in my paths.  The existing tests don't seem to cover this use case.  I was hoping for some assistance to verify before I open a ticket.

> Support glob path for resources adding to Spark
> -----------------------------------------------
>
>                 Key: SPARK-21012
>                 URL: https://issues.apache.org/jira/browse/SPARK-21012
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Submit
>    Affects Versions: 2.2.0
>            Reporter: Saisai Shao
>            Assignee: Saisai Shao
>            Priority: Minor
>             Fix For: 2.3.0
>
>
> Current "\-\-jars (spark.jars)", "\-\-files (spark.files)", "\-\-py-files (spark.submit.pyFiles)" and "\-\-archives (spark.yarn.dist.archives)" only support non-glob path. This is OK for most of the cases, but when user requires to add more jars, files into Spark, it is too verbose to list one by one. So here propose to support glob path for adding resources.
> Also improving the code of downloading resources.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org