You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Kousuke Saruta (Jira)" <ji...@apache.org> on 2020/09/01 20:06:00 UTC

[jira] [Created] (SPARK-32773) The behavior of listJars and listFiles is not consistent with YARN and other cluster managers

Kousuke Saruta created SPARK-32773:
--------------------------------------

             Summary: The behavior of listJars and listFiles is not consistent with YARN and other cluster managers
                 Key: SPARK-32773
                 URL: https://issues.apache.org/jira/browse/SPARK-32773
             Project: Spark
          Issue Type: Bug
          Components: YARN
    Affects Versions: 3.1.0
            Reporter: Kousuke Saruta
            Assignee: Kousuke Saruta


Jars/files specified with --jars/--files options are listed by sc.listJars and listFiles except when we run apps on YARN.

If we run apps not on YARN, those files are served by the embedded file server in the driver and listJars/listFiles list the served files.

But with YARN, such files specified by the options are not served by the embedded file server so listJars and listFiles don't list them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org