[jira] [Assigned] (SPARK-14602) [YARN+Windows] Setting SPARK_YARN_CACHE_FILES exceeds command line length limit on Windows

["Apache Spark (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-14602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-14602:
------------------------------------

    Assignee:     (was: Apache Spark)

> [YARN+Windows] Setting SPARK_YARN_CACHE_FILES exceeds command line length limit on Windows
> ------------------------------------------------------------------------------------------
>
>                 Key: SPARK-14602
>                 URL: https://issues.apache.org/jira/browse/SPARK-14602
>             Project: Spark
>          Issue Type: Bug
>          Components: Deploy, Spark Submit, Windows, YARN
>    Affects Versions: 2.0.0
>         Environment: YARN on Windows
>            Reporter: Sebastian Kochman
>
> After change https://issues.apache.org/jira/browse/SPARK-11157, which removed a single large Spark assembly in favor of multiple small jars, when you try to submit a Spark app to YARN on Windows (using spark-submit.cmd), the app fails with the following error:
> Diagnostics: The command line has a length of 12046 exceeds maximum allowed length of 8191. Command starts with: @set SPARK_YARN_CACHE_FILES=[...]/.sparkStaging/application_[...] Failing this attempt. Failing the application.
> So basically a large number of jars needed for staging in YARN causes exceeding Windows command line length limit.
> Please see more details in the discussion here:
> https://issues.apache.org/jira/browse/SPARK-11157?focusedCommentId=15238151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15238151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org