You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ashic Mahtab (JIRA)" <ji...@apache.org> on 2016/07/04 16:33:11 UTC
[jira] [Commented] (SPARK-16364) Allow spark-submit to upload jars
to nodes in cluster mode
[ https://issues.apache.org/jira/browse/SPARK-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361556#comment-15361556 ]
Ashic Mahtab commented on SPARK-16364:
--------------------------------------
Would it be more reasonable to have spark-submit upload the jar to the master, which would expose it over http? Seeing that the master already runs a web interface, it can then double up as a file server for the jars. Perhaps this could be controlled with a flag?
> Allow spark-submit to upload jars to nodes in cluster mode
> ----------------------------------------------------------
>
> Key: SPARK-16364
> URL: https://issues.apache.org/jira/browse/SPARK-16364
> Project: Spark
> Issue Type: Improvement
> Components: Spark Submit
> Affects Versions: 1.6.2
> Environment: Stand-Alone cluster
> Reporter: Ashic Mahtab
> Priority: Minor
>
> Currently, it seems that with a stand-alone cluster, running spark-submit with --deploy-mode cluster requires the target jar to be visible from all nodes. It would be convenient if spark-submit had the option to copy the jar to the relevant nodes, instead of requiring it to already by present.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org