You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Chris Heller (JIRA)" <ji...@apache.org> on 2014/11/03 14:02:34 UTC
[jira] [Updated] (SPARK-2691) Allow Spark on Mesos to be launched
with Docker
[ https://issues.apache.org/jira/browse/SPARK-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Heller updated SPARK-2691:
--------------------------------
Attachment: spark-docker.patch
Here is the patch for the changes needed to support docker images in the fine grained backend.
The approach taken here was to just populate the DockerInfo of the container info if some properties were set in the properties file. This has no support for versions of mesos which do not support docker, so it is very incomplete.
Additionally there is only support for image name, and volumes.
For volumes, you just provide a string, which takes a value similar in form to the argument to 'docker run -v', ie. it is a comma separated list of [host:]container[:mode] options. I think this is sufficient, it parallels the command line, and so should be familiar.
> Allow Spark on Mesos to be launched with Docker
> -----------------------------------------------
>
> Key: SPARK-2691
> URL: https://issues.apache.org/jira/browse/SPARK-2691
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Assignee: Timothy Chen
> Labels: mesos
> Attachments: spark-docker.patch
>
>
> Currently to launch Spark with Mesos one must upload a tarball and specifiy the executor URI to be passed in that is to be downloaded on each slave or even each execution depending coarse mode or not.
> We want to make Spark able to support launching Executors via a Docker image that utilizes the recent Docker and Mesos integration work.
> With the recent integration Spark can simply specify a Docker image and options that is needed and it should continue to work as-is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org