You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Josh J <jo...@gmail.com> on 2015/01/14 18:13:00 UTC
Re: dockerized spark executor on mesos?
>
> We have dockerized Spark Master and worker(s) separately and are using it
> in
> our dev environment.
Is this setup available on github or dockerhub?
On Tue, Dec 9, 2014 at 3:50 PM, Venkat Subramanian <vs...@gmail.com>
wrote:
> We have dockerized Spark Master and worker(s) separately and are using it
> in
> our dev environment. We don't use Mesos though, running it in Standalone
> mode, but adding Mesos should not be that difficult I think.
>
> Regards
>
> Venkat
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/dockerized-spark-executor-on-mesos-tp20276p20603.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: dockerized spark executor on mesos?
Posted by Tim Chen <ti...@mesosphere.io>.
Just throwing this out here, there is existing PR to add docker support for
spark framework to launch executors with docker image.
https://github.com/apache/spark/pull/3074
Hopefully this will be merged sometime.
Tim
On Thu, Jan 15, 2015 at 9:18 AM, Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:
> The AMPLab maintains a bunch of Docker files for Spark here:
> https://github.com/amplab/docker-scripts
>
> Hasn't been updated since 1.0.0, but might be a good starting point.
>
> On Wed Jan 14 2015 at 12:14:13 PM Josh J <jo...@gmail.com> wrote:
>
>> We have dockerized Spark Master and worker(s) separately and are using it
>>> in
>>> our dev environment.
>>
>>
>> Is this setup available on github or dockerhub?
>>
>> On Tue, Dec 9, 2014 at 3:50 PM, Venkat Subramanian <vs...@gmail.com>
>> wrote:
>>
>>> We have dockerized Spark Master and worker(s) separately and are using
>>> it in
>>> our dev environment. We don't use Mesos though, running it in Standalone
>>> mode, but adding Mesos should not be that difficult I think.
>>>
>>> Regards
>>>
>>> Venkat
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/dockerized-spark-executor-on-mesos-tp20276p20603.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>>
Re: dockerized spark executor on mesos?
Posted by Nicholas Chammas <ni...@gmail.com>.
The AMPLab maintains a bunch of Docker files for Spark here:
https://github.com/amplab/docker-scripts
Hasn't been updated since 1.0.0, but might be a good starting point.
On Wed Jan 14 2015 at 12:14:13 PM Josh J <jo...@gmail.com> wrote:
> We have dockerized Spark Master and worker(s) separately and are using it
>> in
>> our dev environment.
>
>
> Is this setup available on github or dockerhub?
>
> On Tue, Dec 9, 2014 at 3:50 PM, Venkat Subramanian <vs...@gmail.com>
> wrote:
>
>> We have dockerized Spark Master and worker(s) separately and are using it
>> in
>> our dev environment. We don't use Mesos though, running it in Standalone
>> mode, but adding Mesos should not be that difficult I think.
>>
>> Regards
>>
>> Venkat
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/dockerized-spark-executor-on-mesos-tp20276p20603.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>