You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Petr Novak <os...@gmail.com> on 2016/02/12 09:31:16 UTC
Spark runs only on Mesos v0.21?
Hi all,
based on documenation:
"Spark 1.6.0 is designed for use with Mesos 0.21.0 and does not require any
special patches of Mesos."
We are considering Mesos for our use but this concerns me a lot. Mesos is
currently on v0.27 which we need for its Volumes feature. But Spark locks
us to 0.21 only. I understand that it is the problem that Mesos is not 1.0
yet and make breaking changes to its API. But when leading frameworks
doesn't catch up fast it beats the whole purpose of Mesos - run on one
unified platform and share resources, single framework can lock down Mesos
upgrade.
We don't want to develop our own services against Spark 0.21 API.
Is there a reason why Spark is so much behind? Does Spark actually cares
about Mesos and its support or the focus moved to YARN?
Many thanks,
Petr
Re: Spark runs only on Mesos v0.21?
Posted by Iulian Dragoș <iu...@typesafe.com>.
Spark cares about Mesos, and it's safe to run it with Mesos 0.27. 0.21 is a
minimum requirement.
On Fri, Feb 12, 2016 at 9:42 AM, Tamas Szuromi <
tamas.szuromi@odigeo.com.invalid> wrote:
> Hello Petr,
>
> We're running Spark 1.5.2 and 1.6.0 on Mesos 0.25.0 without any problem.
> We upgraded from 0.21.0 originally.
>
> cheers,
> Tamas
>
>
>
>
> On 12 February 2016 at 09:31, Petr Novak <os...@gmail.com> wrote:
>
>> Hi all,
>> based on documenation:
>>
>> "Spark 1.6.0 is designed for use with Mesos 0.21.0 and does not require
>> any special patches of Mesos."
>>
>> We are considering Mesos for our use but this concerns me a lot. Mesos is
>> currently on v0.27 which we need for its Volumes feature. But Spark locks
>> us to 0.21 only. I understand that it is the problem that Mesos is not 1.0
>> yet and make breaking changes to its API. But when leading frameworks
>> doesn't catch up fast it beats the whole purpose of Mesos - run on one
>> unified platform and share resources, single framework can lock down Mesos
>> upgrade.
>>
>> We don't want to develop our own services against Spark 0.21 API.
>>
>> Is there a reason why Spark is so much behind? Does Spark actually cares
>> about Mesos and its support or the focus moved to YARN?
>>
>> Many thanks,
>> Petr
>>
>
>
--
--
Iulian Dragos
------
Reactive Apps on the JVM
www.typesafe.com
Re: Spark runs only on Mesos v0.21?
Posted by Tamas Szuromi <ta...@odigeo.com.INVALID>.
Hello Petr,
We're running Spark 1.5.2 and 1.6.0 on Mesos 0.25.0 without any problem. We
upgraded from 0.21.0 originally.
cheers,
Tamas
On 12 February 2016 at 09:31, Petr Novak <os...@gmail.com> wrote:
> Hi all,
> based on documenation:
>
> "Spark 1.6.0 is designed for use with Mesos 0.21.0 and does not require
> any special patches of Mesos."
>
> We are considering Mesos for our use but this concerns me a lot. Mesos is
> currently on v0.27 which we need for its Volumes feature. But Spark locks
> us to 0.21 only. I understand that it is the problem that Mesos is not 1.0
> yet and make breaking changes to its API. But when leading frameworks
> doesn't catch up fast it beats the whole purpose of Mesos - run on one
> unified platform and share resources, single framework can lock down Mesos
> upgrade.
>
> We don't want to develop our own services against Spark 0.21 API.
>
> Is there a reason why Spark is so much behind? Does Spark actually cares
> about Mesos and its support or the focus moved to YARN?
>
> Many thanks,
> Petr
>