Mesos resource sharing between frameworks

[Stratos Dimopoulos <st...@gmail.com>]

Hi All,

Is there a way to dynamically fair-share resources between the frameworks
that running on top of Mesos? What I would like to do is when running for
example MapReduce and Spark together, allocate 50% of the resources to each
of them (if they need them) and when running MapReduce, Spark, Storm 33% of
the resources to each.

What I observe is that when I am running Spark in fine grained mode and
MapReduce together, MR will gradually take over all the resources as the
task trackers of Spark are finished and in the meantime before Spark
manages to stage new TaskTrackers Hadoop gets more and more resources until
it actually takes over the whole cluster. The opposite happens with I run
Spark in coarse grained mode. In this case Spark is faster staging the task
trackers and it will manage to get 100% of the cluster before Hadoop gets
any.

I checked this:
http://mesos.apache.org/documentation/latest/framework-rate-limiting/
that might help but what I would really want to is to share the resources
equally between registered frameworks. Any ideas?

thanks!

Stratos

Fwd: Mesos resource sharing between frameworks

[Stratos Dimopoulos <st...@gmail.com>]

Following up my earlier question...Do you have any suggestion on the best
way to configure Spark and Hadoop in order to equally share resources?

I see that for Hadoop I have the: mapred.mesos.total.map.slots.minimum
and mapred.mesos.total.reduce.slots.minimum but I cannot find a similar
property for the maximum slots.
Moreover I am not aware of similar configuration properties at Spark . (Is
this the most relevant? spark.scheduler.minRegisteredResourcesRatio)

Are you aware of a place that I can have documented a full list of
available configurations for both frameworks? So far my source is this:
https://github.com/mesos/hadoop/blob/master/configuration.md for hadoop and
this: http://spark.apache.org/docs/1.2.1/configuration.html#spark-properties
for spark but they don't include all the properties.

If you are not aware of any documentation maybe suggestion on the parts of
code that I could check to find these configuration parameters?

thanks,
Stratos

On Thu, Mar 19, 2015 at 7:32 PM, Stratos Dimopoulos <
stratos.dimopoulos@gmail.com> wrote:

> Thank you for the answer Benjamin!
>
> On Thu, Mar 19, 2015 at 6:05 PM, Benjamin Mahler <
> benjamin.mahler@gmail.com> wrote:
>
>> Guaranteeing fairness in such situations requires pre-emption of running
>> tasks/executors, which is not yet provided in mesos.
>>
>> For now, you can try reserving a minimum amount of resources for each
>> framework, note however that this may reduce your efficiency if you
>> over-estimate the minimum reservation needed. This is because reserved
>> resources cannot be used by other roles. In the future, to avoid the
>> efficiency issue, other roles will be able to use reserved resources in a
>> pre-emptable way.
>>
>> On Thu, Mar 19, 2015 at 3:39 PM, Stratos Dimopoulos <
>> stratos.dimopoulos@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Is there a way to dynamically fair-share resources between the
>>> frameworks that running on top of Mesos? What I would like to do is when
>>> running for example MapReduce and Spark together, allocate 50% of the
>>> resources to each of them (if they need them) and when running MapReduce,
>>> Spark, Storm 33% of the resources to each.
>>>
>>> What I observe is that when I am running Spark in fine grained mode and
>>> MapReduce together, MR will gradually take over all the resources as the
>>> task trackers of Spark are finished and in the meantime before Spark
>>> manages to stage new TaskTrackers Hadoop gets more and more resources until
>>> it actually takes over the whole cluster. The opposite happens with I run
>>> Spark in coarse grained mode. In this case Spark is faster staging the task
>>> trackers and it will manage to get 100% of the cluster before Hadoop gets
>>> any.
>>>
>>> I checked this:
>>> http://mesos.apache.org/documentation/latest/framework-rate-limiting/
>>> that might help but what I would really want to is to share the
>>> resources equally between registered frameworks. Any ideas?
>>>
>>> thanks!
>>>
>>> Stratos
>>>
>>
>>
>

Re: Mesos resource sharing between frameworks

[Benjamin Mahler <be...@gmail.com>]

Guaranteeing fairness in such situations requires pre-emption of running
tasks/executors, which is not yet provided in mesos.

For now, you can try reserving a minimum amount of resources for each
framework, note however that this may reduce your efficiency if you
over-estimate the minimum reservation needed. This is because reserved
resources cannot be used by other roles. In the future, to avoid the
efficiency issue, other roles will be able to use reserved resources in a
pre-emptable way.

On Thu, Mar 19, 2015 at 3:39 PM, Stratos Dimopoulos <
stratos.dimopoulos@gmail.com> wrote:

> Hi All,
>
> Is there a way to dynamically fair-share resources between the frameworks
> that running on top of Mesos? What I would like to do is when running for
> example MapReduce and Spark together, allocate 50% of the resources to each
> of them (if they need them) and when running MapReduce, Spark, Storm 33% of
> the resources to each.
>
> What I observe is that when I am running Spark in fine grained mode and
> MapReduce together, MR will gradually take over all the resources as the
> task trackers of Spark are finished and in the meantime before Spark
> manages to stage new TaskTrackers Hadoop gets more and more resources until
> it actually takes over the whole cluster. The opposite happens with I run
> Spark in coarse grained mode. In this case Spark is faster staging the task
> trackers and it will manage to get 100% of the cluster before Hadoop gets
> any.
>
> I checked this:
> http://mesos.apache.org/documentation/latest/framework-rate-limiting/
> that might help but what I would really want to is to share the resources
> equally between registered frameworks. Any ideas?
>
> thanks!
>
> Stratos
>