You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Ben Mahler <be...@gmail.com> on 2013/07/02 20:50:57 UTC
Re: Review Request 11976: Allow specification of mininum slot count.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11976/#review22677
-----------------------------------------------------------
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11976/#comment46361>
How about:
mapred.mesos.total.map.slots.minimum
mapred.mesos.total.reduce.slots.minimum
And for these added variables, can you add the default values in mapred-site.xml.patch and add a human friendly comment for others to know how to use the knob?
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11976/#comment46365>
This math is getting a bit complicated.
It seems like the goal of the minimum here is to have a minimum amount of idle slots in the cluster? If so, we should likely update the configuration names to be:
mapred.mesos.total.idle.map.slots.minimum
mapred.mesos.total.idle.reduce.slots.minimum
- Ben Mahler
On June 19, 2013, 5:50 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11976/
> -----------------------------------------------------------
>
> (Updated June 19, 2013, 5:50 p.m.)
>
>
> Review request for mesos.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Allow specification of mininum slot count.
>
> To ensure Hadoop jobs begin promptly, we can specify a minimum number of
> 'hot slots' to be available for use. This addresses the TaskTracker
> spin up delay that exists with Hadoop on Mesos. This can be a nuisance
> with lower latency applications, such as ad-hoc Hive queries.
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java c5a63bc7a172c44cc11d6bdd6b2eb4bc3df35cb7
>
> Diff: https://reviews.apache.org/r/11976/diff/
>
>
> Testing
> -------
>
> make check && make hadoop
>
>
> Thanks,
>
> Brenden Matthews
>
>