You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Ben Mahler <be...@gmail.com> on 2013/07/02 20:50:57 UTC

Re: Review Request 11976: Allow specification of mininum slot count.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11976/#review22677
-----------------------------------------------------------



hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11976/#comment46361>

    How about:
    mapred.mesos.total.map.slots.minimum
    mapred.mesos.total.reduce.slots.minimum
    
    And for these added variables, can you add the default values in mapred-site.xml.patch and add a human friendly comment for others to know how to use the knob?



hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11976/#comment46365>

    This math is getting a bit complicated.
    
    It seems like the goal of the minimum here is to have a minimum amount of idle slots in the cluster? If so, we should likely update the configuration names to be:
    
    mapred.mesos.total.idle.map.slots.minimum
    mapred.mesos.total.idle.reduce.slots.minimum


- Ben Mahler


On June 19, 2013, 5:50 p.m., Brenden Matthews wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11976/
> -----------------------------------------------------------
> 
> (Updated June 19, 2013, 5:50 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Allow specification of mininum slot count.
> 
> To ensure Hadoop jobs begin promptly, we can specify a minimum number of
> 'hot slots' to be available for use.  This addresses the TaskTracker
> spin up delay that exists with Hadoop on Mesos.  This can be a nuisance
> with lower latency applications, such as ad-hoc Hive queries.
> 
> 
> Diffs
> -----
> 
>   hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java c5a63bc7a172c44cc11d6bdd6b2eb4bc3df35cb7 
> 
> Diff: https://reviews.apache.org/r/11976/diff/
> 
> 
> Testing
> -------
> 
> make check && make hadoop
> 
> 
> Thanks,
> 
> Brenden Matthews
> 
>