You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by Marcin Tustin <mt...@handybook.com> on 2016/04/06 20:31:06 UTC

Configuring capacity scheduling in the face of YARN-3216

Many people will be aware of YARN-3216 (
https://issues.apache.org/jira/browse/YARN-3216). In short, the capacity
scheduler calculates all values only off the default label and queue.

My colleague Dave and I (and really mostly Dave!) configured our cluster to
work pretty much as desired even with that bug extant. I've written up how
we did it here:
https://medium.com/handy-tech/practical-capacity-scheduling-with-yarn-28548ae4fb88#.5ihha0oqy

The short version is: keep the bulk of capacity under the default
queue/node and get a bit clever with calculating limits.

Marcin

-- 
Want to work at Handy? Check out our culture deck and open roles 
<http://www.handy.com/careers>
Latest news <http://www.handy.com/press> at Handy
Handy just raised $50m 
<http://venturebeat.com/2015/11/02/on-demand-home-service-handy-raises-50m-in-round-led-by-fidelity/> led 
by Fidelity