Re: Difference between Capacity scheduler and Fair Scheduler

[Yufei Gu <fl...@gmail.com>]

Hi Ved,

Here is a blog for fair scheduler preemption,
https://blog.cloudera.com/blog/2018/06/yarn-fairscheduler-preemption-deep-dive/.
In terms of performance, I'm not aware of any document to compare both
schedulers. The easiest way to do it yourself is using SLS(
https://hadoop.apache.org/docs/r3.1.1/hadoop-sls/SchedulerLoadSimulator.html).
SLS focuses on the resource manager rather than anything else, like Node
managers, which is a good thing since FS and CS share the same NM
implantation. I would be surprise their performances drastically differ
from each other for the most common use cases, but their performances would
be very different in some cases.

Best,

Yufei

`This is not a contribution`


On Mon, Dec 31, 2018 at 3:36 AM Ved Gaurav <ve...@gslab.com> wrote:

> I have read some blogs about difference between them but I am not able to
> understand the difference. How both works , preemption in capacity vs
> preemption in fair scheduler , How both performs in terms of only one queue
> and hierarchical queues ,what are the practical scenarios where we can use
> them etc ?
>
> Any documentation if available will be nice.
>
> Regards,
> Ved Prakash
>