You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by "Thon, Ingo" <in...@siemens.com> on 2015/04/15 09:56:27 UTC

cluster utiliation when using fair scheduler

Hi,

I'm using the fair scheduler for Yarn. I have not specified any pools, so the fair-scheduler.xml is basically empty.
However, only one third of the cluster is utilized.

On the scheduler page I see a single Queue which is root and it is specified that 33.3% used
This 33.3% is independent of the number of jobs.
Currently, I have a single job running and it is claimed that for this Application pipeline:
Used Resources: <memory:196608, vCores:48>
Num Active Applications:        21
Num Pending Applications:       0
Min Resources:  <memory:0, vCores:0>
Max Resources:  <memory:589824, vCores:48>
Fair Share:     <memory:45372, vCores:0>

The number of vCores appears to be to low as well. The number of cores should be 6x16.

Any suggestions what to check?

Cheers Ingo