You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by "Thon, Ingo" <in...@siemens.com> on 2015/04/15 09:56:27 UTC
cluster utiliation when using fair scheduler
Hi,
I'm using the fair scheduler for Yarn. I have not specified any pools, so the fair-scheduler.xml is basically empty.
However, only one third of the cluster is utilized.
On the scheduler page I see a single Queue which is root and it is specified that 33.3% used
This 33.3% is independent of the number of jobs.
Currently, I have a single job running and it is claimed that for this Application pipeline:
Used Resources: <memory:196608, vCores:48>
Num Active Applications: 21
Num Pending Applications: 0
Min Resources: <memory:0, vCores:0>
Max Resources: <memory:589824, vCores:48>
Fair Share: <memory:45372, vCores:0>
The number of vCores appears to be to low as well. The number of cores should be 6x16.
Any suggestions what to check?
Cheers Ingo