You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-dev@hadoop.apache.org by Prabhu Joseph <pr...@gmail.com> on 2015/12/15 04:59:24 UTC

YARN ReservedMB in GUI and hadoop list is wrong

Hi Experts,

In RM GUI, the Reserved Memory is too high and there is a instance where it
shows 5GB with no applications running. [check image attached]. I ran a
wordcount job with 8GB map opts.

hadoop job -list shows reserved memory for 1 container is 273GB.


job_1447682088038_32310 RUNNING 1450121135056 cstonedip root.dmeer NORMAL
45 *1* 137216M *279552M* 416768M
http://lgpbd1020.gso.aexp.com:8088/proxy/application_1447682088038_32310/

job_1447682088038_32337 RUNNING 1450122255021 meric root.dmeer NORMAL 48 *0*
193536M *0M* 193536M
http://lgpbd1020.gso.aexp.com:8088/proxy/application_1447682088038_32337/

job_1447682088038_32312 RUNNING 1450121158634 cstonedip root.dmeer NORMAL
33 *1* 100352M *49152M* 149504M
http://lgpbd1020.gso.aexp.com:8088/proxy/application_1447682088038_32312/

job_1447682088038_32311 RUNNING 1450121142496 cstonedip root.dmeer NORMAL
28 *1* 84992M *76800M* 161792M
http://lgpbd1020.gso.aexp.com:8088/proxy/application_1447682088038_32311/

job_1447682088038_32349 RUNNING 1450122721560 netqualrptg root.dmeer NORMAL
26 *1* 78848M *141312M* 220160M
http://lgpbd1020.gso.aexp.com:8088/proxy/application_1447682088038_32349/


We can see here for one reserved container, the memory reserved is 279552MB
( 273GB).

Total reserved memory for 4 containers are *279552M + **49152M + **76800M +
*


*141312M =  273 + 48 + 75 + 138 = 534 GB*
Please help me in figuring out what could be wrong in the cluster or is it
a known Bug.



Thanks,
Prabhu Joseph