You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Nicolae Marasoiu <ni...@adswizz.com> on 2015/11/17 15:01:18 UTC

yarn uses nodes non symetrically

Hi,


My nodes are identical, and the yarn-site.xml are identical too.

However, between slaves, one is used to the full but the other, around half, meaning: one gets 4 containers, the other gets 3 (and one of them is the app master which is quite idle), and I don't know why.


Thanks,

Nicu

Re: yarn uses nodes non symetrically

Posted by Eric Payne <er...@yahoo.com.INVALID>.
Nicolae It depends on how big your AM container is compared to the task containers. By default, the AM container size is 1.5GB and the map/reduce containers are 1GB. You can adjust these by setting yarn.app.mapreduce.am.resource.mb, mapreduce.map.memory.mb, and mapreduce.map.memory.mb. If you make them smaller, make sure you also adjust the -Xmx values for the mapreduce.*.java.opts properties as well.Thanks,
-Eric
 


      From: Nicolae Marasoiu <ni...@adswizz.com>
 To: "user@hadoop.apache.org" <us...@hadoop.apache.org> 
 Sent: Tuesday, November 17, 2015 8:01 AM
 Subject: yarn uses nodes non symetrically
   
 #yiv7260601419 #yiv7260601419 -- P {margin-top:0;margin-bottom:0;}#yiv7260601419 Hi,
My nodes are identical, and the yarn-site.xml are identical too.However, between slaves, one is used to the full but the other, around half, meaning: one gets 4 containers, the other gets 3 (and one of them is the app master which is quite idle), and I don't know why.
Thanks,Nicu

Re: yarn uses nodes non symetrically

Posted by Eric Payne <er...@yahoo.com.INVALID>.
Nicolae It depends on how big your AM container is compared to the task containers. By default, the AM container size is 1.5GB and the map/reduce containers are 1GB. You can adjust these by setting yarn.app.mapreduce.am.resource.mb, mapreduce.map.memory.mb, and mapreduce.map.memory.mb. If you make them smaller, make sure you also adjust the -Xmx values for the mapreduce.*.java.opts properties as well.Thanks,
-Eric
 


      From: Nicolae Marasoiu <ni...@adswizz.com>
 To: "user@hadoop.apache.org" <us...@hadoop.apache.org> 
 Sent: Tuesday, November 17, 2015 8:01 AM
 Subject: yarn uses nodes non symetrically
   
 #yiv7260601419 #yiv7260601419 -- P {margin-top:0;margin-bottom:0;}#yiv7260601419 Hi,
My nodes are identical, and the yarn-site.xml are identical too.However, between slaves, one is used to the full but the other, around half, meaning: one gets 4 containers, the other gets 3 (and one of them is the app master which is quite idle), and I don't know why.
Thanks,Nicu

Re: yarn uses nodes non symetrically

Posted by Eric Payne <er...@yahoo.com.INVALID>.
Nicolae It depends on how big your AM container is compared to the task containers. By default, the AM container size is 1.5GB and the map/reduce containers are 1GB. You can adjust these by setting yarn.app.mapreduce.am.resource.mb, mapreduce.map.memory.mb, and mapreduce.map.memory.mb. If you make them smaller, make sure you also adjust the -Xmx values for the mapreduce.*.java.opts properties as well.Thanks,
-Eric
 


      From: Nicolae Marasoiu <ni...@adswizz.com>
 To: "user@hadoop.apache.org" <us...@hadoop.apache.org> 
 Sent: Tuesday, November 17, 2015 8:01 AM
 Subject: yarn uses nodes non symetrically
   
 #yiv7260601419 #yiv7260601419 -- P {margin-top:0;margin-bottom:0;}#yiv7260601419 Hi,
My nodes are identical, and the yarn-site.xml are identical too.However, between slaves, one is used to the full but the other, around half, meaning: one gets 4 containers, the other gets 3 (and one of them is the app master which is quite idle), and I don't know why.
Thanks,Nicu

Re: yarn uses nodes non symetrically

Posted by Eric Payne <er...@yahoo.com.INVALID>.
Nicolae It depends on how big your AM container is compared to the task containers. By default, the AM container size is 1.5GB and the map/reduce containers are 1GB. You can adjust these by setting yarn.app.mapreduce.am.resource.mb, mapreduce.map.memory.mb, and mapreduce.map.memory.mb. If you make them smaller, make sure you also adjust the -Xmx values for the mapreduce.*.java.opts properties as well.Thanks,
-Eric
 


      From: Nicolae Marasoiu <ni...@adswizz.com>
 To: "user@hadoop.apache.org" <us...@hadoop.apache.org> 
 Sent: Tuesday, November 17, 2015 8:01 AM
 Subject: yarn uses nodes non symetrically
   
 #yiv7260601419 #yiv7260601419 -- P {margin-top:0;margin-bottom:0;}#yiv7260601419 Hi,
My nodes are identical, and the yarn-site.xml are identical too.However, between slaves, one is used to the full but the other, around half, meaning: one gets 4 containers, the other gets 3 (and one of them is the app master which is quite idle), and I don't know why.
Thanks,Nicu