You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Gaurav Gupta <ga...@gmail.com> on 2015/03/16 23:11:09 UTC
YARN ignores host-specific resource requests
Hi,
I am trying to allocate containers on a particular host. But I don't get
the back the resources.
I am setting relaxedLocality to false and rack to null.
Any pointers?
Thanks
Gaurav
Re: YARN ignores host-specific resource requests
Posted by Gaurav Gupta <ga...@gmail.com>.
Hey Naga,
Thanks for the suggestion. The configuration works with HDP but with CDH it
doesn't work.
I tried with both Faire and Capacity Scheduler.
Here is my configuration that I have for Capacity Scheduler in CDH
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
</configuration>
Thanks
Gaurav
On Mon, Mar 16, 2015 at 8:41 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com> wrote:
> Hi Gaurav,
> If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay
> to the size of cluster.
> By default its set to -1 which make scheduler to assign the container to
> non local nodes
>
> Regards,
> Naga
> ------------------------------
> *From:* Gaurav Gupta [gaurav.gopi123@gmail.com]
> *Sent:* Tuesday, March 17, 2015 03:41
> *To:* user@hadoop.apache.org
> *Subject:* YARN ignores host-specific resource requests
>
> Hi,
> I am trying to allocate containers on a particular host. But I don't get
> the back the resources.
>
> I am setting relaxedLocality to false and rack to null.
>
> Any pointers?
>
> Thanks
> Gaurav
>
Re: YARN ignores host-specific resource requests
Posted by Gaurav Gupta <ga...@gmail.com>.
Hey Naga,
Thanks for the suggestion. The configuration works with HDP but with CDH it
doesn't work.
I tried with both Faire and Capacity Scheduler.
Here is my configuration that I have for Capacity Scheduler in CDH
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
</configuration>
Thanks
Gaurav
On Mon, Mar 16, 2015 at 8:41 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com> wrote:
> Hi Gaurav,
> If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay
> to the size of cluster.
> By default its set to -1 which make scheduler to assign the container to
> non local nodes
>
> Regards,
> Naga
> ------------------------------
> *From:* Gaurav Gupta [gaurav.gopi123@gmail.com]
> *Sent:* Tuesday, March 17, 2015 03:41
> *To:* user@hadoop.apache.org
> *Subject:* YARN ignores host-specific resource requests
>
> Hi,
> I am trying to allocate containers on a particular host. But I don't get
> the back the resources.
>
> I am setting relaxedLocality to false and rack to null.
>
> Any pointers?
>
> Thanks
> Gaurav
>
Re: YARN ignores host-specific resource requests
Posted by Gaurav Gupta <ga...@gmail.com>.
Hey Naga,
Thanks for the suggestion. The configuration works with HDP but with CDH it
doesn't work.
I tried with both Faire and Capacity Scheduler.
Here is my configuration that I have for Capacity Scheduler in CDH
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
</configuration>
Thanks
Gaurav
On Mon, Mar 16, 2015 at 8:41 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com> wrote:
> Hi Gaurav,
> If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay
> to the size of cluster.
> By default its set to -1 which make scheduler to assign the container to
> non local nodes
>
> Regards,
> Naga
> ------------------------------
> *From:* Gaurav Gupta [gaurav.gopi123@gmail.com]
> *Sent:* Tuesday, March 17, 2015 03:41
> *To:* user@hadoop.apache.org
> *Subject:* YARN ignores host-specific resource requests
>
> Hi,
> I am trying to allocate containers on a particular host. But I don't get
> the back the resources.
>
> I am setting relaxedLocality to false and rack to null.
>
> Any pointers?
>
> Thanks
> Gaurav
>
Re: YARN ignores host-specific resource requests
Posted by Gaurav Gupta <ga...@gmail.com>.
Hey Naga,
Thanks for the suggestion. The configuration works with HDP but with CDH it
doesn't work.
I tried with both Faire and Capacity Scheduler.
Here is my configuration that I have for Capacity Scheduler in CDH
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
</configuration>
Thanks
Gaurav
On Mon, Mar 16, 2015 at 8:41 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com> wrote:
> Hi Gaurav,
> If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay
> to the size of cluster.
> By default its set to -1 which make scheduler to assign the container to
> non local nodes
>
> Regards,
> Naga
> ------------------------------
> *From:* Gaurav Gupta [gaurav.gopi123@gmail.com]
> *Sent:* Tuesday, March 17, 2015 03:41
> *To:* user@hadoop.apache.org
> *Subject:* YARN ignores host-specific resource requests
>
> Hi,
> I am trying to allocate containers on a particular host. But I don't get
> the back the resources.
>
> I am setting relaxedLocality to false and rack to null.
>
> Any pointers?
>
> Thanks
> Gaurav
>
RE: YARN ignores host-specific resource requests
Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.
Hi Gaurav,
If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay to the size of cluster.
By default its set to -1 which make scheduler to assign the container to non local nodes
Regards,
Naga
________________________________
From: Gaurav Gupta [gaurav.gopi123@gmail.com]
Sent: Tuesday, March 17, 2015 03:41
To: user@hadoop.apache.org
Subject: YARN ignores host-specific resource requests
Hi,
I am trying to allocate containers on a particular host. But I don't get the back the resources.
I am setting relaxedLocality to false and rack to null.
Any pointers?
Thanks
Gaurav
RE: YARN ignores host-specific resource requests
Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.
Hi Gaurav,
If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay to the size of cluster.
By default its set to -1 which make scheduler to assign the container to non local nodes
Regards,
Naga
________________________________
From: Gaurav Gupta [gaurav.gopi123@gmail.com]
Sent: Tuesday, March 17, 2015 03:41
To: user@hadoop.apache.org
Subject: YARN ignores host-specific resource requests
Hi,
I am trying to allocate containers on a particular host. But I don't get the back the resources.
I am setting relaxedLocality to false and rack to null.
Any pointers?
Thanks
Gaurav
RE: YARN ignores host-specific resource requests
Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.
Hi Gaurav,
If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay to the size of cluster.
By default its set to -1 which make scheduler to assign the container to non local nodes
Regards,
Naga
________________________________
From: Gaurav Gupta [gaurav.gopi123@gmail.com]
Sent: Tuesday, March 17, 2015 03:41
To: user@hadoop.apache.org
Subject: YARN ignores host-specific resource requests
Hi,
I am trying to allocate containers on a particular host. But I don't get the back the resources.
I am setting relaxedLocality to false and rack to null.
Any pointers?
Thanks
Gaurav
RE: YARN ignores host-specific resource requests
Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.
Hi Gaurav,
If you are using Capacity scheduler then try configuring to yarn.scheduler.capacity.node-locality-delay to the size of cluster.
By default its set to -1 which make scheduler to assign the container to non local nodes
Regards,
Naga
________________________________
From: Gaurav Gupta [gaurav.gopi123@gmail.com]
Sent: Tuesday, March 17, 2015 03:41
To: user@hadoop.apache.org
Subject: YARN ignores host-specific resource requests
Hi,
I am trying to allocate containers on a particular host. But I don't get the back the resources.
I am setting relaxedLocality to false and rack to null.
Any pointers?
Thanks
Gaurav