You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by David Fox <da...@gmail.com> on 2016/03/08 15:38:47 UTC

YARN Fair Scheduler - Queue Label Policies

Hello,

We have heteregenous hadoop cluster and need to run some mapreduce jobs on
specific nodes (with appropriate HW). We are using currently YARN with Fair
Scheduler to provide each user with fair share of resources. Same way there
should be one queue for run jobs on dedicated/labeled nodes only.

We have not found on hadoop documentation website how to configuref Fair
Scheduler for node labels, but only for Capacity Scheduler. (
https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html
)

Is it possible to use node labels and queue label policy for Fair Scheduler?
Do you have some experiance with similar configuration?


Thanks

RE: YARN Fair Scheduler - Queue Label Policies

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

Hi David,

Labels are not yet supported in Fair Scheduler.

Regards,
+ Naga
________________________________
From: David Fox [dafox7777777@gmail.com]
Sent: Tuesday, March 08, 2016 20:08
To: user@hadoop.apache.org
Subject: YARN Fair Scheduler - Queue Label Policies

Hello,

We have heteregenous hadoop cluster and need to run some mapreduce jobs on specific nodes (with appropriate HW). We are using currently YARN with Fair Scheduler to provide each user with fair share of resources. Same way there should be one queue for run jobs on dedicated/labeled nodes only.

We have not found on hadoop documentation website how to configuref Fair Scheduler for node labels, but only for Capacity Scheduler. (https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html)

Is it possible to use node labels and queue label policy for Fair Scheduler?
Do you have some experiance with similar configuration?

Thanks

RE: YARN Fair Scheduler - Queue Label Policies

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

Hi David,

Labels are not yet supported in Fair Scheduler.

Regards,
+ Naga
________________________________
From: David Fox [dafox7777777@gmail.com]
Sent: Tuesday, March 08, 2016 20:08
To: user@hadoop.apache.org
Subject: YARN Fair Scheduler - Queue Label Policies

Hello,

We have heteregenous hadoop cluster and need to run some mapreduce jobs on specific nodes (with appropriate HW). We are using currently YARN with Fair Scheduler to provide each user with fair share of resources. Same way there should be one queue for run jobs on dedicated/labeled nodes only.

We have not found on hadoop documentation website how to configuref Fair Scheduler for node labels, but only for Capacity Scheduler. (https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html)

Is it possible to use node labels and queue label policy for Fair Scheduler?
Do you have some experiance with similar configuration?

Thanks

RE: YARN Fair Scheduler - Queue Label Policies

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

Hi David,

Labels are not yet supported in Fair Scheduler.

Regards,
+ Naga
________________________________
From: David Fox [dafox7777777@gmail.com]
Sent: Tuesday, March 08, 2016 20:08
To: user@hadoop.apache.org
Subject: YARN Fair Scheduler - Queue Label Policies

Hello,

We have heteregenous hadoop cluster and need to run some mapreduce jobs on specific nodes (with appropriate HW). We are using currently YARN with Fair Scheduler to provide each user with fair share of resources. Same way there should be one queue for run jobs on dedicated/labeled nodes only.

We have not found on hadoop documentation website how to configuref Fair Scheduler for node labels, but only for Capacity Scheduler. (https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html)

Is it possible to use node labels and queue label policy for Fair Scheduler?
Do you have some experiance with similar configuration?

Thanks

RE: YARN Fair Scheduler - Queue Label Policies

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

Hi David,

Labels are not yet supported in Fair Scheduler.

Regards,
+ Naga
________________________________
From: David Fox [dafox7777777@gmail.com]
Sent: Tuesday, March 08, 2016 20:08
To: user@hadoop.apache.org
Subject: YARN Fair Scheduler - Queue Label Policies

Hello,

We have heteregenous hadoop cluster and need to run some mapreduce jobs on specific nodes (with appropriate HW). We are using currently YARN with Fair Scheduler to provide each user with fair share of resources. Same way there should be one queue for run jobs on dedicated/labeled nodes only.

We have not found on hadoop documentation website how to configuref Fair Scheduler for node labels, but only for Capacity Scheduler. (https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html)

Is it possible to use node labels and queue label policy for Fair Scheduler?
Do you have some experiance with similar configuration?

Thanks