You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-dev@hadoop.apache.org by Allen Zhang <al...@126.com> on 2015/12/16 15:15:18 UTC

label based scheduler for fair scheduler?

Hello All,


any enhancement JIRA for integrating label based scheduler and fair scheduler? is it planned in 2.8.0 already?


Thanks,
Allen

Re: label based scheduler for fair scheduler?

Posted by Wangda Tan <wh...@gmail.com>.

Hi Allen,

To clarify,
http://doc.mapr.com/display/MapR/Label-based+Scheduling+for+YARN+Applications#Label-basedSchedulingforYARNApplications-DefiningQueueLabelsforFairScheduler
is not in Apache Hadoop, you can take a look at Apache Hadoop's node label
doc:
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md
instead.

Thanks,
Wangda

On Fri, Dec 18, 2015 at 12:42 AM, Allen Zhang <al...@126.com> wrote:

>
>
> Hi Naga,
>
>
> do you know this in this link:
> http://doc.mapr.com/display/MapR/Label-based+Scheduling+for+YARN+Applications#Label-basedSchedulingforYARNApplications-DefiningQueueLabelsforFairScheduler
> Defining Queue Labels for Fair Scheduler
>
>
> I haven't try that yet.
>
>
> Thanks,
> Allen
>
>
>
>
>
>
>
>
>
>
> At 2015-12-18 16:10:00, "Naganarasimha G R (Naga)" <
> garlanaganarasimha@huawei.com> wrote:
> >Sorry to disappoint you !
> >Jira has not yet actively started, so there is less possibility to be
> finished in one or two weeks !.
> >
> >Regards,
> >+ Naga
> >________________________________________
> >From: Allen Zhang [allenzhang010@126.com]
> >Sent: Friday, December 18, 2015 13:35
> >To: yarn-dev@hadoop.apache.org
> >Subject: RE: label based scheduler for fair scheduler?
> >
> >Hi Naga,
> >
> >
> >thank you for you response, I know YARN-2497 is work in progress, will
> you get it closed or resolved in one or two weeks? :) I do need this
> feature in our environment.
> >
> >
> >Thanks,
> >Allen
> >
> >
> >
> >
> >
> >
> >
> >At 2015-12-18 15:57:03, "Naganarasimha G R (Naga)" <
> garlanaganarasimha@huawei.com> wrote:
> >>FYI ... YARN-2497
> >>________________________________________
> >>From: Allen Zhang [allenzhang010@126.com]
> >>Sent: Thursday, December 17, 2015 20:09
> >>To: yarn-dev@hadoop.apache.org
> >>Subject: Re: label based scheduler for fair scheduler?
> >>
> >>any JIRAs related to this? Thanks.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
> >>>I am actively designing the node labels feature for FairScheduler.
> Keeping
> >>>in mind the long-term debuggability/supportability of the scheduler, I
> >>>believe it needs a little bit of re-architecture to support node-labels
> and
> >>>hence the delay.
> >>>
> >>>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com>
> wrote:
> >>>
> >>>> Hello All,
> >>>>
> >>>>
> >>>> any enhancement JIRA for integrating label based scheduler and fair
> >>>> scheduler? is it planned in 2.8.0 already?
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Allen
>

RE: label based scheduler for fair scheduler?

Posted by Allen Zhang <al...@126.com>.


Hi Naga, 


do you know this in this link: http://doc.mapr.com/display/MapR/Label-based+Scheduling+for+YARN+Applications#Label-basedSchedulingforYARNApplications-DefiningQueueLabelsforFairScheduler 
Defining Queue Labels for Fair Scheduler


I haven't try that yet.


Thanks,
Allen










At 2015-12-18 16:10:00, "Naganarasimha G R (Naga)" <ga...@huawei.com> wrote:
>Sorry to disappoint you !  
>Jira has not yet actively started, so there is less possibility to be finished in one or two weeks !.
>
>Regards,
>+ Naga
>________________________________________
>From: Allen Zhang [allenzhang010@126.com]
>Sent: Friday, December 18, 2015 13:35
>To: yarn-dev@hadoop.apache.org
>Subject: RE: label based scheduler for fair scheduler?
>
>Hi Naga,
>
>
>thank you for you response, I know YARN-2497 is work in progress, will you get it closed or resolved in one or two weeks? :) I do need this feature in our environment.
>
>
>Thanks,
>Allen
>
>
>
>
>
>
>
>At 2015-12-18 15:57:03, "Naganarasimha G R (Naga)" <ga...@huawei.com> wrote:
>>FYI ... YARN-2497
>>________________________________________
>>From: Allen Zhang [allenzhang010@126.com]
>>Sent: Thursday, December 17, 2015 20:09
>>To: yarn-dev@hadoop.apache.org
>>Subject: Re: label based scheduler for fair scheduler?
>>
>>any JIRAs related to this? Thanks.
>>
>>
>>
>>
>>
>>
>>
>>At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>I am actively designing the node labels feature for FairScheduler. Keeping
>>>in mind the long-term debuggability/supportability of the scheduler, I
>>>believe it needs a little bit of re-architecture to support node-labels and
>>>hence the delay.
>>>
>>>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>>>
>>>> Hello All,
>>>>
>>>>
>>>> any enhancement JIRA for integrating label based scheduler and fair
>>>> scheduler? is it planned in 2.8.0 already?
>>>>
>>>>
>>>> Thanks,
>>>> Allen

RE: label based scheduler for fair scheduler?

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

Sorry to disappoint you !  
Jira has not yet actively started, so there is less possibility to be finished in one or two weeks !.

Regards,
+ Naga
________________________________________
From: Allen Zhang [allenzhang010@126.com]
Sent: Friday, December 18, 2015 13:35
To: yarn-dev@hadoop.apache.org
Subject: RE: label based scheduler for fair scheduler?

Hi Naga,


thank you for you response, I know YARN-2497 is work in progress, will you get it closed or resolved in one or two weeks? :) I do need this feature in our environment.


Thanks,
Allen







At 2015-12-18 15:57:03, "Naganarasimha G R (Naga)" <ga...@huawei.com> wrote:
>FYI ... YARN-2497
>________________________________________
>From: Allen Zhang [allenzhang010@126.com]
>Sent: Thursday, December 17, 2015 20:09
>To: yarn-dev@hadoop.apache.org
>Subject: Re: label based scheduler for fair scheduler?
>
>any JIRAs related to this? Thanks.
>
>
>
>
>
>
>
>At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>I am actively designing the node labels feature for FairScheduler. Keeping
>>in mind the long-term debuggability/supportability of the scheduler, I
>>believe it needs a little bit of re-architecture to support node-labels and
>>hence the delay.
>>
>>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>>
>>> Hello All,
>>>
>>>
>>> any enhancement JIRA for integrating label based scheduler and fair
>>> scheduler? is it planned in 2.8.0 already?
>>>
>>>
>>> Thanks,
>>> Allen

RE: label based scheduler for fair scheduler?

Posted by Allen Zhang <al...@126.com>.


Hi Naga,


thank you for you response, I know YARN-2497 is work in progress, will you get it closed or resolved in one or two weeks? :) I do need this feature in our environment.


Thanks,
Allen







At 2015-12-18 15:57:03, "Naganarasimha G R (Naga)" <ga...@huawei.com> wrote:
>FYI ... YARN-2497
>________________________________________
>From: Allen Zhang [allenzhang010@126.com]
>Sent: Thursday, December 17, 2015 20:09
>To: yarn-dev@hadoop.apache.org
>Subject: Re: label based scheduler for fair scheduler?
>
>any JIRAs related to this? Thanks.
>
>
>
>
>
>
>
>At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>I am actively designing the node labels feature for FairScheduler. Keeping
>>in mind the long-term debuggability/supportability of the scheduler, I
>>believe it needs a little bit of re-architecture to support node-labels and
>>hence the delay.
>>
>>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>>
>>> Hello All,
>>>
>>>
>>> any enhancement JIRA for integrating label based scheduler and fair
>>> scheduler? is it planned in 2.8.0 already?
>>>
>>>
>>> Thanks,
>>> Allen

RE: label based scheduler for fair scheduler?

Posted by "Naganarasimha G R (Naga)" <ga...@huawei.com>.

FYI ... YARN-2497
________________________________________
From: Allen Zhang [allenzhang010@126.com]
Sent: Thursday, December 17, 2015 20:09
To: yarn-dev@hadoop.apache.org
Subject: Re: label based scheduler for fair scheduler?

any JIRAs related to this? Thanks.

At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>I am actively designing the node labels feature for FairScheduler. Keeping
>in mind the long-term debuggability/supportability of the scheduler, I
>believe it needs a little bit of re-architecture to support node-labels and
>hence the delay.
>
>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>
>> Hello All,
>>
>>
>> any enhancement JIRA for integrating label based scheduler and fair
>> scheduler? is it planned in 2.8.0 already?
>>
>>
>> Thanks,
>> Allen

Re: label based scheduler for fair scheduler?

Posted by Allen Zhang <al...@126.com>.

any JIRAs related to this? Thanks.







At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>I am actively designing the node labels feature for FairScheduler. Keeping
>in mind the long-term debuggability/supportability of the scheduler, I
>believe it needs a little bit of re-architecture to support node-labels and
>hence the delay.
>
>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>
>> Hello All,
>>
>>
>> any enhancement JIRA for integrating label based scheduler and fair
>> scheduler? is it planned in 2.8.0 already?
>>
>>
>> Thanks,
>> Allen

how to list all label-node info?

Posted by Allen Zhang <al...@126.com>.

Hi guys,

I was wondering how to get all the label to node mapping infomation with only one YARN command.

I tried 
1. "yarn cluster -lnl" which just list all labels registered in my cluster and 
2. "yarn node -status sut-1:54321" shows the label info only for the machine I specified. If I want to know all the label infomation for each of my cluster, it's so boring ... 

any shortcut to get that?

Thanks,
Allen

At 2015-12-17 05:34:53, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>I am actively designing the node labels feature for FairScheduler. Keeping
>in mind the long-term debuggability/supportability of the scheduler, I
>believe it needs a little bit of re-architecture to support node-labels and
>hence the delay.
>
>On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:
>
>> Hello All,
>>
>>
>> any enhancement JIRA for integrating label based scheduler and fair
>> scheduler? is it planned in 2.8.0 already?
>>
>>
>> Thanks,
>> Allen

Re: label based scheduler for fair scheduler?

Posted by Karthik Kambatla <ka...@cloudera.com>.

I am actively designing the node labels feature for FairScheduler. Keeping
in mind the long-term debuggability/supportability of the scheduler, I
believe it needs a little bit of re-architecture to support node-labels and
hence the delay.

On Wed, Dec 16, 2015 at 6:15 AM, Allen Zhang <al...@126.com> wrote:

> Hello All,
>
>
> any enhancement JIRA for integrating label based scheduler and fair
> scheduler? is it planned in 2.8.0 already?
>
>
> Thanks,
> Allen