You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by 牛兆捷 <nz...@gmail.com> on 2013/11/21 04:43:55 UTC
Yarn resource request
Currently, resource requests for the same priority and locality are
expected to all be the same size, how will Application Master do for
different size? use different requests?
--
*Regards,*
*Zhaojie*
Re: Yarn resource request
Posted by 牛兆捷 <nz...@gmail.com>.
But I think there is some problems.
For example, the FIFO scheduler, the big memory request of a job may starve
as the small memory requests of other jobs can always be fulfilled. Is it
right?
2013/11/22 Bikas Saha <bi...@hortonworks.com>
> Yes.
>
> -----Original Message-----
> From: 牛兆捷 [mailto:nzjemail@gmail.com]
> Sent: Thursday, November 21, 2013 12:54 AM
> To: yarn-dev@hadoop.apache.org
> Subject: Re: Yarn resource request
>
> Then, if one application master request for two different size container
> with different priority, the resource manager will reserve the resource
> for the higher priority container even the requirement of smaller
> container can be met.
>
>
> 2013/11/21 Bikas Saha <bi...@hortonworks.com>
>
> > That’s a known limitation of the current implementation of schedulers
> > in YARN. The API allows different requests of different sizes within
> > the same priority but the implementations of the schedulers do not
> support this.
> > For different sizes, one can either use different priorities or use
> > the highest size amongst the sizes within a priority.
> >
> > Bikas
> >
> > -----Original Message-----
> > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > Sent: Wednesday, November 20, 2013 7:44 PM
> > To: yarn-dev@hadoop.apache.org
> > Subject: Yarn resource request
> >
> > Currently, resource requests for the same priority and locality are
> > expected to all be the same size, how will Application Master do for
> > different size? use different requests?
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or
> > entity to which it is addressed and may contain information that is
> > confidential, privileged and exempt from disclosure under applicable
> > law. If the reader of this message is not the intended recipient, you
> > are hereby notified that any printing, copying, dissemination,
> > distribution, disclosure or forwarding of this communication is
> > strictly prohibited. If you have received this communication in error,
> > please contact the sender immediately and delete it from your system.
> Thank You.
> >
>
>
>
> --
> *Regards,*
> *Zhaojie*
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
*Regards,*
*Zhaojie*
RE: Yarn resource request
Posted by Bikas Saha <bi...@hortonworks.com>.
Yes.
-----Original Message-----
From: 牛兆捷 [mailto:nzjemail@gmail.com]
Sent: Thursday, November 21, 2013 12:54 AM
To: yarn-dev@hadoop.apache.org
Subject: Re: Yarn resource request
Then, if one application master request for two different size container
with different priority, the resource manager will reserve the resource
for the higher priority container even the requirement of smaller
container can be met.
2013/11/21 Bikas Saha <bi...@hortonworks.com>
> That’s a known limitation of the current implementation of schedulers
> in YARN. The API allows different requests of different sizes within
> the same priority but the implementations of the schedulers do not
support this.
> For different sizes, one can either use different priorities or use
> the highest size amongst the sizes within a priority.
>
> Bikas
>
> -----Original Message-----
> From: 牛兆捷 [mailto:nzjemail@gmail.com]
> Sent: Wednesday, November 20, 2013 7:44 PM
> To: yarn-dev@hadoop.apache.org
> Subject: Yarn resource request
>
> Currently, resource requests for the same priority and locality are
> expected to all be the same size, how will Application Master do for
> different size? use different requests?
>
> --
> *Regards,*
> *Zhaojie*
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
> entity to which it is addressed and may contain information that is
> confidential, privileged and exempt from disclosure under applicable
> law. If the reader of this message is not the intended recipient, you
> are hereby notified that any printing, copying, dissemination,
> distribution, disclosure or forwarding of this communication is
> strictly prohibited. If you have received this communication in error,
> please contact the sender immediately and delete it from your system.
Thank You.
>
--
*Regards,*
*Zhaojie*
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Yarn resource request
Posted by 牛兆捷 <nz...@gmail.com>.
Then, if one application master request for two different size container
with different priority, the resource manager will reserve the resource for
the higher priority container even the requirement of smaller container can
be met.
2013/11/21 Bikas Saha <bi...@hortonworks.com>
> That’s a known limitation of the current implementation of schedulers in
> YARN. The API allows different requests of different sizes within the same
> priority but the implementations of the schedulers do not support this.
> For different sizes, one can either use different priorities or use the
> highest size amongst the sizes within a priority.
>
> Bikas
>
> -----Original Message-----
> From: 牛兆捷 [mailto:nzjemail@gmail.com]
> Sent: Wednesday, November 20, 2013 7:44 PM
> To: yarn-dev@hadoop.apache.org
> Subject: Yarn resource request
>
> Currently, resource requests for the same priority and locality are
> expected to all be the same size, how will Application Master do for
> different size? use different requests?
>
> --
> *Regards,*
> *Zhaojie*
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
*Regards,*
*Zhaojie*
Re: Yarn resource request
Posted by Steve Loughran <st...@hortonworks.com>.
I'm trying to track which servers aren't running a copy of my container
(HBase RS, Accumulo Tablet), yet were running them recently -so I keep a
list of recently used servers, write that to disk after changes, on startup
reread the latest re-readable version and rebuild that list.
I want to know which requests for a node were satisfied with a different
node -and so the the node I asked for can be added to the list of
recent-yet-available servers as the request is no longer outstanding, -but
instead satisfied elsewhere
full code is here, check it out and build it
https://github.com/hortonworks/hoya
I do plan to push some of this code into Hadoop common & hadoop YARN, but
am keeping it out so I can evolve it fast & find problems from everyday
use. Especially
1. Generic YARN service launcher
https://github.com/hortonworks/hoya/tree/master/hoya-core/src/main/java/org/apache/hadoop/yarn/service/launcher
2. Pool of services with different lifecycles, and some other operations
(delayable event callback, release RPC reference on stop(), exec process
https://github.com/hortonworks/hoya/tree/master/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service
If you are writing complex YARN apps, I'd also point you at the newly
incubated Apache Twill project
On 21 November 2013 11:36, 牛兆捷 <nz...@gmail.com> wrote:
> On receive node update event, scheduler will assign resources of this node
> to containers, isn't it?
>
> Why you say we can not know which one is satisfied?
>
> I do not know what your code do? Testing resource request of response of
> resource manager? How to run it?
>
>
> 2013/11/21 Steve Loughran <st...@hortonworks.com>
>
> > On 21 November 2013 09:04, 牛兆捷 <nz...@gmail.com> wrote:
> >
> > > Another question.
> > >
> > > For two request with the same priority, which one will be chosen?
> > >
> >
> > No way to know. An equally interesting question is : how do you know
> which
> > one has been satisfied, and which one is oustanding?
> >
> > I spent some time trying to do some of this for a service where I'd
> request
> > some instances on specific nodes, some anywhere, and tried to see if I
> > could determine which response satisified which request
> >
> >
> >
> https://github.com/hortonworks/hoya/blob/master/src/site/markdown/rolehistory.md
> >
> > This all lives in a state library, which handles AMClientAsync events,
> > generates lists of actions, but doesn't talk to the AM itself
> >
> >
> >
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state
> >
> > This is matched by an in-VM mock YARN service
> >
> >
> >
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/test/groovy/org/apache/hadoop/hoya/yarn/model/mock
> >
> > I'd recommend this model-controller architecture purely for its
> testability
> >
> > Returning to request tracking I ended up doing was keeping a map of
> > outstanding requests, and when they arrived I'd remove them from the
> list.
> > When every outstanding request had been satisfied, I'd clear the list of
> > any outstanding entries
> >
> >
> >
> https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state/OutstandingRequestTracker.java
> >
> >
> >
> >
> > >
> > >
> > > 2013/11/21 Bikas Saha <bi...@hortonworks.com>
> > >
> > > > That’s a known limitation of the current implementation of schedulers
> > in
> > > > YARN. The API allows different requests of different sizes within the
> > > same
> > > > priority but the implementations of the schedulers do not support
> this.
> > > > For different sizes, one can either use different priorities or use
> the
> > > > highest size amongst the sizes within a priority.
> > > >
> > > > Bikas
> > > >
> > > > -----Original Message-----
> > > > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > > > Sent: Wednesday, November 20, 2013 7:44 PM
> > > > To: yarn-dev@hadoop.apache.org
> > > > Subject: Yarn resource request
> > > >
> > > > Currently, resource requests for the same priority and locality are
> > > > expected to all be the same size, how will Application Master do for
> > > > different size? use different requests?
> > > >
> > > > --
> > > > *Regards,*
> > > > *Zhaojie*
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> > >
> > >
> > > --
> > > *Regards,*
> > > *Zhaojie*
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> *Regards,*
> *Zhaojie*
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Yarn resource request
Posted by 牛兆捷 <nz...@gmail.com>.
On receive node update event, scheduler will assign resources of this node
to containers, isn't it?
Why you say we can not know which one is satisfied?
I do not know what your code do? Testing resource request of response of
resource manager? How to run it?
2013/11/21 Steve Loughran <st...@hortonworks.com>
> On 21 November 2013 09:04, 牛兆捷 <nz...@gmail.com> wrote:
>
> > Another question.
> >
> > For two request with the same priority, which one will be chosen?
> >
>
> No way to know. An equally interesting question is : how do you know which
> one has been satisfied, and which one is oustanding?
>
> I spent some time trying to do some of this for a service where I'd request
> some instances on specific nodes, some anywhere, and tried to see if I
> could determine which response satisified which request
>
>
> https://github.com/hortonworks/hoya/blob/master/src/site/markdown/rolehistory.md
>
> This all lives in a state library, which handles AMClientAsync events,
> generates lists of actions, but doesn't talk to the AM itself
>
>
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state
>
> This is matched by an in-VM mock YARN service
>
>
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/test/groovy/org/apache/hadoop/hoya/yarn/model/mock
>
> I'd recommend this model-controller architecture purely for its testability
>
> Returning to request tracking I ended up doing was keeping a map of
> outstanding requests, and when they arrived I'd remove them from the list.
> When every outstanding request had been satisfied, I'd clear the list of
> any outstanding entries
>
>
> https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state/OutstandingRequestTracker.java
>
>
>
>
> >
> >
> > 2013/11/21 Bikas Saha <bi...@hortonworks.com>
> >
> > > That’s a known limitation of the current implementation of schedulers
> in
> > > YARN. The API allows different requests of different sizes within the
> > same
> > > priority but the implementations of the schedulers do not support this.
> > > For different sizes, one can either use different priorities or use the
> > > highest size amongst the sizes within a priority.
> > >
> > > Bikas
> > >
> > > -----Original Message-----
> > > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > > Sent: Wednesday, November 20, 2013 7:44 PM
> > > To: yarn-dev@hadoop.apache.org
> > > Subject: Yarn resource request
> > >
> > > Currently, resource requests for the same priority and locality are
> > > expected to all be the same size, how will Application Master do for
> > > different size? use different requests?
> > >
> > > --
> > > *Regards,*
> > > *Zhaojie*
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> >
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
*Regards,*
*Zhaojie*
Re: Yarn resource request
Posted by Steve Loughran <st...@hortonworks.com>.
On 21 November 2013 09:04, 牛兆捷 <nz...@gmail.com> wrote:
> Another question.
>
> For two request with the same priority, which one will be chosen?
>
No way to know. An equally interesting question is : how do you know which
one has been satisfied, and which one is oustanding?
I spent some time trying to do some of this for a service where I'd request
some instances on specific nodes, some anywhere, and tried to see if I
could determine which response satisified which request
https://github.com/hortonworks/hoya/blob/master/src/site/markdown/rolehistory.md
This all lives in a state library, which handles AMClientAsync events,
generates lists of actions, but doesn't talk to the AM itself
https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state
This is matched by an in-VM mock YARN service
https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/test/groovy/org/apache/hadoop/hoya/yarn/model/mock
I'd recommend this model-controller architecture purely for its testability
Returning to request tracking I ended up doing was keeping a map of
outstanding requests, and when they arrived I'd remove them from the list.
When every outstanding request had been satisfied, I'd clear the list of
any outstanding entries
https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state/OutstandingRequestTracker.java
>
>
> 2013/11/21 Bikas Saha <bi...@hortonworks.com>
>
> > That’s a known limitation of the current implementation of schedulers in
> > YARN. The API allows different requests of different sizes within the
> same
> > priority but the implementations of the schedulers do not support this.
> > For different sizes, one can either use different priorities or use the
> > highest size amongst the sizes within a priority.
> >
> > Bikas
> >
> > -----Original Message-----
> > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > Sent: Wednesday, November 20, 2013 7:44 PM
> > To: yarn-dev@hadoop.apache.org
> > Subject: Yarn resource request
> >
> > Currently, resource requests for the same priority and locality are
> > expected to all be the same size, how will Application Master do for
> > different size? use different requests?
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> *Regards,*
> *Zhaojie*
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Yarn resource request
Posted by 牛兆捷 <nz...@gmail.com>.
Another question.
For two request with the same priority, which one will be chosen?
2013/11/21 Bikas Saha <bi...@hortonworks.com>
> That’s a known limitation of the current implementation of schedulers in
> YARN. The API allows different requests of different sizes within the same
> priority but the implementations of the schedulers do not support this.
> For different sizes, one can either use different priorities or use the
> highest size amongst the sizes within a priority.
>
> Bikas
>
> -----Original Message-----
> From: 牛兆捷 [mailto:nzjemail@gmail.com]
> Sent: Wednesday, November 20, 2013 7:44 PM
> To: yarn-dev@hadoop.apache.org
> Subject: Yarn resource request
>
> Currently, resource requests for the same priority and locality are
> expected to all be the same size, how will Application Master do for
> different size? use different requests?
>
> --
> *Regards,*
> *Zhaojie*
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
*Regards,*
*Zhaojie*
RE: Yarn resource request
Posted by Bikas Saha <bi...@hortonworks.com>.
That’s a known limitation of the current implementation of schedulers in
YARN. The API allows different requests of different sizes within the same
priority but the implementations of the schedulers do not support this.
For different sizes, one can either use different priorities or use the
highest size amongst the sizes within a priority.
Bikas
-----Original Message-----
From: 牛兆捷 [mailto:nzjemail@gmail.com]
Sent: Wednesday, November 20, 2013 7:44 PM
To: yarn-dev@hadoop.apache.org
Subject: Yarn resource request
Currently, resource requests for the same priority and locality are
expected to all be the same size, how will Application Master do for
different size? use different requests?
--
*Regards,*
*Zhaojie*
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.