You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Steve Loughran <st...@hortonworks.com> on 2013/11/21 15:06:03 UTC
API to determine max limits (vcores, vram, &c from the client?)
Is there any way to determine the max resources you can ask for from an RM
queue before you actually create the AM? I know the AM gets that data once
it's been created, but without knowing those limits client side, you can't
tune the requirements of the AM itself
If not, I'm going to file a JIRA to add this info to QueueInfo, so a call
to ApplicationClientProtocol.getQueueInfo() will let a YARN client find it
out.
-steve
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: API to determine max limits (vcores, vram, &c from the client?)
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
ah, make sense.
(I didn't run into that as I'm mostly using unmanaged AMs)
On Thu, Nov 21, 2013 at 8:46 AM, Steve Loughran <st...@hortonworks.com>wrote:
> I want to know the limits I can use when creating an AM, such as ask for
> one with the max allowable RAM. I can do that for any container the AM asks
> for, but not for the initial AM/app request
>
>
> On 21 November 2013 14:37, Alejandro Abdelnur <tu...@cloudera.com> wrote:
>
> > Steve, what kind of tuning do you need to do based on the max resources
> > that would affect the creation/configuration of the Yarn client itself?
> >
> > Thanks.
> >
> >
> > On Thu, Nov 21, 2013 at 6:06 AM, Steve Loughran <stevel@hortonworks.com
> > >wrote:
> >
> > > Is there any way to determine the max resources you can ask for from an
> > RM
> > > queue before you actually create the AM? I know the AM gets that data
> > once
> > > it's been created, but without knowing those limits client side, you
> > can't
> > > tune the requirements of the AM itself
> > >
> > > If not, I'm going to file a JIRA to add this info to QueueInfo, so a
> call
> > > to ApplicationClientProtocol.getQueueInfo() will let a YARN client find
> > it
> > > out.
> > >
> > > -steve
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> >
> >
> > --
> > Alejandro
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
Alejandro
Re: API to determine max limits (vcores, vram, &c from the client?)
Posted by Steve Loughran <st...@hortonworks.com>.
I want to know the limits I can use when creating an AM, such as ask for
one with the max allowable RAM. I can do that for any container the AM asks
for, but not for the initial AM/app request
On 21 November 2013 14:37, Alejandro Abdelnur <tu...@cloudera.com> wrote:
> Steve, what kind of tuning do you need to do based on the max resources
> that would affect the creation/configuration of the Yarn client itself?
>
> Thanks.
>
>
> On Thu, Nov 21, 2013 at 6:06 AM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > Is there any way to determine the max resources you can ask for from an
> RM
> > queue before you actually create the AM? I know the AM gets that data
> once
> > it's been created, but without knowing those limits client side, you
> can't
> > tune the requirements of the AM itself
> >
> > If not, I'm going to file a JIRA to add this info to QueueInfo, so a call
> > to ApplicationClientProtocol.getQueueInfo() will let a YARN client find
> it
> > out.
> >
> > -steve
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> Alejandro
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: API to determine max limits (vcores, vram, &c from the client?)
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Steve, what kind of tuning do you need to do based on the max resources
that would affect the creation/configuration of the Yarn client itself?
Thanks.
On Thu, Nov 21, 2013 at 6:06 AM, Steve Loughran <st...@hortonworks.com>wrote:
> Is there any way to determine the max resources you can ask for from an RM
> queue before you actually create the AM? I know the AM gets that data once
> it's been created, but without knowing those limits client side, you can't
> tune the requirements of the AM itself
>
> If not, I'm going to file a JIRA to add this info to QueueInfo, so a call
> to ApplicationClientProtocol.getQueueInfo() will let a YARN client find it
> out.
>
> -steve
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
Alejandro