You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Albert Shau <al...@continuuity.com> on 2013/09/13 22:18:10 UTC
Semantics of ApplicationResourceUsageReport
Hi,
Given an application id the YarnClient can return an
ApplicationResourceUsageReport that gives information about resources that
are 'used', 'reserved', and 'needed'. I'm trying to get a better
understanding of what determines whether a resource is 'used', 'reserved',
or 'needed'. If I request a container with some resource, is that 'used'
once the resource manager has given the container to the application master
even if the application master has not started the container yet? In what
situation is a resource 'reserved'? Is 'needed' the sum of the two, or
does it have some other meaning?
Thanks,
Albert
Re: Semantics of ApplicationResourceUsageReport
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Albert,
You're correct about used.
Reserved is a little bit more arcane - it refers to a mechanism that
schedulers use to prevent applications with larger container sizes from
starving. Applications place container "reservations" on nodes, and no
other containers can be placed on the node until the reservation is
fulfilled.
Needed is the amount requested that is not yet used.
-Sandy
On Sat, Sep 14, 2013 at 3:18 AM, Albert Shau <al...@continuuity.com> wrote:
> Hi,
>
> Given an application id the YarnClient can return an
> ApplicationResourceUsageReport that gives information about resources that
> are 'used', 'reserved', and 'needed'. I'm trying to get a better
> understanding of what determines whether a resource is 'used', 'reserved',
> or 'needed'. If I request a container with some resource, is that 'used'
> once the resource manager has given the container to the application master
> even if the application master has not started the container yet? In what
> situation is a resource 'reserved'? Is 'needed' the sum of the two, or
> does it have some other meaning?
>
> Thanks,
> Albert
>
Re: Semantics of ApplicationResourceUsageReport
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Albert,
You're correct about used.
Reserved is a little bit more arcane - it refers to a mechanism that
schedulers use to prevent applications with larger container sizes from
starving. Applications place container "reservations" on nodes, and no
other containers can be placed on the node until the reservation is
fulfilled.
Needed is the amount requested that is not yet used.
-Sandy
On Sat, Sep 14, 2013 at 3:18 AM, Albert Shau <al...@continuuity.com> wrote:
> Hi,
>
> Given an application id the YarnClient can return an
> ApplicationResourceUsageReport that gives information about resources that
> are 'used', 'reserved', and 'needed'. I'm trying to get a better
> understanding of what determines whether a resource is 'used', 'reserved',
> or 'needed'. If I request a container with some resource, is that 'used'
> once the resource manager has given the container to the application master
> even if the application master has not started the container yet? In what
> situation is a resource 'reserved'? Is 'needed' the sum of the two, or
> does it have some other meaning?
>
> Thanks,
> Albert
>
Re: Semantics of ApplicationResourceUsageReport
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Albert,
You're correct about used.
Reserved is a little bit more arcane - it refers to a mechanism that
schedulers use to prevent applications with larger container sizes from
starving. Applications place container "reservations" on nodes, and no
other containers can be placed on the node until the reservation is
fulfilled.
Needed is the amount requested that is not yet used.
-Sandy
On Sat, Sep 14, 2013 at 3:18 AM, Albert Shau <al...@continuuity.com> wrote:
> Hi,
>
> Given an application id the YarnClient can return an
> ApplicationResourceUsageReport that gives information about resources that
> are 'used', 'reserved', and 'needed'. I'm trying to get a better
> understanding of what determines whether a resource is 'used', 'reserved',
> or 'needed'. If I request a container with some resource, is that 'used'
> once the resource manager has given the container to the application master
> even if the application master has not started the container yet? In what
> situation is a resource 'reserved'? Is 'needed' the sum of the two, or
> does it have some other meaning?
>
> Thanks,
> Albert
>
Re: Semantics of ApplicationResourceUsageReport
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Albert,
You're correct about used.
Reserved is a little bit more arcane - it refers to a mechanism that
schedulers use to prevent applications with larger container sizes from
starving. Applications place container "reservations" on nodes, and no
other containers can be placed on the node until the reservation is
fulfilled.
Needed is the amount requested that is not yet used.
-Sandy
On Sat, Sep 14, 2013 at 3:18 AM, Albert Shau <al...@continuuity.com> wrote:
> Hi,
>
> Given an application id the YarnClient can return an
> ApplicationResourceUsageReport that gives information about resources that
> are 'used', 'reserved', and 'needed'. I'm trying to get a better
> understanding of what determines whether a resource is 'used', 'reserved',
> or 'needed'. If I request a container with some resource, is that 'used'
> once the resource manager has given the container to the application master
> even if the application master has not started the container yet? In what
> situation is a resource 'reserved'? Is 'needed' the sum of the two, or
> does it have some other meaning?
>
> Thanks,
> Albert
>