Unexpected containers allocated

["Fabio C." <an...@gmail.com>]

Hi everyone,
I was running the tez wordcount example on a 6 nodes cluster. The input
file is made up by 11 splits (so I expect 11 containers to be allocated for
the first vertex).
I notice that the Capacity Scheduler always allocates 12 container, 11 will
start (they find a pending task), while one is immediately released by tez,
since there is no other pending task.
I'm sorry I cannot enable DEBUG logging level (to see the actual request
content) since the cluster is shared among several users, but I was
wondering if it is Tez asking for an extra container (and why) or if it's a
RM policy (do you know what it is?).
This was not happening while I was playing with 2 VM on my local computer,
nor it happens on the cluster when running the MR wordcunt without Tez.

Thanks a lot

Fabio

Re: Unexpected containers allocated

["Fabio C." <an...@gmail.com>]

Thanks, I will try the per-app debug logging. Actually I thought about the
race condition, but I thought it is strange it happens every time. I'll let
you know.

Regards

Fabio

On Tue, Feb 10, 2015 at 10:35 PM, Hitesh Shah <hi...@apache.org> wrote:

> Just to be clear, if you are looking for AM debug logs, you can enable
> debug logging per app. Just use “word count -Dtez.am.log.level=DEBUG input
> output”.
>
> — Hitesh
>
>
> On Feb 10, 2015, at 4:02 AM, Fabio C. <an...@gmail.com> wrote:
>
> > Hi everyone,
> > I was running the tez wordcount example on a 6 nodes cluster. The input
> file is made up by 11 splits (so I expect 11 containers to be allocated for
> the first vertex).
> > I notice that the Capacity Scheduler always allocates 12 container, 11
> will start (they find a pending task), while one is immediately released by
> tez, since there is no other pending task.
> > I'm sorry I cannot enable DEBUG logging level (to see the actual request
> content) since the cluster is shared among several users, but I was
> wondering if it is Tez asking for an extra container (and why) or if it's a
> RM policy (do you know what it is?).
> > This was not happening while I was playing with 2 VM on my local
> computer, nor it happens on the cluster when running the MR wordcunt
> without Tez.
> >
> > Thanks a lot
> >
> > Fabio
> >
>
>

Re: Unexpected containers allocated

[Hitesh Shah <hi...@apache.org>]

Just to be clear, if you are looking for AM debug logs, you can enable debug logging per app. Just use “word count -Dtez.am.log.level=DEBUG input output”.

— Hitesh


On Feb 10, 2015, at 4:02 AM, Fabio C. <an...@gmail.com> wrote:

> Hi everyone,
> I was running the tez wordcount example on a 6 nodes cluster. The input file is made up by 11 splits (so I expect 11 containers to be allocated for the first vertex). 
> I notice that the Capacity Scheduler always allocates 12 container, 11 will start (they find a pending task), while one is immediately released by tez, since there is no other pending task.
> I'm sorry I cannot enable DEBUG logging level (to see the actual request content) since the cluster is shared among several users, but I was wondering if it is Tez asking for an extra container (and why) or if it's a RM policy (do you know what it is?).
> This was not happening while I was playing with 2 VM on my local computer, nor it happens on the cluster when running the MR wordcunt without Tez.
> 
> Thanks a lot
> 
> Fabio
> 

RE: Unexpected containers allocated

[Bikas Saha <bi...@hortonworks.com>]

Without scheduler debug logs it would be hard to say.

However, there is a well-known answer to these race conditions. YARN allocation protocol is prone to race conditions by design. And it is expected that a few extra allocations can occur. This is orthogonal to Tez.

Bikas

From: Fabio C. [mailto:anytek88@gmail.com]
Sent: Tuesday, February 10, 2015 4:03 AM
To: user@tez.apache.org
Subject: Unexpected containers allocated

Hi everyone,
I was running the tez wordcount example on a 6 nodes cluster. The input file is made up by 11 splits (so I expect 11 containers to be allocated for the first vertex).
I notice that the Capacity Scheduler always allocates 12 container, 11 will start (they find a pending task), while one is immediately released by tez, since there is no other pending task.
I'm sorry I cannot enable DEBUG logging level (to see the actual request content) since the cluster is shared among several users, but I was wondering if it is Tez asking for an extra container (and why) or if it's a RM policy (do you know what it is?).
This was not happening while I was playing with 2 VM on my local computer, nor it happens on the cluster when running the MR wordcunt without Tez.

Thanks a lot

Fabio