YARN scheduler configuration

[Tommy Becker <to...@tivo.com>]

We are currently running Samza on a YARN grid. We unintentionally got into a situation where we needed more capacity than was available (according to YARN), and we found that replacement container requests would just sit there indefinitely waiting to be fulfilled rather than failing. Our monitoring was unable to detect that there were jobs that, despite being in a "RUNNING" state, were not doing anything because they were starved for containers. Is there a way to configure YARN to reject container requests that can't be immediately satisfied?

--
Tommy Becker
Senior Software Engineer

Digitalsmiths
A TiVo Company

www.digitalsmiths.com<http://www.digitalsmiths.com>
tobecker@tivo.com<ma...@tivo.com>

________________________________

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

Re: YARN scheduler configuration

[Yi Pan <ni...@gmail.com>]

Hi, Tommy,

Sorry to notice this late. I will reach out to our SRE team to see whether
Yarn provides any feature like that. Meanwhile, I think that it is worth to
check w/ Yarn's mailing list for this question.

-Yi

On Mon, Sep 14, 2015 at 1:16 PM, Tommy Becker <to...@tivo.com> wrote:

> We are currently running Samza on a YARN grid. We unintentionally got into
> a situation where we needed more capacity than was available (according to
> YARN), and we found that replacement container requests would just sit
> there indefinitely waiting to be fulfilled rather than failing. Our
> monitoring was unable to detect that there were jobs that, despite being in
> a "RUNNING" state, were not doing anything because they were starved for
> containers. Is there a way to configure YARN to reject container requests
> that can't be immediately satisfied?
>
> --
> Tommy Becker
> Senior Software Engineer
>
> Digitalsmiths
> A TiVo Company
>
> www.digitalsmiths.com<http://www.digitalsmiths.com>
> tobecker@tivo.com<ma...@tivo.com>
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo Inc. is authorized to conclude any binding
> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> Inc. may only be made by a signed written agreement.
>