You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Chris Riccomini <cr...@linkedin.com> on 2013/06/12 19:53:34 UTC

YARN CGroup cpu.shares question

Hey guys,

I was able to get YARN up and running using CGroups yesterday, but I'm not seeing YARN write to cpu.shares. As a result, the CPU-level isolation is just limited to equal-sharing between all NM processes. That is, if one proc asks for 10 virtual cores, and the other asks for 2, on a 12 vcore machine, they will still split 50/50, since their cpu.shares have 1024 for each.

Did I miss a configuration, or is cpu.shares currently not supported? Is it in the works?

Thanks!
Chris

Re: YARN CGroup cpu.shares question

Posted by Sandy Ryza <sa...@cloudera.com>.
That's my intention.

-Sandy


On Wed, Jun 12, 2013 at 11:06 AM, Chris Riccomini
<cr...@linkedin.com>wrote:

> Hey Sandy,
>
> Fantastic. Is this planned for 2.1.0?
>
> Cheers,
> Chris
>
> On 6/12/13 10:59 AM, "Sandy Ryza" <sa...@cloudera.com> wrote:
>
> >Hi Chris,
> >
> >This is currently in the works:
> >https://issues.apache.org/jira/browse/YARN-600.
> >
> >-Sandy
> >
> >
> >On Wed, Jun 12, 2013 at 10:53 AM, Chris Riccomini
> ><cr...@linkedin.com>wrote:
> >
> >> Hey guys,
> >>
> >> I was able to get YARN up and running using CGroups yesterday, but I'm
> >>not
> >> seeing YARN write to cpu.shares. As a result, the CPU-level isolation is
> >> just limited to equal-sharing between all NM processes. That is, if one
> >> proc asks for 10 virtual cores, and the other asks for 2, on a 12 vcore
> >> machine, they will still split 50/50, since their cpu.shares have 1024
> >>for
> >> each.
> >>
> >> Did I miss a configuration, or is cpu.shares currently not supported? Is
> >> it in the works?
> >>
> >> Thanks!
> >> Chris
> >>
>
>

Re: YARN CGroup cpu.shares question

Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Sandy,

Fantastic. Is this planned for 2.1.0?

Cheers,
Chris

On 6/12/13 10:59 AM, "Sandy Ryza" <sa...@cloudera.com> wrote:

>Hi Chris,
>
>This is currently in the works:
>https://issues.apache.org/jira/browse/YARN-600.
>
>-Sandy
>
>
>On Wed, Jun 12, 2013 at 10:53 AM, Chris Riccomini
><cr...@linkedin.com>wrote:
>
>> Hey guys,
>>
>> I was able to get YARN up and running using CGroups yesterday, but I'm
>>not
>> seeing YARN write to cpu.shares. As a result, the CPU-level isolation is
>> just limited to equal-sharing between all NM processes. That is, if one
>> proc asks for 10 virtual cores, and the other asks for 2, on a 12 vcore
>> machine, they will still split 50/50, since their cpu.shares have 1024
>>for
>> each.
>>
>> Did I miss a configuration, or is cpu.shares currently not supported? Is
>> it in the works?
>>
>> Thanks!
>> Chris
>>

Re: YARN CGroup cpu.shares question

Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Chris,

This is currently in the works:
https://issues.apache.org/jira/browse/YARN-600.

-Sandy


On Wed, Jun 12, 2013 at 10:53 AM, Chris Riccomini
<cr...@linkedin.com>wrote:

> Hey guys,
>
> I was able to get YARN up and running using CGroups yesterday, but I'm not
> seeing YARN write to cpu.shares. As a result, the CPU-level isolation is
> just limited to equal-sharing between all NM processes. That is, if one
> proc asks for 10 virtual cores, and the other asks for 2, on a 12 vcore
> machine, they will still split 50/50, since their cpu.shares have 1024 for
> each.
>
> Did I miss a configuration, or is cpu.shares currently not supported? Is
> it in the works?
>
> Thanks!
> Chris
>