You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by John Lilley <jo...@redpoint.net> on 2013/10/20 16:34:19 UTC
yarn.nodemanager.vmem-check-enabled
Is this option typically enabled in practice? Is there a way for a specific AM to disable it, or is it site-wide?
We are finding that our YARN application, because it launches many threads, each with a somewhat large stack, has a large vmem/pmem ratio (say, 2GB / 200MB). Unless we disable this option, the NM tends to kill our tasks.
Thanks
John
Re: yarn.nodemanager.vmem-check-enabled
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi John,
At Cloudera we turn it off by default and recommend doing so because of the
way that Java can behave weirdly with high vmems.
-Sandy
On Sun, Oct 20, 2013 at 7:34 AM, John Lilley <jo...@redpoint.net>wrote:
> Is this option typically enabled in practice? Is there a way for a
> specific AM to disable it, or is it site-wide?****
>
> We are finding that our YARN application, because it launches many
> threads, each with a somewhat large stack, has a large vmem/pmem ratio
> (say, 2GB / 200MB). Unless we disable this option, the NM tends to kill
> our tasks.****
>
> Thanks****
>
> John****
>
> ** **
>
Re: yarn.nodemanager.vmem-check-enabled
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi John,
At Cloudera we turn it off by default and recommend doing so because of the
way that Java can behave weirdly with high vmems.
-Sandy
On Sun, Oct 20, 2013 at 7:34 AM, John Lilley <jo...@redpoint.net>wrote:
> Is this option typically enabled in practice? Is there a way for a
> specific AM to disable it, or is it site-wide?****
>
> We are finding that our YARN application, because it launches many
> threads, each with a somewhat large stack, has a large vmem/pmem ratio
> (say, 2GB / 200MB). Unless we disable this option, the NM tends to kill
> our tasks.****
>
> Thanks****
>
> John****
>
> ** **
>
Re: yarn.nodemanager.vmem-check-enabled
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi John,
At Cloudera we turn it off by default and recommend doing so because of the
way that Java can behave weirdly with high vmems.
-Sandy
On Sun, Oct 20, 2013 at 7:34 AM, John Lilley <jo...@redpoint.net>wrote:
> Is this option typically enabled in practice? Is there a way for a
> specific AM to disable it, or is it site-wide?****
>
> We are finding that our YARN application, because it launches many
> threads, each with a somewhat large stack, has a large vmem/pmem ratio
> (say, 2GB / 200MB). Unless we disable this option, the NM tends to kill
> our tasks.****
>
> Thanks****
>
> John****
>
> ** **
>
Re: yarn.nodemanager.vmem-check-enabled
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi John,
At Cloudera we turn it off by default and recommend doing so because of the
way that Java can behave weirdly with high vmems.
-Sandy
On Sun, Oct 20, 2013 at 7:34 AM, John Lilley <jo...@redpoint.net>wrote:
> Is this option typically enabled in practice? Is there a way for a
> specific AM to disable it, or is it site-wide?****
>
> We are finding that our YARN application, because it launches many
> threads, each with a somewhat large stack, has a large vmem/pmem ratio
> (say, 2GB / 200MB). Unless we disable this option, the NM tends to kill
> our tasks.****
>
> Thanks****
>
> John****
>
> ** **
>