You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Prashant Kommireddi <pr...@gmail.com> on 2013/10/22 00:53:04 UTC
ResourceManager webapp code runs OOM
Hello,
We are noticing the RM running out of memory in the webapp code. It happens
in
org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
html).
The StringBuilder object "appsTableData" grows too large in this case while
appending AppInfo. Ignoring the heap size (this should be fixed if I
increased the heapsize for sometime), is there a way to configure the
number of AppInfos that should be loaded? Or pagination?
Thanks,
Prashant
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You're welcome. I believe for MR JHS it ought to be mapreduce.jobhistory.joblist.cache.size
HTH
Ravi
On Tuesday, October 22, 2013 7:29 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hi Ravi,
Are you aware of a similar config for JH server?
On Tue, Oct 22, 2013 at 2:41 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Thanks Ravi!
>
>
>
>On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Hi Prashant!
>>
>>
>>
>>You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
>>
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
>>
>>Hello,
>>
>>
>>We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
>>
>>
>>The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
>>
>>
>>Thanks,
>>Prashant
>>
>>
>
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You're welcome. I believe for MR JHS it ought to be mapreduce.jobhistory.joblist.cache.size
HTH
Ravi
On Tuesday, October 22, 2013 7:29 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hi Ravi,
Are you aware of a similar config for JH server?
On Tue, Oct 22, 2013 at 2:41 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Thanks Ravi!
>
>
>
>On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Hi Prashant!
>>
>>
>>
>>You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
>>
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
>>
>>Hello,
>>
>>
>>We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
>>
>>
>>The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
>>
>>
>>Thanks,
>>Prashant
>>
>>
>
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You're welcome. I believe for MR JHS it ought to be mapreduce.jobhistory.joblist.cache.size
HTH
Ravi
On Tuesday, October 22, 2013 7:29 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hi Ravi,
Are you aware of a similar config for JH server?
On Tue, Oct 22, 2013 at 2:41 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Thanks Ravi!
>
>
>
>On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Hi Prashant!
>>
>>
>>
>>You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
>>
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
>>
>>Hello,
>>
>>
>>We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
>>
>>
>>The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
>>
>>
>>Thanks,
>>Prashant
>>
>>
>
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You're welcome. I believe for MR JHS it ought to be mapreduce.jobhistory.joblist.cache.size
HTH
Ravi
On Tuesday, October 22, 2013 7:29 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hi Ravi,
Are you aware of a similar config for JH server?
On Tue, Oct 22, 2013 at 2:41 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Thanks Ravi!
>
>
>
>On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Hi Prashant!
>>
>>
>>
>>You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
>>
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
>>
>>Hello,
>>
>>
>>We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
>>
>>
>>The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
>>
>>
>>Thanks,
>>Prashant
>>
>>
>
Re: ResourceManager webapp code runs OOM
Posted by Prashant Kommireddi <pr...@gmail.com>.
Thanks Ravi!
On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Prashant!
>
> You can set yarn.resourcemanager.max-completed-applications in
> yarn-site.xml of RM to limit the maximum number of apps it keeps track of
> (It defaults to 10000). You're right that the Heap may also be increased.
>
> HTH
> Ravi
>
>
> On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <
> prash1784@gmail.com> wrote:
> Hello,
>
> We are noticing the RM running out of memory in the webapp code. It
> happens in
> org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
> html).
>
> The StringBuilder object "appsTableData" grows too large in this case
> while appending AppInfo. Ignoring the heap size (this should be fixed if I
> increased the heapsize for sometime), is there a way to configure the
> number of AppInfos that should be loaded? Or pagination?
>
> Thanks,
> Prashant
>
>
>
Re: ResourceManager webapp code runs OOM
Posted by Prashant Kommireddi <pr...@gmail.com>.
Thanks Ravi!
On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Prashant!
>
> You can set yarn.resourcemanager.max-completed-applications in
> yarn-site.xml of RM to limit the maximum number of apps it keeps track of
> (It defaults to 10000). You're right that the Heap may also be increased.
>
> HTH
> Ravi
>
>
> On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <
> prash1784@gmail.com> wrote:
> Hello,
>
> We are noticing the RM running out of memory in the webapp code. It
> happens in
> org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
> html).
>
> The StringBuilder object "appsTableData" grows too large in this case
> while appending AppInfo. Ignoring the heap size (this should be fixed if I
> increased the heapsize for sometime), is there a way to configure the
> number of AppInfos that should be loaded? Or pagination?
>
> Thanks,
> Prashant
>
>
>
Re: ResourceManager webapp code runs OOM
Posted by Prashant Kommireddi <pr...@gmail.com>.
Thanks Ravi!
On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Prashant!
>
> You can set yarn.resourcemanager.max-completed-applications in
> yarn-site.xml of RM to limit the maximum number of apps it keeps track of
> (It defaults to 10000). You're right that the Heap may also be increased.
>
> HTH
> Ravi
>
>
> On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <
> prash1784@gmail.com> wrote:
> Hello,
>
> We are noticing the RM running out of memory in the webapp code. It
> happens in
> org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
> html).
>
> The StringBuilder object "appsTableData" grows too large in this case
> while appending AppInfo. Ignoring the heap size (this should be fixed if I
> increased the heapsize for sometime), is there a way to configure the
> number of AppInfos that should be loaded? Or pagination?
>
> Thanks,
> Prashant
>
>
>
Re: ResourceManager webapp code runs OOM
Posted by Prashant Kommireddi <pr...@gmail.com>.
Thanks Ravi!
On Tue, Oct 22, 2013 at 12:55 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Prashant!
>
> You can set yarn.resourcemanager.max-completed-applications in
> yarn-site.xml of RM to limit the maximum number of apps it keeps track of
> (It defaults to 10000). You're right that the Heap may also be increased.
>
> HTH
> Ravi
>
>
> On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <
> prash1784@gmail.com> wrote:
> Hello,
>
> We are noticing the RM running out of memory in the webapp code. It
> happens in
> org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
> html).
>
> The StringBuilder object "appsTableData" grows too large in this case
> while appending AppInfo. Ignoring the heap size (this should be fixed if I
> increased the heapsize for sometime), is there a way to configure the
> number of AppInfos that should be loaded? Or pagination?
>
> Thanks,
> Prashant
>
>
>
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
HTH
Ravi
On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hello,
We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
Thanks,
Prashant
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
HTH
Ravi
On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hello,
We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
Thanks,
Prashant
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
HTH
Ravi
On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hello,
We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
Thanks,
Prashant
Re: ResourceManager webapp code runs OOM
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Prashant!
You can set yarn.resourcemanager.max-completed-applicationsin yarn-site.xml of RM to limit the maximum number of apps it keeps track of (It defaults to 10000). You're right that the Heap may also be increased.
HTH
Ravi
On Monday, October 21, 2013 5:54 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
Hello,
We are noticing the RM running out of memory in the webapp code. It happens in org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block html).
The StringBuilder object "appsTableData" grows too large in this case while appending AppInfo. Ignoring the heap size (this should be fixed if I increased the heapsize for sometime), is there a way to configure the number of AppInfos that should be loaded? Or pagination?
Thanks,
Prashant