You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Shawn Du <sh...@neulion.com.cn> on 2017/01/22 05:03:21 UTC
答复: ignite client memory issue
I review the code, find a never released HashMap which store some cache
configurations.
private Map<TableSchema, CacheConfiguration<String, BinaryObject>>
cacheConfigurations = new HashMap<>();
I cache these configurations for performance consideration.
I get ignite cache by calling:
IgniteCache<String, BinaryObject> cache =
IgniteManager.getIgnite().getOrCreateCache(configuration);
My question:
If cache configuration doesn’t release, it will prevent some memeory/cache
entries/something else release?
Thanks
Shawn
发件人: Shawn Du [mailto:shawn.du@neulion.com.cn]
发送时间: 2017年1月22日 9:09
收件人: user@ignite.apache.org
主题: ignite client memory issue
Hi,
My ignite client died many times recently because of OOM. Of course, I can
increase the Max heap size. But I want to know why these memory are not
released.
This is part of analysis of Eclipse Memory Analyzer. My ignite client use
Async cache API. please help.
Class Name
| Objects | Shallow Heap | Retained Heap
----------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
-------
java.lang.Thread
| 109 | 13,080 | >= 561,086,296
java.lang.ThreadLocal$ThreadLocalMap
| 98 | 2,352 | >= 537,825,592
java.lang.ThreadLocal$ThreadLocalMap$Entry[]
| 98 | 139,808 | >= 537,823,232
java.lang.ThreadLocal$ThreadLocalMap$Entry
| 15,202 | 486,464 | >= 535,750,616
org.apache.ignite.internal.processors.cache.IgniteCacheFutureImpl
| 14,476 | 231,616 | >= 534,866,208
org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture
| 14,476 | 926,464 | >= 534,634,600
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearA
tomicSingleUpdateFuture | 14,466 | 2,198,832 | >=
531,444,584
byte[]
| 18,438 | 530,369,360 | >= 530,369,360
org.apache.ignite.internal.binary.BinaryObjectImpl
| 14,466 | 578,640 | >= 527,749,368
Thanks
Shawn
Re: ignite client memory issue
Posted by Denis Magda <dm...@apache.org>.
Also keep in mind that every Ignite node requires at least ~ 300 MB for its internal purposes. This capacity planning page might be useful as well for you:
https://apacheignite.readme.io/docs/capacity-planning-bak <https://apacheignite.readme.io/docs/capacity-planning-bak>
—
Denis
> On Jan 21, 2017, at 9:29 PM, Shawn Du <sh...@neulion.com.cn> wrote:
>
> My heap max size is 768M. more than 500M are consumed by ignite.
>
> Your advice is quite reasonable. I will refactor my code.
>
> Thanks
> 发件人: Denis Magda [mailto:dmagda@apache.org]
> 发送时间: 2017年1月22日 13:18
> 收件人: user@ignite.apache.org
> 主题: Re: ignite client memory issue
>
> Hi Shawn,
>
> What is the maximum size of the heap?
>
> I don’t think the cache configurations can be a reason of the OOM. As a side note, there is no reason to keep the configurations at all. Once a cache is started with a configuration you can either keep a single reference to it and reuse by multiple app threads or get a new one by passing the cache name into a respective Ignite method.
>
> —
> Denis
>
>> On Jan 21, 2017, at 9:03 PM, Shawn Du <shawn.du@neulion.com.cn <ma...@neulion.com.cn>> wrote:
>>
>> I review the code, find a never released HashMap which store some cache configurations.
>> private Map<TableSchema, CacheConfiguration<String, BinaryObject>> cacheConfigurations = new HashMap<>();
>> I cache these configurations for performance consideration.
>> I get ignite cache by calling:
>> IgniteCache<String, BinaryObject> cache = IgniteManager.getIgnite().getOrCreateCache(configuration);
>>
>> My question:
>> If cache configuration doesn’t release, it will prevent some memeory/cache entries/something else release?
>>
>> Thanks
>> Shawn
>>
>> 发件人: Shawn Du [mailto:shawn.du@neulion.com.cn <ma...@neulion.com.cn>]
>> 发送时间: 2017年1月22日 9:09
>> 收件人: user@ignite.apache.org <ma...@ignite.apache.org>
>> 主题: ignite client memory issue
>>
>> Hi,
>>
>> My ignite client died many times recently because of OOM. Of course, I can increase the Max heap size. But I want to know why these memory are not released.
>> This is part of analysis of Eclipse Memory Analyzer. My ignite client use Async cache API. please help.
>>
>> Class Name | Objects | Shallow Heap | Retained Heap
>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> java.lang.Thread | 109 | 13,080 | >= 561,086,296
>> java.lang.ThreadLocal$ThreadLocalMap | 98 | 2,352 | >= 537,825,592
>> java.lang.ThreadLocal$ThreadLocalMap$Entry[] | 98 | 139,808 | >= 537,823,232
>> java.lang.ThreadLocal$ThreadLocalMap$Entry | 15,202 | 486,464 | >= 535,750,616
>> org.apache.ignite.internal.processors.cache.IgniteCacheFutureImpl | 14,476 | 231,616 | >= 534,866,208
>> org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture | 14,476 | 926,464 | >= 534,634,600
>> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture | 14,466 | 2,198,832 | >= 531,444,584
>> byte[] | 18,438 | 530,369,360 | >= 530,369,360
>> org.apache.ignite.internal.binary.BinaryObjectImpl | 14,466 | 578,640 | >= 527,749,368
>>
>> Thanks
>> Shawn
答复: ignite client memory issue
Posted by Shawn Du <sh...@neulion.com.cn>.
My heap max size is 768M. more than 500M are consumed by ignite.
Your advice is quite reasonable. I will refactor my code.
Thanks
发件人: Denis Magda [mailto:dmagda@apache.org]
发送时间: 2017年1月22日 13:18
收件人: user@ignite.apache.org
主题: Re: ignite client memory issue
Hi Shawn,
What is the maximum size of the heap?
I don’t think the cache configurations can be a reason of the OOM. As a side note, there is no reason to keep the configurations at all. Once a cache is started with a configuration you can either keep a single reference to it and reuse by multiple app threads or get a new one by passing the cache name into a respective Ignite method.
—
Denis
On Jan 21, 2017, at 9:03 PM, Shawn Du <shawn.du@neulion.com.cn <ma...@neulion.com.cn> > wrote:
I review the code, find a never released HashMap which store some cache configurations.
private Map<TableSchema, CacheConfiguration<String, BinaryObject>> cacheConfigurations = new HashMap<>();
I cache these configurations for performance consideration.
I get ignite cache by calling:
IgniteCache<String, BinaryObject> cache = IgniteManager.getIgnite().getOrCreateCache(configuration);
My question:
If cache configuration doesn’t release, it will prevent some memeory/cache entries/something else release?
Thanks
Shawn
发件人: Shawn Du [mailto:shawn.du@neulion.com.cn]
发送时间: 2017年1月22日 9:09
收件人: user@ignite.apache.org <ma...@ignite.apache.org>
主题: ignite client memory issue
Hi,
My ignite client died many times recently because of OOM. Of course, I can increase the Max heap size. But I want to know why these memory are not released.
This is part of analysis of Eclipse Memory Analyzer. My ignite client use Async cache API. please help.
Class Name | Objects | Shallow Heap | Retained Heap
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
java.lang.Thread | 109 | 13,080 | >= 561,086,296
java.lang.ThreadLocal$ThreadLocalMap | 98 | 2,352 | >= 537,825,592
java.lang.ThreadLocal$ThreadLocalMap$Entry[] | 98 | 139,808 | >= 537,823,232
java.lang.ThreadLocal$ThreadLocalMap$Entry | 15,202 | 486,464 | >= 535,750,616
org.apache.ignite.internal.processors.cache.IgniteCacheFutureImpl | 14,476 | 231,616 | >= 534,866,208
org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture | 14,476 | 926,464 | >= 534,634,600
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture | 14,466 | 2,198,832 | >= 531,444,584
byte[] | 18,438 | 530,369,360 | >= 530,369,360
org.apache.ignite.internal.binary.BinaryObjectImpl | 14,466 | 578,640 | >= 527,749,368
Thanks
Shawn
Re: ignite client memory issue
Posted by Denis Magda <dm...@apache.org>.
Hi Shawn,
What is the maximum size of the heap?
I don’t think the cache configurations can be a reason of the OOM. As a side note, there is no reason to keep the configurations at all. Once a cache is started with a configuration you can either keep a single reference to it and reuse by multiple app threads or get a new one by passing the cache name into a respective Ignite method.
—
Denis
> On Jan 21, 2017, at 9:03 PM, Shawn Du <sh...@neulion.com.cn> wrote:
>
> I review the code, find a never released HashMap which store some cache configurations.
> private Map<TableSchema, CacheConfiguration<String, BinaryObject>> cacheConfigurations = new HashMap<>();
> I cache these configurations for performance consideration.
> I get ignite cache by calling:
> IgniteCache<String, BinaryObject> cache = IgniteManager.getIgnite().getOrCreateCache(configuration);
>
> My question:
> If cache configuration doesn’t release, it will prevent some memeory/cache entries/something else release?
>
> Thanks
> Shawn
>
> 发件人: <> Shawn Du [mailto:shawn.du@neulion.com.cn]
> 发送时间: 2017年1月22日 9:09
> 收件人: user@ignite.apache.org
> 主题: ignite client memory issue
>
> Hi,
>
> My ignite client died many times recently because of OOM. Of course, I can increase the Max heap size. But I want to know why these memory are not released.
> This is part of analysis of Eclipse Memory Analyzer. My ignite client use Async cache API. please help.
>
> Class Name | Objects | Shallow Heap | Retained Heap
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> java.lang.Thread | 109 | 13,080 | >= 561,086,296
> java.lang.ThreadLocal$ThreadLocalMap | 98 | 2,352 | >= 537,825,592
> java.lang.ThreadLocal$ThreadLocalMap$Entry[] | 98 | 139,808 | >= 537,823,232
> java.lang.ThreadLocal$ThreadLocalMap$Entry | 15,202 | 486,464 | >= 535,750,616
> org.apache.ignite.internal.processors.cache.IgniteCacheFutureImpl | 14,476 | 231,616 | >= 534,866,208
> org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture | 14,476 | 926,464 | >= 534,634,600
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture | 14,466 | 2,198,832 | >= 531,444,584
> byte[] | 18,438 | 530,369,360 | >= 530,369,360
> org.apache.ignite.internal.binary.BinaryObjectImpl | 14,466 | 578,640 | >= 527,749,368
>
> Thanks
> Shawn