You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Prabuddha <pr...@taqniatech.com> on 2016/04/05 08:43:51 UTC
Heap memory getting increase while doing cache.get()
Hi,
We have a simple client server mode of apache ignite and we see a memory
increase in server node heap while doing "cache.get(cacheKey)".
Below is our scenario.
1. First we insert a one object list with "cacheKey".
2. Then we get it for like 200 times. In that case server node heap
memory grows and finally it shuts down.
Why the heap is growing while just doing the "get(cacheKey)".
Below is our cache configuration.
CacheConfiguration cacheCfg = new CacheConfiguration("TEST CACHE");
cacheCfg.setExpiryPolicyFactory(FactoryBuilder.factoryOf(new
CreatedExpiryPolicy(new Duration(TimeUnit.MINUTES, 15))));
cacheCfg.setEagerTtl(true);
cacheCfg.setDefaultLockTimeout(2000);
cacheCfg.setBackups(1);
cacheCfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
cacheCfg.setCacheMode(CacheMode.PARTITIONED);
cacheCfg.setMemoryMode(CacheMemoryMode.ONHEAP_TIERED);
cacheCfg.setOffHeapMaxMemory(1*1024*1024*1024);
cacheCfg.setEvictSynchronized(true);
cacheCfg.setEvictionPolicy(new LruEvictionPolicy(500));
Below is the sample code
IgniteCache igniteCache =
Ignition.ignite("CACHE_GRID").getOrCreateCache(cacheCfg,nearCfg);
List<CacheData> cacheList = new ArrayList<CacheData>();
for(int y =0;y<100;y++)
{
cacheList.add(new CacheData() );
}
System.out.println("100 objects added");
igniteCache.putIfAbsent("A", cacheList);
int k =100;
for(int i=0;i<k;i++)
{
igniteCache.get("A");
}
Thanks and Regards,
Prabuddha
RE: Heap memory getting increase while doing cache.get()
Posted by vkulichenko <va...@gmail.com>.
Hi,
Can you please attach the whole log? Most likely the node was still
suffering from GC pauses or OOM and was segmented.
Answering your questions:
1. Yes, data in partitioned cache is evenly split across nodes. But note
that there is always Ignite overhead on top of the data itself. Refer to [1]
for some details and hints. Also keep in mind that memory is managed by JVM,
so part of the heap is usually consumed by garbage which will be eventually
collected. So the reported amount of consumed memory is not always accurate.
To check this you can use any tool like VisualVM to force garbage
collection. After it's done, you will get more informative picture.
2. Client node will connect to one of the addresses provided by the IP
finder. There is no single point where all clients connect to.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Heap-memory-getting-increase-while-doing-cache-get-tp3925p3981.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
RE: Heap memory getting increase while doing cache.get()
Posted by Prabuddha <pr...@taqniatech.com>.
Hi,
Thank you for the response and it reduces the growing of heap.
Now we have got below exception when there are two server nodes and both
servers shuts down because of this.
[19:00:49,685][SEVERE][tcp-disco-msg-worker-#2%IBECACHE_GRID%][TcpDiscoveryS
pi] TcpDiscoverSpi's message worker thread failed abnormally. Stopping the
node in order to prevent cluster wide instability.
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.report
InterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitN
anos(AbstractQueuedSynchronizer.java:2095)
at
java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:
519)
at
java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:682)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(Ser
verImpl.java:5779)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(Server
Impl.java:2161)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
[19:00:53] Topology snapshot [ver=5, servers=1, clients=0, CPUs=8,
heap=4.0GB]
[19:01:06] Ignite node stopped OK [name=IBECACHE_GRID, uptime=00:14:33:947]
See our concerns below and correct us if anything misunderstood.
1. In a PARTITIONED cache how the data is divided within the cache
nodes ?
If the Total data size is 1GB, does it divide 500 MB for each ?
In our case we have seen both nodes' heap size is looks equal to
1GB.
2. When there are multiple server nodes to which node client
connects ? Is it the oldest node ?
Also could you let us know is there any other document that we can refer on
cluster configuration.
Regards,
Prabuddha.
-----Original Message-----
From: Denis Magda [mailto:dmagda@gridgain.com]
Sent: Tuesday, April 05, 2016 12:34 PM
To: user@ignite.apache.org
Subject: Re: Heap memory getting increase while doing cache.get()
Hi Prabuddha,
Cache.get() operation returns copy of an object stored in a cache by
default. If you set CacheConfiguration.setCopyOnRead to false then a copy
won't be created but you mustn't modify returned object directly because it
can effect consistency.
In any case the objects returned by cache.get() must be cleaned by Garbage
Collector. Probably you should modify your GC settings to overcome
application shutdown due to insufficient memory or long GC.
Please set this generic recommended settings to see the difference
https://apacheignite.readme.io/docs/performance-tips#tune-garbage-collection
--
Denis
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Heap-memory-getting-increase-
while-doing-cache-get-tp3925p3930.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Heap memory getting increase while doing cache.get()
Posted by Denis Magda <dm...@gridgain.com>.
Hi Prabuddha,
Cache.get() operation returns copy of an object stored in a cache by
default. If you set CacheConfiguration.setCopyOnRead to false then a copy
won't be created but you mustn't modify returned object directly because it
can effect consistency.
In any case the objects returned by cache.get() must be cleaned by Garbage
Collector. Probably you should modify your GC settings to overcome
application shutdown due to insufficient memory or long GC.
Please set this generic recommended settings to see the difference
https://apacheignite.readme.io/docs/performance-tips#tune-garbage-collection
--
Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Heap-memory-getting-increase-while-doing-cache-get-tp3925p3930.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.