You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by ignite_user2016 <ri...@gmail.com> on 2017/09/04 14:14:30 UTC
client thread dumps
Hello Igniters,
curious about client thread dumps, I see so many ignite threads on client
side, wondering what are there and how can I reduce the size of threads ?
we run ignite 2.0 in production on SB instances.
see the attached image for more information.
<http://apache-ignite-users.70518.x6.nabble.com/file/t871/ignite_thread_dump.png>
And our JVM setting as follows -
-Xms2g -Xmx2g -server -XX:+AggressiveOpts -XX:MaxMetaspaceSize=256m
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=DIR_PATH -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m
Since we use ignite for bare minimum do I need some tuning here ?
Thank you for all your help ..
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by Cristica <bo...@yahoo.com>.
Could you please specify what value you set for IGNITE_EXCHANGE_HISTORY_SIZE
What would be the impact if we set it to 0 for all our java client
applications?
We have a similar issue presented on
post:http://apache-ignite-users.70518.x6.nabble.com/IGNITE-EXCHANGE-HISTORY-SIZE-value-for-Ignite-client-td24995.html
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by ignite_user2016 <ri...@gmail.com>.
Thank you for all your help ...
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
I think it's a collection of objects from previous exchanges. By
default, GridCachePartitionExchangeManager stores 1000 objects, but since
version 2.1 it's possible to reduce this size.
I would recommend you to update to version 2.1 and try to reduce this
property:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html#IGNITE_EXCHANGE_HISTORY_SIZE
Evgenii
2017-09-05 17:57 GMT+03:00 ignite_user2016 <ri...@gmail.com>:
> Hello Evgenii,
>
> We only used ignite for 2nd level cache, mostly we used get operation on
> cache however we monitor with Ignite visor console, not sure if that would
> make difference.
>
> Do you know what could be the reason for leak which shows on heap dump ?
>
> One instance of
> "org.apache.ignite.internal.processors.cache.
> GridCachePartitionExchangeManager"
> loaded by "sun.misc.Launcher$AppClassLoader @ 0x88003d40" occupies
> 375,647,072 (83.93%) bytes. The memory is accumulated in one instance of
> "java.util.LinkedList" loaded by "<system class loader>".
>
> Keywords
> java.util.LinkedList
> sun.misc.Launcher$AppClassLoader @ 0x88003d40
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeMana
> ger
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: client thread dumps
Posted by ignite_user2016 <ri...@gmail.com>.
Hello Evgenii,
We only used ignite for 2nd level cache, mostly we used get operation on
cache however we monitor with Ignite visor console, not sure if that would
make difference.
Do you know what could be the reason for leak which shows on heap dump ?
One instance of
"org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager"
loaded by "sun.misc.Launcher$AppClassLoader @ 0x88003d40" occupies
375,647,072 (83.93%) bytes. The memory is accumulated in one instance of
"java.util.LinkedList" loaded by "<system class loader>".
Keywords
java.util.LinkedList
sun.misc.Launcher$AppClassLoader @ 0x88003d40
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by Evgenii Zhuravlev <e....@gmail.com>.
>I am also seeing frequent GC in GC monitor, wondering what it can relate to
>that is with CPU spike ?
I think these spikes could be related to the GC, yes. But they also could
be related to the work that happens on this node. Could you describe how
you use this node? Which operation do you invoke on it?
Evgenii
2017-09-04 22:03 GMT+03:00 ezhuravlev <e....@gmail.com>:
> Hi,
>
> Here is a description of all thread pools:
> https://apacheignite.readme.io/v2.1/docs/thread-pools
> You can try to reduce their size, but do it carefully, these changes could
> lead to performance degradation.
>
> If you will never use visor or rest-api you can also set
> ConnectorConfiguration to null:
>
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setConnectorConfiguration(null); // disables the tcp rest connector
>
>
> Also, I found thread with pretty the same questions that you have about
> threads:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-
> Thread-count-td7636.html
>
> Evgenii
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: client thread dumps
Posted by ezhuravlev <e....@gmail.com>.
Hi,
Here is a description of all thread pools:
https://apacheignite.readme.io/v2.1/docs/thread-pools
You can try to reduce their size, but do it carefully, these changes could
lead to performance degradation.
If you will never use visor or rest-api you can also set
ConnectorConfiguration to null:
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setConnectorConfiguration(null); // disables the tcp rest connector
Also, I found thread with pretty the same questions that you have about
threads:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Thread-count-td7636.html
Evgenii
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by ignite_user2016 <ri...@gmail.com>.
I took a heap dump and it all points to following class -
One instance of
"org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager"
loaded by "sun.misc.Launcher$AppClassLoader @ 0x88003d40" occupies
375,647,072 (83.93%) bytes. The memory is accumulated in one instance of
"java.util.LinkedList" loaded by "<system class loader>".
Keywords
java.util.LinkedList
sun.misc.Launcher$AppClassLoader @ 0x88003d40
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager
see the attached images.
<http://apache-ignite-users.70518.x6.nabble.com/file/t871/heapdump_1.png>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client thread dumps
Posted by ignite_user2016 <ri...@gmail.com>.
I am also seeing frequent GC in GC monitor, wondering what it can relate to
that is with CPU spike ?
<http://apache-ignite-users.70518.x6.nabble.com/file/t871/CPU_Utilization.png>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/