You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by wbyeh <wb...@hotmail.com> on 2019/05/14 07:43:11 UTC
Ignite WebSessionFilter client threads eating up CPU usage
We are using Ignite 2.7 as a 4-server-node session cluster but recent weeks
we found some Ignite WebSessionFilter client threads eating up CPU usage
after days of running.
The GC is running smoothly.
How to address the root cause?
Is it a Ignite 2.x known issue with StripedExecutor?
Can experts here point out any solutions?
Attached file contained thread dump & 'top' usage.
811261.jstack
<http://apache-ignite-users.70518.x6.nabble.com/file/t645/811261.jstack>
Thank you!
-Patrick
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite WebSessionFilter client threads eating up CPU usage
Posted by Denis Mekhanikov <dm...@gmail.com>.
Patrick,
Could you explain a bit more, why do you think that striped pool is the
reason?
Striped pool is where cache operations happen. So, if you see, that it
consumes a lot of CPU, then it probably means, that a lot of cache
operations are coming to the cluster.
You can record a JFR and see, where most time is spent. For more
information refer to the following page:
https://apacheignite.readme.io/docs/jvm-and-system-tuning#section-flightrecorder-settings
Denis
вт, 14 мая 2019 г. в 10:43, wbyeh <wb...@hotmail.com>:
> We are using Ignite 2.7 as a 4-server-node session cluster but recent weeks
> we found some Ignite WebSessionFilter client threads eating up CPU usage
> after days of running.
> The GC is running smoothly.
>
> How to address the root cause?
> Is it a Ignite 2.x known issue with StripedExecutor?
>
> Can experts here point out any solutions?
>
> Attached file contained thread dump & 'top' usage.
>
> 811261.jstack
> <http://apache-ignite-users.70518.x6.nabble.com/file/t645/811261.jstack>
>
>
> Thank you!
>
> -Patrick
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Ignite WebSessionFilter client threads eating up CPU usage
Posted by wbyeh <wb...@hotmail.com>.
Ilya ,After we switched build-in openJDK (jdk-8u181-ojdkbuild-linux-x64) to
Zulu JDK (zulu8.38.0.13-ca-jdk8.0.212-linux_x64), it still has a high
loading CPU usage.And we found the linux node has a low CPU frequency than
other nodes w/ same configurations on CentOS 7.5.
<http://apache-ignite-users.70518.x6.nabble.com/file/t645/photo_2019-05-20_23-41-55.jpg>
<http://apache-ignite-users.70518.x6.nabble.com/file/t645/photo_2019-05-20_23-37-50.jpg>
Finally, we correct the CPU frequency with cpupower idle-set -d 0 #1 2 3 4 5
6 cpupower frequency-set -g performance.The high CPU loading situation is
getting well.So, this check could be a first check while system
tuning.cpupower frequency-infousing a supported JDK.Thank you all!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite WebSessionFilter client threads eating up CPU usage
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I can only guess that maybe your Web Sessions are very large so there are
performance implications. Can you make sure you don't hold bulky objects in
Sessions?
Regards,
--
Ilya Kasnacheev
вт, 14 мая 2019 г. в 10:43, wbyeh <wb...@hotmail.com>:
> We are using Ignite 2.7 as a 4-server-node session cluster but recent weeks
> we found some Ignite WebSessionFilter client threads eating up CPU usage
> after days of running.
> The GC is running smoothly.
>
> How to address the root cause?
> Is it a Ignite 2.x known issue with StripedExecutor?
>
> Can experts here point out any solutions?
>
> Attached file contained thread dump & 'top' usage.
>
> 811261.jstack
> <http://apache-ignite-users.70518.x6.nabble.com/file/t645/811261.jstack>
>
>
> Thank you!
>
> -Patrick
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>