Only one node has a high number of SystemExecutor CompletedTask

[dark <ek...@gmail.com>]

Hi, team.

I have a problem about cluster cpu usage.

Ignite Cluster is running on 10 nodes. 
Unusually, only one node has a high number of SystemExecutor and CPU
utilization and Minor GC time are higher than other nodes.

Is there a separate role between the nodes, so that one node becomes the
coordinator? 
Or why only one node is overloaded?

GridSystemExecutor CompletedTaskCount
<http://apache-ignite-users.70518.x6.nabble.com/file/t1415/GridSystemExecutor.png> 

System Resource Usage
<http://apache-ignite-users.70518.x6.nabble.com/file/t1415/Ignite_System.png> 

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Only one node has a high number of SystemExecutor CompletedTask

[dark <ek...@gmail.com>]

Thanks to your kind reply.

We use Ignite to Group By Query about time series data. However, I have seen
a lot of data, so when I create TimeWindow through Expiry Strategy in one
cache, I see that it uses too many resources to expire entries. 

As a result, I have adopted a method of simply destroying the cache and
erasing all entries in that time window. Therefore, creating and destroying
multiple Cache seems inevitable. :(

I first noticed that the oldest node acts as a coordinator. 
Thank you for your valuable information. :)




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Only one node has a high number of SystemExecutor CompletedTask

[Mikhail <mi...@gmail.com>]

Hi 

as you already noted, ignite cluster has a special node - coordinator, the
oldest node in a cluster is coordinator node: 
ignite.cluster().forServers().forOldest().node()

In case of creation/destroying a cache, coordinator needs to coordinate this
process and notify each node about cache creation/destroying. So coordinator
indeed does more work than other nodes, but it doesn't significant work so
it should not be noticeable for your cluster. However if you create and
destroy caches too often then I think this might be noticed, so might be you
should clear and re-use existing caches instead of destroying them?

Thanks,
Mike.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Only one node has a high number of SystemExecutor CompletedTask

[dark <ek...@gmail.com>]

I found something that could be a clue.
The node exposed by ignite.cluster().forServers().forOldest().node() seems
to be a problem. 
Do you have any expectations?




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Only one node has a high number of SystemExecutor CompletedTask

[dark <ek...@gmail.com>]

The pattern we use is a structure that periodically creates and destroys
Cache. 

At this point, could a particular node not be able to clean up the heap?

<http://apache-ignite-users.70518.x6.nabble.com/file/t1415/HeapMemoryUsage.png> 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/