Ways to improve cluster activation time

[Naveen <na...@gmail.com>]

HI All

Initiating a new thread on the cluster activation time

We are using Ignite 2.6 
You were saying, usage of cache groups is definitely needed to improve to
the cluster activation time ? 

I could see below documentation on the usage of cache groups. 
  
Should the grouping be used all the times? 
With all the benefits the cache groups have, they might impact the
performance of read operations and indexes lookups. This is caused by the
fact that all the data and indexes get mixed in shared data structures
(partition maps, B+trees) and it will take more time querying over them. 

Thus, consider using the cache groups if you have a cluster of dozens and
hundreds of nodes and caches, and you spot increased Java heap usage by
internal structures, checkpointing performance drop, slow node connectivity
to the cluster. 


In our case, we do have around 50 caches, and at the max we may have 10
nodes. do you still recommend cache groups for our use case. 

And, ours upserts TPS is very low, may be 1 k per sec, but query or read TPS
is quite high, close to 10 K TPS. So as per the above lines, it says
performance of read operations is impacted since all the caches are gong to
use the shared structures. 
We are looking for a design which can improve the cluster activation time,
but not at the expense of compromising on the query performance, since our
solution being a read intensive, we cant afford to reduce query performance.
In worst scenario, we can live with poor cluster activation process as well,
since it only affects us at the time cluster restart which is performed only
in case of cluster crash or planned maintenance. 

One more thing, if at all we need to change the system pool, below is the
command to change the system pool ?? 
IgniteConfiguration.setSystemThreadPoolSize(...) 
We have 128 CPU machines, what would be the ideal system thread pool size ??
of course, it should be tried and tested but still some number.. 

Regarding the cache groups design, anything I should consider when we are
grouping the cache 

1. We have around 40 caches, no indexes, we only have lookup on primary key,
some of them are simple keys and some of them are having complex primary
key. Some of the caches are queries together, does it help if we group them
into a cache group ? 
2. What if we are trying query the caches which are part of different cache
groups ? 
3. We are going to have close to half a billion records in each cache, so
how do we group them 
4. Some of the caches are independent , does not have any relation with
other caches 

So in case, I am going with cache group, shall I change the partition to 128
or keep the default ? 


Thanks 
Naveen



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ways to improve cluster activation time

[Naveen <na...@gmail.com>]

We can ignore this as there was already another ticket on the same issue



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/