Estimation of cluster effective capacity

[vkondraschenko <vk...@griddynamics.com>]

Dear support team,

I am trying to estimate memory efficiently available for cache payload in
cluster. I base my calculation on 
https://apacheignite.readme.io/docs/capacity-planning. Can you please
confirm that the following formula gives rough but realistic estimation of
memory available for caches in a small/medium size cluster:

Effective capacity = (<Total_RAM> - 300Mb (OS) - 300Mb (Ignite JVM) - 100Mb
(Monitoring Agent JVM in deployed on each node)) * Number_of_nodes

So if there are 2 nodes in cluster with 4Gb of RAM on each one, effective
cluster capacity is about (4096 - 700) * 2 = 6792Mb. Another example, if
there are 4 nodes with 4Gb of RAM on each one and caches use replication
factor of 2 (two), then effective cluster capacity is (4096 - 700) * 4 / 2 =
6792Mb (the same volume, but there is redundancy of data).



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Estimation of cluster effective capacity

[Andrey Mashenkov <an...@gmail.com>]

Hi,

It is very rough estimations.

1. Recommended minimal heap size for server node is 512Mb.
If you plan to retrieve large dataset with getAll or some queries with
large page size, you should bother with requested data will fit in memory.
Also JVM GarbageCollector feels much better if there is 15-30% free heap
memory.

2. JVM also requires some memory for itself and for ignite code.

3. Ignite uses PageMemory from 2.0 version that allow to keep data in
offheap.
It is possible, some pages will be partially used. It is ok, as data can be
deleted.
Ignite tries to reuse free space of existed pages before allocate new pages.
Also, you may need more memory for checkpointing if persistence is used.

4. Overhead for object with many small fields can be higher than objects
with few large fields as Ignite have to save additional meta data for fast
field access.

The better way here is to make a test environment close to production: fill
cache with sample data, put some load to grid and check memory occupation
and GC behavior.


On Mon, Oct 30, 2017 at 1:08 PM, vkondraschenko <
vkondraschenko@griddynamics.com> wrote:

> Dear support team,
>
> I am trying to estimate memory efficiently available for cache payload in
> cluster. I base my calculation on
> https://apacheignite.readme.io/docs/capacity-planning. Can you please
> confirm that the following formula gives rough but realistic estimation of
> memory available for caches in a small/medium size cluster:
>
> Effective capacity = (<Total_RAM> - 300Mb (OS) - 300Mb (Ignite JVM) - 100Mb
> (Monitoring Agent JVM in deployed on each node)) * Number_of_nodes
>
> So if there are 2 nodes in cluster with 4Gb of RAM on each one, effective
> cluster capacity is about (4096 - 700) * 2 = 6792Mb. Another example, if
> there are 4 nodes with 4Gb of RAM on each one and caches use replication
> factor of 2 (two), then effective cluster capacity is (4096 - 700) * 4 / 2
> =
> 6792Mb (the same volume, but there is redundancy of data).
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov