Re: off heap memory usage

[Denis Mekhanikov <dm...@gmail.com>]

Hi!

Looks like this metric shows only non-heap memory, consumed by JVM itself.
Looks like a bug.
If you want to get a picture of how much memory Ignite pages consumed, you
can get *DataRegionMetrics#PhysicalMemoryPages* and multiply it by a size
of a data page.
To do this you should enable memory metrics first. Here you can find how:
https://apacheignite.readme.io/docs/memory-metrics
Note, that pages may be not full, will still be included into this metric.

Denis

чт, 4 янв. 2018 г. в 11:05, shawn.du <sh...@neulion.com.cn>:

> Hi community,
>
> I want monitor ignite cluster off heap memory usage. if off heap available
> memory is less than 1G then alert.
> I went through ignite MBean,  find ClusterLocalNodeMetricsMXBeanImpl's
> NonHeapMemoryUsed.
> The value is too small. I don't think it is.  How can I get it?
>
> Thanks
> Shawn
>

Re: off heap memory usage

[Alexey Popov <ta...@gmail.com>]

Hi Colin,

You should use TotalAllocatedPages if you have persistence disabled.

There is an know issue with PhysicalMemoryPages:
https://issues.apache.org/jira/browse/IGNITE-6963

Thank you,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[colinc <co...@yahoo.co.uk>]

Thanks for this. The referenced post mentions AllocatedPages rather than
PhysicalPages. Which would you advise is the most appropriate for this
application?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[Alexey Popov <ta...@gmail.com>]

Hi Colin,

Unfortunately, you can't get the exact off-heap size.

There are several tickets here 
https://issues.apache.org/jira/browse/IGNITE-6814
https://issues.apache.org/jira/browse/IGNITE-5583
http://apache-ignite-users.70518.x6.nabble.com/Cache-size-in-Memory-td17226.html

You are using (pages * pageSize * pagesFillFactor) correctly (if you have
only one memory region).

BTW, pagesFillFactor is calculated using FreeLists [1] buckets. FreeLists
has a limited amount of buckets (for instance, approx 25% free, ~50% free,
~75% free).
In this case (pages * pageSize * pagesFillFactor) will give you an approx
value of memory usage (based on buckets) and you can see drops in the usage
when a page is moved from one bucket to another.

[1]
https://apacheignite.readme.io/docs/memory-architecture#section-free-lists

Thanks,
Alexey




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[colinc <co...@yahoo.co.uk>]

I have given this (pages * pageSize * pagesFillFactor) a go now, but it
doesn't seem to be returning the values I'm expecting. In particular, the
value can drop significantly even when data is being inserted into the
cache.

Am I using pagesFillFactor incorrectly?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[colinc <co...@yahoo.co.uk>]

Please ignite my above comment, I am now able to retrieve the factor.

As such, is the following correct?

memoryUsed = pages * pageSize * pagesFillFactor



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[colinc <co...@yahoo.co.uk>]

I can confirm that I have metrics enabled for my region - I am able to read
allocatedPages, it's just the fillFactor that always seems to return zero.

Colin.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: off heap memory usage

[Denis Mekhanikov <dm...@gmail.com>]

Collin,

To be able to see data region metrics, you should enable them either in
configuration, or via MXBean.
You can find a note about it here:
https://apacheignite.readme.io/v2.3/docs/memory-metrics

Note, that memory metrics collection should be enabled per data region.

Denis

пт, 5 янв. 2018 г. в 12:13, colinc <co...@yahoo.co.uk>:

> Hi Denis,
>
> As per your comment, I can see pages*pageSize rising as entries are put
> into
> the cache - but this metric doesn't come down e.g. when new nodes are added
> to the cluster. I assume that the pages remain allocated but with a lower
> fill factor.
>
> So pages*pageSize gives a misleadingly pessimistic view of available
> memory.
> It would be very useful to see the effect of adding extra capacity or
> purging entries.
>
> I note that the pagesFillFactor metric sounds like it should be able to
> supply the missing information - but Ignite 2.3 seems to always return a
> value of zero for me. Does this indicate a 100% fill rate, or is the metric
> not in use?
>
> Regards,
> Colin.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: off heap memory usage

[colinc <co...@yahoo.co.uk>]

Hi Denis,

As per your comment, I can see pages*pageSize rising as entries are put into
the cache - but this metric doesn't come down e.g. when new nodes are added
to the cluster. I assume that the pages remain allocated but with a lower
fill factor.

So pages*pageSize gives a misleadingly pessimistic view of available memory.
It would be very useful to see the effect of adding extra capacity or
purging entries.

I note that the pagesFillFactor metric sounds like it should be able to
supply the missing information - but Ignite 2.3 seems to always return a
value of zero for me. Does this indicate a 100% fill rate, or is the metric
not in use?

Regards,
Colin.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/