You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Shaomin Zhang <Sh...@tudor.com> on 2016/04/14 15:24:26 UTC

Off-heap memory usage questions

I am trying to use OFFHEAP_TIERED caches, and have this setting for the memory <property name="offHeapMaxMemory" value="#{10 * 1024L * 1024L * 1024L}"/>, virtual memory is disenabled. I have a couple questions:

1. Does this mean that 10G off-heap memory is allocated for each cache? If I have two Ignite instances on a single hard node, 20G will be allocated? What if there is not enough memory (REM) ?
2. I am using CacheMetrics. getOffHeapAllocatedSize() to report the size of the off-heap memory used. Is the reported value for the single JVM, for the all the Ignite JMVs on the single hardware node or for the whole cluster composed of server hardware nodes?

Thanks

Shaomin

_________________________________________________________

This email, its contents, and any attachments transmitted with it are intended only for the addressee(s) and may be confidential and legally privileged. We do not waive any confidentiality by misdelivery. If you have received this email in error, please notify the sender immediately and delete it. You should not copy it, forward it or otherwise use the contents, attachments or information in any way. Any liability for viruses is excluded to the fullest extent permitted by law.

Tudor Capital Europe LLP (TCE) is authorised and regulated by The Financial Conduct Authority (the FCA). TCE is registered as a limited liability partnership in England and Wales No: OC340673 with its registered office at 10 New Burlington Street, London, W1S 3BE, United Kingdom

Re: Off-heap memory usage questions

Posted by Denis Magda <dm...@gridgain.com>.

HI Shaomin,

I would suggest making a memory snapshot using Visual VM or a similar tool and see which objects consumes so much memory.
Make sure to explicitly trigger garbage collection (GC) before making the snapshot because sometimes GC postpones its job depending on the settings.

If it’s still unclear for you who holds 6 GB please share the snapshot and ignite configuration you use with us.

—
Denis

> On Apr 15, 2016, at 10:49 AM, Shaomin Zhang <Sh...@tudor.com> wrote:
> 
> Thank you Val.
> 
> Another question is that although the reported off-heap memory is not big (around 72M), I observed big heap memory consumption - some as high as 6G. I am not using any expiry policy. Is this normal?
> 
> Shaomin
> 
> -----Original Message-----
> From: vkulichenko [mailto:valentin.kulichenko@gmail.com]
> Sent: 14 April 2016 19:08
> To: user@ignite.apache.org
> Subject: Re: Off-heap memory usage questions
> 
> Hi Shaomin,
> 
> 1. Yes, this is a per node setting. So if there are two nodes on one box, it's possible that 20G will be allocated on this box. You should make sure that this limit correlates with the amount of physical memory you have.
> 2. In 1.5. IgniteCache.metrics() return values for local node only, but you can use IgniteCache.metrics(ClusterGroup) to get aggregated values for a set of nodes. This is already fixed in master - in 1.6 aggregated metrics will be returned by default.
> 
> -Val
> 
> 
> 
> --
> View this message in context: https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Off-2Dheap-2Dmemory-2Dusage-2Dquestions-2Dtp4163p4185.html&d=CwICAg&c=lcVbikor4usg5Rj5OmznbA&r=TO3grc1lvgRzUij7SCtFhBPVz_ocKy44E1ncA3VjNmM&m=wyWdHvphkxlTm_p6Z8-GIgNpDLDXOEQsbe7FL1Z0_ys&s=BO6ZXcUxjJ59SEUkw6i8dWBDkpODn2sJz66r9g0Mzns&e=
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> _________________________________________________________
> 
> This email, its contents, and any attachments transmitted with it are intended only for the addressee(s) and may be confidential and legally privileged. We do not waive any confidentiality by misdelivery. If you have received this email in error, please notify the sender immediately and delete it. You should not copy it, forward it or otherwise use the contents, attachments or information in any way. Any liability for viruses is excluded to the fullest extent permitted by law.
> 
> Tudor Capital Europe LLP (TCE) is authorised and regulated by The Financial Conduct Authority (the FCA). TCE is registered as a limited liability partnership in England and Wales No: OC340673 with its registered office at 10 New Burlington Street, London, W1S 3BE, United Kingdom

RE: Off-heap memory usage questions

Posted by Shaomin Zhang <Sh...@tudor.com>.

Thank you Val.

Another question is that although the reported off-heap memory is not big (around 72M), I observed big heap memory consumption - some as high as 6G. I am not using any expiry policy. Is this normal?

Shaomin

-----Original Message-----
From: vkulichenko [mailto:valentin.kulichenko@gmail.com]
Sent: 14 April 2016 19:08
To: user@ignite.apache.org
Subject: Re: Off-heap memory usage questions

Hi Shaomin,

1. Yes, this is a per node setting. So if there are two nodes on one box, it's possible that 20G will be allocated on this box. You should make sure that this limit correlates with the amount of physical memory you have.
2. In 1.5. IgniteCache.metrics() return values for local node only, but you can use IgniteCache.metrics(ClusterGroup) to get aggregated values for a set of nodes. This is already fixed in master - in 1.6 aggregated metrics will be returned by default.

-Val

--
View this message in context: https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Off-2Dheap-2Dmemory-2Dusage-2Dquestions-2Dtp4163p4185.html&d=CwICAg&c=lcVbikor4usg5Rj5OmznbA&r=TO3grc1lvgRzUij7SCtFhBPVz_ocKy44E1ncA3VjNmM&m=wyWdHvphkxlTm_p6Z8-GIgNpDLDXOEQsbe7FL1Z0_ys&s=BO6ZXcUxjJ59SEUkw6i8dWBDkpODn2sJz66r9g0Mzns&e=
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
_________________________________________________________

This email, its contents, and any attachments transmitted with it are intended only for the addressee(s) and may be confidential and legally privileged. We do not waive any confidentiality by misdelivery. If you have received this email in error, please notify the sender immediately and delete it. You should not copy it, forward it or otherwise use the contents, attachments or information in any way. Any liability for viruses is excluded to the fullest extent permitted by law.

Tudor Capital Europe LLP (TCE) is authorised and regulated by The Financial Conduct Authority (the FCA). TCE is registered as a limited liability partnership in England and Wales No: OC340673 with its registered office at 10 New Burlington Street, London, W1S 3BE, United Kingdom

Re: Off-heap memory usage questions

Posted by vkulichenko <va...@gmail.com>.

Hi Shaomin,

1. Yes, this is a per node setting. So if there are two nodes on one box,
it's possible that 20G will be allocated on this box. You should make sure
that this limit correlates with the amount of physical memory you have.
2. In 1.5. IgniteCache.metrics() return values for local node only, but you
can use IgniteCache.metrics(ClusterGroup) to get aggregated values for a set
of nodes. This is already fixed in master - in 1.6 aggregated metrics will
be returned by default.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-heap-memory-usage-questions-tp4163p4185.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.