You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Megha Mittal <me...@gmail.com> on 2017/06/09 05:49:21 UTC
Off-Heap On-Heap in Ignite-2.0.0
Hi, I am new to * apache ignite-2.0.0 * and would like to know more on how
ignite works on off-heap and on-heap concept.
Also I have few doubts
1. if we have 5 GB of data so is it possible that all of it resides on-heap
with no objects in off-heap. i.e we allocate 5GB to on heap and 0 to off
heap.
2. Also I would like to know do I have to allocate atleast 5 GB to off-heap
so that all objects are able to reside there . Also if i allocate suppose 1
GB to on-heap with maxEvictionSize as 100000 records then does it imply that
these 100000 records will be replicated on on-heap (i.e 5 GB on off-heap is
a definite requirement) or they will be moved to on-heap with no existance
on off-heap(i.e objects are partitioned between on-heap and off-heap) and
when the eviction occurs they will be moved to off-heap memory.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Megha Mittal <me...@gmail.com>.
Denis,
Thanks for clearing it out. That might be the reason for the memory
difference.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548p13683.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Denis Magda <dm...@apache.org>.
Megha,
The objects are stored in the deserialized form both in heap (AI 1.x) and off-heap (AI 2.0). The difference is that when an object is in Java heap we need to an create extra wrapper object around it so that it can be used by an application running on top of JVM. Plus there might be some extra costs having data in Java heap.
—
Denis
> On Jun 12, 2017, at 10:20 PM, Megha Mittal <me...@gmail.com> wrote:
>
> Hi Denis, thanks for answering. That completely cleared out my doubt.
>
> I would like to know one more thing. I was testing with ~10000000 (10
> million) records with each records of ~1400 bytes. When I was trying to
> keep everything on-heap I had to allocate around 15GB to on-heap to store
> all these records. But when I switched off on-heap and kept everything
> off-heap, same number of records got fit in 10 GB off heap memory region.
> Does it imply that when ignite saves in on-heap it's in deserialized form
> hence exact memory(no. of records * size of one record) is required. But
> when it keeps in off-heap it is stored in serialized form hence less memory
> than expected is required. Or is there some other reason for this memory
> requirement difference.
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548p13637.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Megha Mittal <me...@gmail.com>.
Hi Denis, thanks for answering. That completely cleared out my doubt.
I would like to know one more thing. I was testing with ~10000000 (10
million) records with each records of ~1400 bytes. When I was trying to
keep everything on-heap I had to allocate around 15GB to on-heap to store
all these records. But when I switched off on-heap and kept everything
off-heap, same number of records got fit in 10 GB off heap memory region.
Does it imply that when ignite saves in on-heap it's in deserialized form
hence exact memory(no. of records * size of one record) is required. But
when it keeps in off-heap it is stored in serialized form hence less memory
than expected is required. Or is there some other reason for this memory
requirement difference.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548p13637.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Denis Magda <dm...@apache.org>.
Megha,
> Or is it required to give 1 GB to off heap
> memory region with no relation how much I allocate to on-heap.
That’s the correct point.
All the data is stored in off-heap by default. If you give 500 MB to the off-heap then you’ll get out of memory exception once you go beyond that boundary. Java heap is no longer treated as a data storage and might be used as an extra caching layer for entries you have in the off-heap.
—
Denis
> On Jun 12, 2017, at 1:03 AM, Megha Mittal <me...@gmail.com> wrote:
>
> Hi Denis,
>
> Thanks for your reply. The video was helpful. But I am still not clear on
> the fact that if I have 1 GB data, with onHeapCaching enabled if I give my
> off heap memory region a maximum size of 500 MB and 1 GB to my heap, then
> will this configuration work. Or is it required to give 1 GB to off heap
> memory region with no relation how much I allocate to on-heap.
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548p13617.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Megha Mittal <me...@gmail.com>.
Hi Denis,
Thanks for your reply. The video was helpful. But I am still not clear on
the fact that if I have 1 GB data, with onHeapCaching enabled if I give my
off heap memory region a maximum size of 500 MB and 1 GB to my heap, then
will this configuration work. Or is it required to give 1 GB to off heap
memory region with no relation how much I allocate to on-heap.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548p13617.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Off-Heap On-Heap in Ignite-2.0.0
Posted by Denis Magda <dm...@apache.org>.
Hi,
That documentation should answer all of your questions:
https://apacheignite.readme.io/docs/page-memory <https://apacheignite.readme.io/docs/page-memory>
Plus you might need to watch this webinar to get even more insights:
Apache Ignite 2.0 : Prelude to a Distributed SQL Database <https://www.youtube.com/watch?v=bDrGueQ16UQ>
—
Denis
> On Jun 8, 2017, at 10:49 PM, Megha Mittal <me...@gmail.com> wrote:
>
> Hi, I am new to * apache ignite-2.0.0 * and would like to know more on how
> ignite works on off-heap and on-heap concept.
>
> Also I have few doubts
>
> 1. if we have 5 GB of data so is it possible that all of it resides on-heap
> with no objects in off-heap. i.e we allocate 5GB to on heap and 0 to off
> heap.
>
> 2. Also I would like to know do I have to allocate atleast 5 GB to off-heap
> so that all objects are able to reside there . Also if i allocate suppose 1
> GB to on-heap with maxEvictionSize as 100000 records then does it imply that
> these 100000 records will be replicated on on-heap (i.e 5 GB on off-heap is
> a definite requirement) or they will be moved to on-heap with no existance
> on off-heap(i.e objects are partitioned between on-heap and off-heap) and
> when the eviction occurs they will be moved to off-heap memory.
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-On-Heap-in-Ignite-2-0-0-tp13548.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.