Re: OFFHEAP_TIERED and memory fragmentation

[Alexey Goncharuk <al...@gmail.com>]

Peter,

A page is a minimal entity that is allowed to be moved on OS level. If I
have a chunk of memory allocated in the middle of a page, OS won't be able
to move it during defragmentation (at least this is how things worked last
time I checked). In the new approach, we are able to move this chunk within
the page, this is the key difference compared to the old off-heap approach.

Note, that fine-grained per-entry locking does not imply a byte array per
entry. We will still have our old GridCacheMapEntry object, but it will be
created only during a cache read or update, and will be GC-ed immediately
after cache operation completes. A value itself, on the other hand, will be
stored off-heap. Your measurements confirm that pure off-heap cache
outperforms both options for on-heap caches in terms of GC, so I do not see
any problems here.

Thanks,
Alexey

2017-01-26 23:23 GMT+03:00 Peter Schmitt <pe...@gmail.com>:

> Hi Alexey,
>
> a modern OS won't give you a physical address. Instead you get virtual
> memory which allows the OS to support features like defragmentation.
> "fine-grained per-entry" sounds like a byte-array for every entry.
> We tested something like that and the result is again higher GC pauses.
> In our case a FGC with off-heap caches takes e.g. up to 1s.
> With the same data and using on-heap caches it takes up to 25s.
> With a byte-array per entry we get about 4s for a FGC.
> With some optimizations you might get down a bit, however, in view of the
> FGC time I fear that such an approach causes a significant impact for
> applications which need to keep huge caches.
>
> Kind regards,
> Peter
>
>
>
> 2017-01-26 15:57 GMT+01:00 Alexey Goncharuk <al...@gmail.com>:
>
>> Hi Peter,
>>
>> Leaving defragmentation to Ignite is one of the reasons we are trying
>> PageMemory approach. In Ignite 1.x we basically use OS memory allocator to
>> place a value off-heap. Once the OS has given us a pointer, the memory
>> cannot be moved around unless we free this region, thus the fragmentation.
>> On the other hand, with PageMemory approach we can define the page layout
>> ourselves and, given fine-grained per-entry locking, move values around in
>> memory. This makes memory management more complicated and usually defines
>> some restrictions on the data structures, but we expect this will give us
>> more control over the data.
>>
>> For example, one of the tricks we can do in the future is cluster-wide
>> cache snapshots: we simply dump the continuous region of memory on disk and
>> get a node-local caches snapshot. This is impossible to do using current
>> off-heap approach.
>>
>> --AG
>>
>> 2017-01-25 12:32 GMT+03:00 Peter Schmitt <pe...@gmail.com>:
>>
>>> Hi Val,
>>>
>>> fair enough :-)
>>> However, in the source-code it looks like "Flexible and precise
>>> per-cache memory limit" is supported already.
>>> At least each GridUnsafeMemory instance (in each GridCacheContext)
>>> receives the value from CacheConfiguration.getOffHeapMaxMemory).
>>> I haven't debugged it, but the instances are created per cache.
>>>
>>> Furthermore, I'm not sure about the memory-page concept, because you
>>> would get pages on top of pages (used by a modern OS).
>>> Sounds like even more fragmentation and in this case Ignite would need
>>> to do the defragmentation.
>>>
>>> Kind regards,
>>> Peter
>>>
>>>
>>>
>>> 2017-01-25 1:56 GMT+01:00 vkulichenko <va...@gmail.com>:
>>>
>>>> I don't think I will be able to explain better than it's done in the
>>>> ticket
>>>> :) Current approach is naive and not always effective for multiple
>>>> reasons.
>>>>
>>>> -Val
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-ignite-users.705
>>>> 18.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp1
>>>> 0218p10235.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>
>

Re: OFFHEAP_TIERED and memory fragmentation

[Peter Schmitt <pe...@gmail.com>]

Hi Alexey,

I see. That sounds reasonable on that level (and is something different).
What is your plan to limit the overhead/impact of detecting and moving
around such small chunks?

Kind regards,
Peter



2017-02-03 14:52 GMT+01:00 Alexey Goncharuk <al...@gmail.com>:

> Peter,
>
> A page is a minimal entity that is allowed to be moved on OS level. If I
> have a chunk of memory allocated in the middle of a page, OS won't be able
> to move it during defragmentation (at least this is how things worked last
> time I checked). In the new approach, we are able to move this chunk within
> the page, this is the key difference compared to the old off-heap approach.
>
> Note, that fine-grained per-entry locking does not imply a byte array per
> entry. We will still have our old GridCacheMapEntry object, but it will be
> created only during a cache read or update, and will be GC-ed immediately
> after cache operation completes. A value itself, on the other hand, will be
> stored off-heap. Your measurements confirm that pure off-heap cache
> outperforms both options for on-heap caches in terms of GC, so I do not see
> any problems here.
>
> Thanks,
> Alexey
>
> 2017-01-26 23:23 GMT+03:00 Peter Schmitt <pe...@gmail.com>:
>
>> Hi Alexey,
>>
>> a modern OS won't give you a physical address. Instead you get virtual
>> memory which allows the OS to support features like defragmentation.
>> "fine-grained per-entry" sounds like a byte-array for every entry.
>> We tested something like that and the result is again higher GC pauses.
>> In our case a FGC with off-heap caches takes e.g. up to 1s.
>> With the same data and using on-heap caches it takes up to 25s.
>> With a byte-array per entry we get about 4s for a FGC.
>> With some optimizations you might get down a bit, however, in view of the
>> FGC time I fear that such an approach causes a significant impact for
>> applications which need to keep huge caches.
>>
>> Kind regards,
>> Peter
>>
>>
>>
>> 2017-01-26 15:57 GMT+01:00 Alexey Goncharuk <al...@gmail.com>:
>>
>>> Hi Peter,
>>>
>>> Leaving defragmentation to Ignite is one of the reasons we are trying
>>> PageMemory approach. In Ignite 1.x we basically use OS memory allocator to
>>> place a value off-heap. Once the OS has given us a pointer, the memory
>>> cannot be moved around unless we free this region, thus the fragmentation.
>>> On the other hand, with PageMemory approach we can define the page layout
>>> ourselves and, given fine-grained per-entry locking, move values around in
>>> memory. This makes memory management more complicated and usually defines
>>> some restrictions on the data structures, but we expect this will give us
>>> more control over the data.
>>>
>>> For example, one of the tricks we can do in the future is cluster-wide
>>> cache snapshots: we simply dump the continuous region of memory on disk and
>>> get a node-local caches snapshot. This is impossible to do using current
>>> off-heap approach.
>>>
>>> --AG
>>>
>>> 2017-01-25 12:32 GMT+03:00 Peter Schmitt <pe...@gmail.com>:
>>>
>>>> Hi Val,
>>>>
>>>> fair enough :-)
>>>> However, in the source-code it looks like "Flexible and precise
>>>> per-cache memory limit" is supported already.
>>>> At least each GridUnsafeMemory instance (in each GridCacheContext)
>>>> receives the value from CacheConfiguration.getOffHeapMaxMemory).
>>>> I haven't debugged it, but the instances are created per cache.
>>>>
>>>> Furthermore, I'm not sure about the memory-page concept, because you
>>>> would get pages on top of pages (used by a modern OS).
>>>> Sounds like even more fragmentation and in this case Ignite would need
>>>> to do the defragmentation.
>>>>
>>>> Kind regards,
>>>> Peter
>>>>
>>>>
>>>>
>>>> 2017-01-25 1:56 GMT+01:00 vkulichenko <va...@gmail.com>:
>>>>
>>>>> I don't think I will be able to explain better than it's done in the
>>>>> ticket
>>>>> :) Current approach is naive and not always effective for multiple
>>>>> reasons.
>>>>>
>>>>> -Val
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context: http://apache-ignite-users.705
>>>>> 18.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp1
>>>>> 0218p10235.html
>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>>
>>>>
>>>>
>>>
>>
>