Simplified interface and non fragmentation work : Slab allocator and ByteBuffers

[Benoit Perroud <be...@noisette.ch>]

Hi All,

I worked on a slab allocation at some point, and while speaking about
separation of concerns, I tried to put both together.

Slab allocation basically means that the memory manager will point to
several slab, each one split in advance in slices of same size, let
say slab1 is composed of 1024 slices of 128 bytes, slab2 256 slices of
512 bytes and slab3 64 slices of 2048 bytes. When I try to store a new
byte array, I will search for the smallest slab that can fit the size
of the byte array, and store the content on this slice.

This is deadly simple to implement, fast at usage but the tradeoff is
some memory waste (if your size is not well sized for your business
you will have lot of unused bytes in slices).

Starting to implement the slab OffHeapMemoryBuffer, I start to find
overkill to have a Pointer object every time. Why not having a
MemoryManager layer that only deal with ByteBuffer. No expiration, LRU
and so on. This would be done by the Caching layer, where storing a
value will serialize it and store this alltogether in appropriate
class.

Please give a chance to the attached patch that implement the
simplified MemoryManager layer, with Builder pattern style.

Thanks,

Benoit.

Re: Simplified interface and non fragmentation work : Slab allocator and ByteBuffers

[Benoit Perroud <be...@noisette.ch>]

Thanks for your answer.

Yes we already discussed about this.

I will open a Jira and link it to all related existing jiras, I just
find more convenient to discuss per email :)

Cheers,

Benoit.



2012/1/18 Raffaele P. Guidi <ra...@gmail.com>:
> I totally agree, I thought we already discussed this and maybe opened a
> jira about it?
>
> Well done, anyway.
>
> Ciao,
>     R
> Il giorno 18/gen/2012 15:20, "Benoit Perroud" <be...@noisette.ch> ha
> scritto:
>
>> Hi All,
>>
>> I worked on a slab allocation at some point, and while speaking about
>> separation of concerns, I tried to put both together.
>>
>> Slab allocation basically means that the memory manager will point to
>> several slab, each one split in advance in slices of same size, let
>> say slab1 is composed of 1024 slices of 128 bytes, slab2 256 slices of
>> 512 bytes and slab3 64 slices of 2048 bytes. When I try to store a new
>> byte array, I will search for the smallest slab that can fit the size
>> of the byte array, and store the content on this slice.
>>
>> This is deadly simple to implement, fast at usage but the tradeoff is
>> some memory waste (if your size is not well sized for your business
>> you will have lot of unused bytes in slices).
>>
>> Starting to implement the slab OffHeapMemoryBuffer, I start to find
>> overkill to have a Pointer object every time. Why not having a
>> MemoryManager layer that only deal with ByteBuffer. No expiration, LRU
>> and so on. This would be done by the Caching layer, where storing a
>> value will serialize it and store this alltogether in appropriate
>> class.
>>
>> Please give a chance to the attached patch that implement the
>> simplified MemoryManager layer, with Builder pattern style.
>>
>> Thanks,
>>
>> Benoit.
>>

Re: Simplified interface and non fragmentation work : Slab allocator and ByteBuffers

["Raffaele P. Guidi" <ra...@gmail.com>]

I totally agree, I thought we already discussed this and maybe opened a
jira about it?

Well done, anyway.

Ciao,
     R
Il giorno 18/gen/2012 15:20, "Benoit Perroud" <be...@noisette.ch> ha
scritto:

> Hi All,
>
> I worked on a slab allocation at some point, and while speaking about
> separation of concerns, I tried to put both together.
>
> Slab allocation basically means that the memory manager will point to
> several slab, each one split in advance in slices of same size, let
> say slab1 is composed of 1024 slices of 128 bytes, slab2 256 slices of
> 512 bytes and slab3 64 slices of 2048 bytes. When I try to store a new
> byte array, I will search for the smallest slab that can fit the size
> of the byte array, and store the content on this slice.
>
> This is deadly simple to implement, fast at usage but the tradeoff is
> some memory waste (if your size is not well sized for your business
> you will have lot of unused bytes in slices).
>
> Starting to implement the slab OffHeapMemoryBuffer, I start to find
> overkill to have a Pointer object every time. Why not having a
> MemoryManager layer that only deal with ByteBuffer. No expiration, LRU
> and so on. This would be done by the Caching layer, where storing a
> value will serialize it and store this alltogether in appropriate
> class.
>
> Please give a chance to the attached patch that implement the
> simplified MemoryManager layer, with Builder pattern style.
>
> Thanks,
>
> Benoit.
>