You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@directmemory.apache.org by Benoit Perroud <be...@noisette.ch> on 2011/12/27 15:37:58 UTC

Remarks on OffHeapMemoryBuffer

Hi,

I played a little bit with DM while waiting for Christmas father :) I
started to write tests on MemoryManagerService, and I have the
followings remarks :

- OffHeapMemoryBuffer shouldn't throw a NPE when store function don't
find "goodOne" pointer. Should return null instead to allow
MemoryManagerService to try another buffer if available. See
testAllocateMultipleBuffers function in the attached class :
initializing 4 buffers of 4 bytes, I should be able to store 4 objects
of 4 bytes each.

- OffHeapMemoryBuffer leaks one byte at every allocation. This is
because the pointer is 0 indexed, so a pointer starting at 0 and
ending at 3 is 4 bytes, not 3. See testByteLeaking function  :
initializing 1 buffer of 40 bytes, I should be able to store 10
objects of 4 bytes each. I would also suggest to have a function
Pointer.getSize() { return end - start + 1; }

- OffHeapMemoryBuffer.firstMatch should compare pointer capacity to
find a match and not only the ending offset. Doing the correct
comparison will avoid a costly BufferOverflowException.

- OffHeapMemoryBuffer don't report the correct used size (once the
leak is removed) when freeing a pointer. See
testReportCorrectUsedMemory : allocating the whole buffer, freeing a
pointer, reallocating this pointer. The resulting usage should again
be the full buffer.

Please see the attached patch on OffHeapMemoryBuffer that correct
those problems. Also tests are corrected. Pointer.getSize is
definitely a good idea :)

Some other remarks : once a buffer is splitted in a lot of small
pointers, there is no way to merge two adjacent pointers together to
make a bigger that could be reallocated later. Is it a missing feature
that worth opening a Jira ?

Best,

Benoit.

Re: Remarks on OffHeapMemoryBuffer

Posted by Benoit Perroud <be...@noisette.ch>.
Hi,

Thanks for your answer.

https://issues.apache.org/jira/browse/DIRECTMEMORY-40

Benoit.


2011/12/27 Raffaele P. Guidi <ra...@gmail.com>:
> Great job - and yes it is well worth opening the issue
>
> Il giorno 27/dic/2011 15:38, "Benoit Perroud" <be...@noisette.ch> ha
> scritto:
>
>> Hi,
>>
>> I played a little bit with DM while waiting for Christmas father :) I
>> started to write tests on MemoryManagerService, and I have the
>> followings remarks :
>>
>> - OffHeapMemoryBuffer shouldn't throw a NPE when store function don't
>> find "goodOne" pointer. Should return null instead to allow
>> MemoryManagerService to try another buffer if available. See
>> testAllocateMultipleBuffers function in the attached class :
>> initializing 4 buffers of 4 bytes, I should be able to store 4 objects
>> of 4 bytes each.
>>
>> - OffHeapMemoryBuffer leaks one byte at every allocation. This is
>> because the pointer is 0 indexed, so a pointer starting at 0 and
>> ending at 3 is 4 bytes, not 3. See testByteLeaking function  :
>> initializing 1 buffer of 40 bytes, I should be able to store 10
>> objects of 4 bytes each. I would also suggest to have a function
>> Pointer.getSize() { return end - start + 1; }
>>
>> - OffHeapMemoryBuffer.firstMatch should compare pointer capacity to
>> find a match and not only the ending offset. Doing the correct
>> comparison will avoid a costly BufferOverflowException.
>>
>> - OffHeapMemoryBuffer don't report the correct used size (once the
>> leak is removed) when freeing a pointer. See
>> testReportCorrectUsedMemory : allocating the whole buffer, freeing a
>> pointer, reallocating this pointer. The resulting usage should again
>> be the full buffer.
>>
>> Please see the attached patch on OffHeapMemoryBuffer that correct
>> those problems. Also tests are corrected. Pointer.getSize is
>> definitely a good idea :)
>>
>> Some other remarks : once a buffer is splitted in a lot of small
>> pointers, there is no way to merge two adjacent pointers together to
>> make a bigger that could be reallocated later. Is it a missing feature
>> that worth opening a Jira ?
>>
>> Best,
>>
>> Benoit.



-- 
sent from my Nokia 3210

Re: Remarks on OffHeapMemoryBuffer

Posted by "Raffaele P. Guidi" <ra...@gmail.com>.
Great job - and yes it is well worth opening the issue
Il giorno 27/dic/2011 15:38, "Benoit Perroud" <be...@noisette.ch> ha
scritto:

> Hi,
>
> I played a little bit with DM while waiting for Christmas father :) I
> started to write tests on MemoryManagerService, and I have the
> followings remarks :
>
> - OffHeapMemoryBuffer shouldn't throw a NPE when store function don't
> find "goodOne" pointer. Should return null instead to allow
> MemoryManagerService to try another buffer if available. See
> testAllocateMultipleBuffers function in the attached class :
> initializing 4 buffers of 4 bytes, I should be able to store 4 objects
> of 4 bytes each.
>
> - OffHeapMemoryBuffer leaks one byte at every allocation. This is
> because the pointer is 0 indexed, so a pointer starting at 0 and
> ending at 3 is 4 bytes, not 3. See testByteLeaking function  :
> initializing 1 buffer of 40 bytes, I should be able to store 10
> objects of 4 bytes each. I would also suggest to have a function
> Pointer.getSize() { return end - start + 1; }
>
> - OffHeapMemoryBuffer.firstMatch should compare pointer capacity to
> find a match and not only the ending offset. Doing the correct
> comparison will avoid a costly BufferOverflowException.
>
> - OffHeapMemoryBuffer don't report the correct used size (once the
> leak is removed) when freeing a pointer. See
> testReportCorrectUsedMemory : allocating the whole buffer, freeing a
> pointer, reallocating this pointer. The resulting usage should again
> be the full buffer.
>
> Please see the attached patch on OffHeapMemoryBuffer that correct
> those problems. Also tests are corrected. Pointer.getSize is
> definitely a good idea :)
>
> Some other remarks : once a buffer is splitted in a lot of small
> pointers, there is no way to merge two adjacent pointers together to
> make a bigger that could be reallocated later. Is it a missing feature
> that worth opening a Jira ?
>
> Best,
>
> Benoit.
>