You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directmemory.apache.org by Benoit Perroud <be...@noisette.ch> on 2012/03/02 08:15:03 UTC
DIRECTMEMORY-9
Hi All,
Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
wonder if DIRECTMEMORY-9 is still relevant or if it could also be
closed.
Thanks,
Benoit.
Re: DIRECTMEMORY-9
Posted by Benoit Perroud <be...@noisette.ch>.
Right, thanks.
2012/3/2 Michael André Pearce <mi...@me.com>:
> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>
>
> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>
>> Hi All,
>>
>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>> closed.
>>
>> Thanks,
>>
>> Benoit.
>
--
sent from my Nokia 3210
Re: DIRECTMEMORY-9
Posted by Michael André Pearce <mi...@me.com>.
Hi Benoit,
Though i should explain why i dont think issue 9 can be closed yet.
Conceived reason why it still is needed in my mind.
So buffer space is as such after loading it with some data
Pointer Size State
1 1m full
2 2m full
3 1m full
4 2m full
5 1m full
6 2m full
7 1m full
I then free, pointers, 1,3,5 and 7.
Pointer Size State
1 1m free
2 2m full
3 1m free
4 2m full
5 1m free
6 2m full
7 1m free
I then want to put 2m in the cache. I cant but there is 4m actually avail, needs defrag.
So atm im -1 for closing this story, i think the merging memory is a great idea, as it gives a quick win, without a more expensive defrag, but still think a defrag routine or algo is needed for the above.
On 2 Mar 2012, at 07:22, Michael André Pearce wrote:
> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>
>
> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>
>> Hi All,
>>
>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>> closed.
>>
>> Thanks,
>>
>> Benoit.
>
Re: DIRECTMEMORY-9
Posted by Michael André Pearce <mi...@me.com>.
Forgot to add, best place to start looking is DirectMemoryMapTest and DirectMemoryMap, its just some simple wrappers of concurrentmap interface, as its good to test and compare against something well known, especially checking things like concurrency.
On 2 Mar 2012, at 08:37, Michael André Pearce wrote:
> Ive just added some sandbox ideas in a zip ive been playing with this week locally to directmemory-9, its not meant for inclusion, especially as its on the old memory buffer, and i agree that if we have something better we should just remove the old. But from it i want to just share some ideas and solutions ive found which could be migrated into the real project, whilst i would look to do this myself, i know i dont have time this weekend or next due to work commitments, as such just wanted to share with the team.
>
> I will pick back up probably in 2 weeks, and look to maybe implement some of the solutions on the new buffers/latest code myself if it hasn't been, and propose inclusion, but if someone else picks up i wont be upset ;-)
>
>
>
>
> On 2 Mar 2012, at 08:12, Simone Tripodi wrote:
>
>> Great report Michael, thanks!
>> -Simo
>>
>> http://people.apache.org/~simonetripodi/
>> http://simonetripodi.livejournal.com/
>> http://twitter.com/simonetripodi
>> http://www.99soft.org/
>>
>>
>>
>> On Fri, Mar 2, 2012 at 8:58 AM, Michael André Pearce
>> <mi...@me.com> wrote:
>>> Hi Benoit,
>>>
>>> Though i should explain why i dont think issue 9 can be closed yet.
>>>
>>> Conceived reason why it still is needed in my mind.
>>>
>>> So buffer space is as such after loading it with some data
>>> Pointer Size State
>>> 1 1m full
>>> 2 2m full
>>> 3 1m full
>>> 4 2m full
>>> 5 1m full
>>> 6 2m full
>>> 7 1m full
>>>
>>> I then free, pointers, 1,3,5 and 7.
>>> Pointer Size State
>>> 1 1m free
>>> 2 2m full
>>> 3 1m free
>>> 4 2m full
>>> 5 1m free
>>> 6 2m full
>>> 7 1m free
>>>
>>> I then want to put 2m in the cache. I cant but there is 4m actually avail, needs defrag.
>>>
>>> So atm im -1 for closing this story, i think the merging memory is a great idea, as it gives a quick win, without a more expensive defrag, but still think a defrag routine or algo is needed for the above.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 2 Mar 2012, at 07:22, Michael André Pearce wrote:
>>>
>>>> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>>>>
>>>>
>>>> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>>>>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>>>>> closed.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Benoit.
>>>>
>>>
>
Re: DIRECTMEMORY-9
Posted by Michael André Pearce <mi...@me.com>.
Ive just added some sandbox ideas in a zip ive been playing with this week locally to directmemory-9, its not meant for inclusion, especially as its on the old memory buffer, and i agree that if we have something better we should just remove the old. But from it i want to just share some ideas and solutions ive found which could be migrated into the real project, whilst i would look to do this myself, i know i dont have time this weekend or next due to work commitments, as such just wanted to share with the team.
I will pick back up probably in 2 weeks, and look to maybe implement some of the solutions on the new buffers/latest code myself if it hasn't been, and propose inclusion, but if someone else picks up i wont be upset ;-)
On 2 Mar 2012, at 08:12, Simone Tripodi wrote:
> Great report Michael, thanks!
> -Simo
>
> http://people.apache.org/~simonetripodi/
> http://simonetripodi.livejournal.com/
> http://twitter.com/simonetripodi
> http://www.99soft.org/
>
>
>
> On Fri, Mar 2, 2012 at 8:58 AM, Michael André Pearce
> <mi...@me.com> wrote:
>> Hi Benoit,
>>
>> Though i should explain why i dont think issue 9 can be closed yet.
>>
>> Conceived reason why it still is needed in my mind.
>>
>> So buffer space is as such after loading it with some data
>> Pointer Size State
>> 1 1m full
>> 2 2m full
>> 3 1m full
>> 4 2m full
>> 5 1m full
>> 6 2m full
>> 7 1m full
>>
>> I then free, pointers, 1,3,5 and 7.
>> Pointer Size State
>> 1 1m free
>> 2 2m full
>> 3 1m free
>> 4 2m full
>> 5 1m free
>> 6 2m full
>> 7 1m free
>>
>> I then want to put 2m in the cache. I cant but there is 4m actually avail, needs defrag.
>>
>> So atm im -1 for closing this story, i think the merging memory is a great idea, as it gives a quick win, without a more expensive defrag, but still think a defrag routine or algo is needed for the above.
>>
>>
>>
>>
>>
>>
>> On 2 Mar 2012, at 07:22, Michael André Pearce wrote:
>>
>>> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>>>
>>>
>>> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>>>
>>>> Hi All,
>>>>
>>>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>>>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>>>> closed.
>>>>
>>>> Thanks,
>>>>
>>>> Benoit.
>>>
>>
Re: DIRECTMEMORY-9
Posted by Simone Tripodi <si...@apache.org>.
Great report Michael, thanks!
-Simo
http://people.apache.org/~simonetripodi/
http://simonetripodi.livejournal.com/
http://twitter.com/simonetripodi
http://www.99soft.org/
On Fri, Mar 2, 2012 at 8:58 AM, Michael André Pearce
<mi...@me.com> wrote:
> Hi Benoit,
>
> Though i should explain why i dont think issue 9 can be closed yet.
>
> Conceived reason why it still is needed in my mind.
>
> So buffer space is as such after loading it with some data
> Pointer Size State
> 1 1m full
> 2 2m full
> 3 1m full
> 4 2m full
> 5 1m full
> 6 2m full
> 7 1m full
>
> I then free, pointers, 1,3,5 and 7.
> Pointer Size State
> 1 1m free
> 2 2m full
> 3 1m free
> 4 2m full
> 5 1m free
> 6 2m full
> 7 1m free
>
> I then want to put 2m in the cache. I cant but there is 4m actually avail, needs defrag.
>
> So atm im -1 for closing this story, i think the merging memory is a great idea, as it gives a quick win, without a more expensive defrag, but still think a defrag routine or algo is needed for the above.
>
>
>
>
>
>
> On 2 Mar 2012, at 07:22, Michael André Pearce wrote:
>
>> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>>
>>
>> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>>
>>> Hi All,
>>>
>>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>>> closed.
>>>
>>> Thanks,
>>>
>>> Benoit.
>>
>
Re: DIRECTMEMORY-9
Posted by Michael André Pearce <mi...@me.com>.
Hi Benoit,
Though i should explain why i dont think issue 9 can be closed yet.
Conceived reason why it still is needed in my mind.
So buffer space is as such after loading it with some data
Pointer Size State
1 1m full
2 2m full
3 1m full
4 2m full
5 1m full
6 2m full
7 1m full
I then free, pointers, 1,3,5 and 7.
Pointer Size State
1 1m free
2 2m full
3 1m free
4 2m full
5 1m free
6 2m full
7 1m free
I then want to put 2m in the cache. I cant but there is 4m actually avail, needs defrag.
So atm im -1 for closing this story, i think the merging memory is a great idea, as it gives a quick win, without a more expensive defrag, but still think a defrag routine or algo is needed for the above.
On 2 Mar 2012, at 07:22, Michael André Pearce wrote:
> I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
>
>
> On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
>
>> Hi All,
>>
>> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
>> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
>> closed.
>>
>> Thanks,
>>
>> Benoit.
>
Re: DIRECTMEMORY-9
Posted by Michael André Pearce <mi...@me.com>.
I think you may want some defragmentation still, especially if the buffers a fair % full and the free pointers are spread, would mean that if any larger object that the free pointers arent large enough for, but in total could hold, without defrag would mean wouldn't be able to store.
On 2 Mar 2012, at 07:15, Benoit Perroud wrote:
> Hi All,
>
> Now with DIRECTMEMORY-40 done and a new slab's style allocator, I
> wonder if DIRECTMEMORY-9 is still relevant or if it could also be
> closed.
>
> Thanks,
>
> Benoit.