You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directmemory.apache.org by "Benoit Perroud (Created) (JIRA)" <ji...@apache.org> on 2012/01/01 16:09:30 UTC
[jira] [Created] (DIRECTMEMORY-53) MemoryMamagerService buffers
allocation policy
MemoryMamagerService buffers allocation policy
----------------------------------------------
Key: DIRECTMEMORY-53
URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
Project: Apache DirectMemory
Issue Type: Improvement
Reporter: Benoit Perroud
Priority: Minor
MemoryMamagerService with multiple buffers has a really simple, even buggy allocation policy : once a allocation fail (return null), we try the next buffer. If the allocation fails a second time, we return a failure (a null Pointer) to the caller.
If all the previous buffer were full at one point, we stick to the last buffer, without any chance to try to allocate into a previous different buffer (which could have again free space due to expiration or pointer freeing).
My idea here is to first correct this behavior, but then enhance the fonctionality by providing configurable buffer allocation policy. For example :
- round robin allocation, with a certain amount of retries (this would also relax some concurrency contention)
- allocate into the buffer with the more free space
- allocate into the less fragmented buffer
- random
- ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Created] (DIRECTMEMORY-53) MemoryMamagerService buffers
allocation policy
Posted by "Raffaele P. Guidi" <ra...@gmail.com>.
Right, that was supposed to be a placeholder for an allocation policy, not
a real one :) I agree with the "a little fix now and we'll rewrite from
scratch later" approach. Go ahead.
Thanks,
Raffaele
Il giorno 01/gen/2012 21:55, "Benoit Perroud" <be...@noisette.ch> ha
scritto:
> The eviction only runs when "inShortage" returns true, which means too
> many failures happened.
> But this needs to be run manually, there is no automatic way for
> MemoryManagerService to come back to the first buffer is there is more
> than 1 buffer.
>
> My view is more a cyclic list of buffers, where you can try to the
> first one after the last if failing.
>
> So instead of
> if ( activeBuffer.bufferNumber + 1 == buffers.size() ){ return null;}
>
> having something like activeBuffer = buffers.get(
> (activeBuffer.bufferNumber + 1) % buffers.size() );
>
> 2012/1/1 Raffaele P. Guidi <ra...@gmail.com>:
> > Uhm, I'm almost sure I put some mechanism to avoid sticking without any
> > free buffer... maybe it was into the eviction section, probably clearing
> > one buffer when too many nulls are returned
> >
> > On Sun, Jan 1, 2012 at 4:09 PM, Benoit Perroud (Created) (JIRA) <
> > jira@apache.org> wrote:
> >
> >> MemoryMamagerService buffers allocation policy
> >> ----------------------------------------------
> >>
> >> Key: DIRECTMEMORY-53
> >> URL:
> https://issues.apache.org/jira/browse/DIRECTMEMORY-53
> >> Project: Apache DirectMemory
> >> Issue Type: Improvement
> >> Reporter: Benoit Perroud
> >> Priority: Minor
> >>
> >>
> >> MemoryMamagerService with multiple buffers has a really simple, even
> buggy
> >> allocation policy : once a allocation fail (return null), we try the
> next
> >> buffer. If the allocation fails a second time, we return a failure (a
> null
> >> Pointer) to the caller.
> >> If all the previous buffer were full at one point, we stick to the last
> >> buffer, without any chance to try to allocate into a previous different
> >> buffer (which could have again free space due to expiration or pointer
> >> freeing).
> >>
> >> My idea here is to first correct this behavior, but then enhance the
> >> fonctionality by providing configurable buffer allocation policy. For
> >> example :
> >> - round robin allocation, with a certain amount of retries (this would
> >> also relax some concurrency contention)
> >> - allocate into the buffer with the more free space
> >> - allocate into the less fragmented buffer
> >> - random
> >> - ...
> >>
> >> --
> >> This message is automatically generated by JIRA.
> >> If you think it was sent incorrectly, please contact your JIRA
> >> administrators:
> >>
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> >> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
> >>
> >>
> >>
>
>
>
> --
> sent from my Nokia 3210
>
Re: [jira] [Created] (DIRECTMEMORY-53) MemoryMamagerService buffers
allocation policy
Posted by Benoit Perroud <be...@noisette.ch>.
The eviction only runs when "inShortage" returns true, which means too
many failures happened.
But this needs to be run manually, there is no automatic way for
MemoryManagerService to come back to the first buffer is there is more
than 1 buffer.
My view is more a cyclic list of buffers, where you can try to the
first one after the last if failing.
So instead of
if ( activeBuffer.bufferNumber + 1 == buffers.size() ){ return null;}
having something like activeBuffer = buffers.get(
(activeBuffer.bufferNumber + 1) % buffers.size() );
2012/1/1 Raffaele P. Guidi <ra...@gmail.com>:
> Uhm, I'm almost sure I put some mechanism to avoid sticking without any
> free buffer... maybe it was into the eviction section, probably clearing
> one buffer when too many nulls are returned
>
> On Sun, Jan 1, 2012 at 4:09 PM, Benoit Perroud (Created) (JIRA) <
> jira@apache.org> wrote:
>
>> MemoryMamagerService buffers allocation policy
>> ----------------------------------------------
>>
>> Key: DIRECTMEMORY-53
>> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
>> Project: Apache DirectMemory
>> Issue Type: Improvement
>> Reporter: Benoit Perroud
>> Priority: Minor
>>
>>
>> MemoryMamagerService with multiple buffers has a really simple, even buggy
>> allocation policy : once a allocation fail (return null), we try the next
>> buffer. If the allocation fails a second time, we return a failure (a null
>> Pointer) to the caller.
>> If all the previous buffer were full at one point, we stick to the last
>> buffer, without any chance to try to allocate into a previous different
>> buffer (which could have again free space due to expiration or pointer
>> freeing).
>>
>> My idea here is to first correct this behavior, but then enhance the
>> fonctionality by providing configurable buffer allocation policy. For
>> example :
>> - round robin allocation, with a certain amount of retries (this would
>> also relax some concurrency contention)
>> - allocate into the buffer with the more free space
>> - allocate into the less fragmented buffer
>> - random
>> - ...
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA
>> administrators:
>> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
--
sent from my Nokia 3210
Re: [jira] [Created] (DIRECTMEMORY-53) MemoryMamagerService buffers
allocation policy
Posted by "Raffaele P. Guidi" <ra...@gmail.com>.
Uhm, I'm almost sure I put some mechanism to avoid sticking without any
free buffer... maybe it was into the eviction section, probably clearing
one buffer when too many nulls are returned
On Sun, Jan 1, 2012 at 4:09 PM, Benoit Perroud (Created) (JIRA) <
jira@apache.org> wrote:
> MemoryMamagerService buffers allocation policy
> ----------------------------------------------
>
> Key: DIRECTMEMORY-53
> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
> Project: Apache DirectMemory
> Issue Type: Improvement
> Reporter: Benoit Perroud
> Priority: Minor
>
>
> MemoryMamagerService with multiple buffers has a really simple, even buggy
> allocation policy : once a allocation fail (return null), we try the next
> buffer. If the allocation fails a second time, we return a failure (a null
> Pointer) to the caller.
> If all the previous buffer were full at one point, we stick to the last
> buffer, without any chance to try to allocate into a previous different
> buffer (which could have again free space due to expiration or pointer
> freeing).
>
> My idea here is to first correct this behavior, but then enhance the
> fonctionality by providing configurable buffer allocation policy. For
> example :
> - round robin allocation, with a certain amount of retries (this would
> also relax some concurrency contention)
> - allocate into the buffer with the more free space
> - allocate into the less fragmented buffer
> - random
> - ...
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
[jira] [Updated] (DIRECTMEMORY-53) MemoryMamagerService buffers
allocation policy
Posted by "Benoit Perroud (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DIRECTMEMORY-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Perroud updated DIRECTMEMORY-53:
---------------------------------------
Attachment: RoundRobinAllocationPolicy.java
MemoryManagerServiceWithAllocationPolicyImpl.java
AllocationPolicy.java
PoC of allocation policy.
- Interface allocationPolicy
- Default Round Robin implementation : an internal counter is incremented at every request, resulting in returning the next buffer at every call
- A MemoryManagerService implementation with allocation policy usage : loop calling allocationPolicy.activeBuffer, trying to store the data, repeating till activeBuffer is null
> MemoryMamagerService buffers allocation policy
> ----------------------------------------------
>
> Key: DIRECTMEMORY-53
> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
> Project: Apache DirectMemory
> Issue Type: Improvement
> Reporter: Benoit Perroud
> Priority: Minor
> Attachments: AllocationPolicy.java, MemoryManagerServiceWithAllocationPolicyImpl.java, RoundRobinAllocationPolicy.java
>
>
> MemoryMamagerService with multiple buffers has a really simple, even buggy allocation policy : once a allocation fail (return null), we try the next buffer. If the allocation fails a second time, we return a failure (a null Pointer) to the caller.
> If all the previous buffer were full at one point, we stick to the last buffer, without any chance to try to allocate into a previous different buffer (which could have again free space due to expiration or pointer freeing).
> My idea here is to first correct this behavior, but then enhance the fonctionality by providing configurable buffer allocation policy. For example :
> - round robin allocation, with a certain amount of retries (this would also relax some concurrency contention)
> - allocate into the buffer with the more free space
> - allocate into the less fragmented buffer
> - random
> - ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (DIRECTMEMORY-53) MemoryManagerService buffers
allocation policy
Posted by "Benoit Perroud (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DIRECTMEMORY-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Perroud resolved DIRECTMEMORY-53.
----------------------------------------
Resolution: Fixed
Fix Version/s: 0.6.0
Assignee: Benoit Perroud
> MemoryManagerService buffers allocation policy
> ----------------------------------------------
>
> Key: DIRECTMEMORY-53
> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
> Project: Apache DirectMemory
> Issue Type: Improvement
> Reporter: Benoit Perroud
> Assignee: Benoit Perroud
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: AllocationPolicy.java, MemoryManagerServiceWithAllocationPolicyImpl.java, RoundRobinAllocationPolicy.java
>
>
> MemoryMamagerService with multiple buffers has a really simple, even buggy allocation policy : once a allocation fail (return null), we try the next buffer. If the allocation fails a second time, we return a failure (a null Pointer) to the caller.
> If all the previous buffer were full at one point, we stick to the last buffer, without any chance to try to allocate into a previous different buffer (which could have again free space due to expiration or pointer freeing).
> My idea here is to first correct this behavior, but then enhance the fonctionality by providing configurable buffer allocation policy. For example :
> - round robin allocation, with a certain amount of retries (this would also relax some concurrency contention)
> - allocate into the buffer with the more free space
> - allocate into the less fragmented buffer
> - random
> - ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DIRECTMEMORY-53) MemoryManagerService buffers
allocation policy
Posted by "Benoit Perroud (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DIRECTMEMORY-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Perroud updated DIRECTMEMORY-53:
---------------------------------------
Summary: MemoryManagerService buffers allocation policy (was: MemoryMamagerService buffers allocation policy)
> MemoryManagerService buffers allocation policy
> ----------------------------------------------
>
> Key: DIRECTMEMORY-53
> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-53
> Project: Apache DirectMemory
> Issue Type: Improvement
> Reporter: Benoit Perroud
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: AllocationPolicy.java, MemoryManagerServiceWithAllocationPolicyImpl.java, RoundRobinAllocationPolicy.java
>
>
> MemoryMamagerService with multiple buffers has a really simple, even buggy allocation policy : once a allocation fail (return null), we try the next buffer. If the allocation fails a second time, we return a failure (a null Pointer) to the caller.
> If all the previous buffer were full at one point, we stick to the last buffer, without any chance to try to allocate into a previous different buffer (which could have again free space due to expiration or pointer freeing).
> My idea here is to first correct this behavior, but then enhance the fonctionality by providing configurable buffer allocation policy. For example :
> - round robin allocation, with a certain amount of retries (this would also relax some concurrency contention)
> - allocate into the buffer with the more free space
> - allocate into the less fragmented buffer
> - random
> - ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira