[Mavibot] ApacheCon discussions

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys,

we had some interesting discussion with Kiran and Shawn during the
Apache Conference this week. We spent some time on Mavibot and here are
a few thoughts we had about it :

- We should not have a support of multiple^values in Mavibot. This makes
the code too complex to handle, and this could be done at a upper level
(like what we have for JDBM). At the end, Mavibot should only be a
strict Key/Value store
- We don't need to have two flavor of BTrees (Persisted and InMemory).
This is the Manager role : we can have a PersistedManager and a
InMemoryManager
- The transaction support has been lenghtly discussed, and we think we
have a valid specification for it :
    - hat we need to explicitely start and terminate a transaction,
instead of have an implicit start (as we currently have)
    - we need a WorkingMemory to hold modified pages
    - as we are going to modify pages instead of copying them, we need a
different implementation fo the very basic B-tree insert/remove
operations for this case
    - revisions will be cross-btrees, and actually will reflect the
transaction manager revision, so we may have holes in teh numbering for
a goven btree
    - managing versions will be done using a list with no lock
- Cleaning up the pages can be done at teh end of an update, by what we
now call the Recycler. We can clean old revisions, starting by the
oldest and up the list, until we meet a revision which is in use. We can
also cleanup not use revisions, at least for the pages that have been
copied in the revision immediately upper.
- Last, not least, we discussed about the fact that we may have two
threads to manage updates and recycling : the contention will be minimal
and teh gain can be hudge, as we won't anymore have to process with the
cleaning after the update is completed.

I'll come with some more complete explaination and document on those
specification when back home.

I have created a branch to paly with those ideas :
mavibot/branches/single-value. This branch is where we are going to
experiment.

Thanks !

Re: [Mavibot] ApacheCon discussions

[Kiran Ayyagari <ka...@apache.org>]

On Sat, Oct 3, 2015 at 2:17 PM, Emmanuel Lécharny <el...@gmail.com>
wrote:

> Hi guys,
>
> we had some interesting discussion with Kiran and Shawn during the
> Apache Conference this week. We spent some time on Mavibot and here are
> a few thoughts we had about it :
>
> - We should not have a support of multiple^values in Mavibot. This makes
> the code too complex to handle, and this could be done at a upper level
> (like what we have for JDBM). At the end, Mavibot should only be a
> strict Key/Value store
> - We don't need to have two flavor of BTrees (Persisted and InMemory).
> This is the Manager role : we can have a PersistedManager and a
> InMemoryManager
> - The transaction support has been lenghtly discussed, and we think we
> have a valid specification for it :
>     - hat we need to explicitely start and terminate a transaction,
> instead of have an implicit start (as we currently have)
>     - we need a WorkingMemory to hold modified pages
>     - as we are going to modify pages instead of copying them, we need a
> different implementation fo the very basic B-tree insert/remove
> operations for this case
>     - revisions will be cross-btrees, and actually will reflect the
> transaction manager revision, so we may have holes in teh numbering for
> a goven btree
>     - managing versions will be done using a list with no lock
> - Cleaning up the pages can be done at teh end of an update, by what we
> now call the Recycler. We can clean old revisions, starting by the
> oldest and up the list, until we meet a revision which is in use. We can
> also cleanup not use revisions, at least for the pages that have been
> copied in the revision immediately upper.
> - Last, not least, we discussed about the fact that we may have two
> threads to manage updates and recycling : the contention will be minimal
> and teh gain can be hudge, as we won't anymore have to process with the
> cleaning after the update is completed.
>
> I'll come with some more complete explaination and document on those
> specification when back home.
>
> I have created a branch to paly with those ideas :
> mavibot/branches/single-value. This branch is where we are going to
> experiment.
>
> Thanks !
>
that is a perfect summary of all things we discussed

thanks Emmanuel



-- 
Kiran Ayyagari
http://keydap.com